Possible Loss of Fraction

后端 未结 5 1589
慢半拍i
慢半拍i 2020-12-02 14:12

Forgive me if this is a naïve question, however I am at a loss today.

I have a simple division calculation such as follows:

double returnValue = (myO         


        
相关标签:
5条回答
  • 2020-12-02 14:23

    Assuming that myObject.Value is an int, the equation myObject.Value / 10 will be an integer division which will then be cast to a double.

    That means that myObject.Value being 12 will result in returnValue becoming 1, not 1.2.

    You need to cast the value(s) first:

    double returnValue = (double)(myObject.Value) / 10.0;
    

    This would result in the correct value 1.2, at least as correct as doubles will allow given their limitations but that's discussed elsewhere on SO, almost endlessly :-).

    0 讨论(0)
  • 2020-12-02 14:27

    When you divide two int's into a floating point value the fraction portion is lost. If you cast one of the items to a float, you won't get this error.

    So for example turn 10 into a 10.0

    double returnValue = (myObject.Value / 10.0);
    
    0 讨论(0)
  • 2020-12-02 14:28

    An integer divided by an integer will return your an integer. Cast either Value to a double or divide by 10.0.

    0 讨论(0)
  • 2020-12-02 14:37

    You're doing integer division if myObject.Value is an int, since both sides of the / are of integer type.

    To do floating-point division, one of the numbers in the expression must be of floating-point type. That would be true if myObject.Value were a double, or any of the following:

    double returnValue = myObject.Value / 10.0;
    double returnValue = myObject.Value / 10d; //"d" is the double suffix
    double returnValue = (double)myObject.Value / 10;
    double returnValue = myObject.Value / (double)10;
    
    0 讨论(0)
  • 2020-12-02 14:48

    I think since myObject is an int, you should

    double returnValue=(myObject.Value/10.0); 
    
    0 讨论(0)
提交回复
热议问题