IEEE 754 floating point arithmetic rounding error in c# and javascript

前端 未结 4 1944
野性不改
野性不改 2020-12-06 21:21

I just read a book about javascript. The author mentioned a floating point arithmetic rounding error in the IEEE 754 standard.

For example adding 0.1 and 0.2 yields

4条回答
  •  伪装坚强ぢ
    2020-12-06 21:37

    The error is NOT a rounding error, it's simply that some values cannot be exactly represented by the IEEE 754 standard. see Jon Skeet's article on binary floating point numbers in .net for further reading.

    For dealing with numbers like your example (base-10) you should be using the decimal datatype in C# as it can represent these numbers exactly, so you get values you'd expect.

    A typical way is to define some epsilon value, and check whether the result is within targetvalue +- epsilon:

    double const epsilon = 0.000001; // or whatever
    
    if(valueA >= valueB - epsilon && valueA <= valueB + epsilon)
    {
        // treat as valueA = valueB
    }
    

提交回复
热议问题