IEEE 754 floating point arithmetic rounding error in c# and javascript

前端 未结 4 1980
野性不改
野性不改 2020-12-06 21:21

I just read a book about javascript. The author mentioned a floating point arithmetic rounding error in the IEEE 754 standard.

For example adding 0.1 and 0.2 yields

4条回答
  •  半阙折子戏
    2020-12-06 21:55

    The closest representations of those three numbers in double precision floating point are:

    • 0.1 --> 0.10000000000000001 = D(3FB99999 9999999A)
    • 0.2 --> 0.20000000000000001 = D(3FC99999 9999999A)
    • 0.3 --> 0.29999999999999999 = D(3FD33333 33333333)

    The next larger representable number beyond 0.29999999999999999 is:

    • 0.30000000000000004 = D(3FD33333 33333334)

    The closest representation of

    • 0.10000000000000001 + 0.20000000000000001 is 0.30000000000000004

    So you are comparing 0.29999999999999999 and 0.30000000000000004. Does this give you more insight as to what is happening?

    As far as the use of decimal instead of binary representations, that doesn't work either. Take for example one third:

    • 1/3 = 0.3333333333333333333333333333333...

    which has no exact representation even using decimal digits. Any computations should always take representation error into account.

提交回复
热议问题