Specification of Excel floating point behaviour

后端 未结 2 870
悲&欢浪女
悲&欢浪女 2020-12-18 15:53

Though Excel uses the standard IEEE754 binary64 format, it doesn\'t use the same rules for arithmetic and comparisons, e.g.

=0.1+0.2=0.3

r

2条回答
  •  自闭症患者
    2020-12-18 15:59

    This documentation tangentially addresses this:

    https://support.microsoft.com/en-us/kb/78113

    See especially:

    Excel 97, however, introduced an optimization that attempts to correct for this problem. Should an addition or subtraction operation result in a value at or very close to zero, Excel 97 and later will compensate for any error introduced as a result of converting an operand to and from binary.

    If they apply this "optimization" to the difference between 0.1 + 0.2 and 0.3 it apparently is converted to 0, which does mean that they are not following the standards strictly.

提交回复
热议问题