Floating-point precision problems

情到浓时终转凉″ 提交于 2019-12-22 18:49:22

问题


Using Microsoft Excel 2010.

Why do these two formulae produce different results?

= (0.5 - 0.4 - 0.1)

produces -2.775E-17 while

= 0.5 - 0.4 - 0.1

produces exactly 0.

The only difference is in the brackets. Does Excel try to do clever things with floating-point numbers that it thinks should be a certain value instead of another value in order to hide the usual problems of binary floating-point and make it look as though it's using decimal arithmetic? Is this documented?

来源:https://stackoverflow.com/questions/53337274/floating-point-precision-problems

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!