Result of casting double to int is wrong

北城余情 提交于 2019-12-01 03:47:07

问题


There seems to be some kind of obscure rounding error when I run the following code:

int roundedTotal = (int)(PriorityJob * 100.0);

Initially PriorityJob = 1.4 and roundedTotal is undefined. Evaluating PriorityJob * 100.0 at that point gives 140. Afterwards roundedTotal = 139.

Apparently, 140.0 is being interpreted as 139.99999. Is this a deficiency in the floating point engine? I have never seen anything like it.


回答1:


Just about every modern computer uses a binary representation for floating-point numbers.

Just as 1/3 = 0.33333333... can't be represented exactly as a decimal fraction, so 1/10 (and hence most non-integer decimal values, including 1.4) can't be represented exactly as a binary fraction. It will instead be represented by the nearest representable value, which may be slightly more or less than the "true" value.

You might want to round to the nearest integer instead: (int)(PriorityJob * 100.0 + 0.5)



来源:https://stackoverflow.com/questions/12424779/result-of-casting-double-to-int-is-wrong

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!