VB.Net Double comparison after some additions

那年仲夏 提交于 2019-12-11 02:42:45

问题


I've faced a weird case with a double variable after adding some values to it. The problem occurs when adding (0.2) to a double variable more than one time -I think it only happens with (0.2)- for example: consider this code:

Dim i As Double = 2
i = i + 0.2
MsgBox(i) '2.2
MsgBox(i > 2.2) 'False >> No problem

But if I add (0.2) more than one time:

Dim i As Double = 2
i = i + 0.2
i = i + 0.2
MsgBox(i) '2.4
Msgbox(i > 2.4) 'True >> !!!!

Also

Dim i As Double = 2
For x As Integer = 1 to 5
    i = i + 0.2
Next
MsgBox(i) '3
Msgbox(i > 3) 'True >> !!!!

I tried the same code with other values, I don't get this issue:

Dim i As Double = 2
i = i + 0.5
i = i + 0.5
MsgBox(i) '3
Msgbox(i > 3) 'False >> No problem

Anyone has an explanation for this?? Thank you


回答1:


If you take the example 3 you would see that the result is actually 3.0000000000000009.

The problem is in the rounding of a double.

If you change the data type decimal the problem is fixed:

Sub Main()

    Dim i As Decimal = 2

    For x As Integer = 1 To 5
        i = i + 0.2
    Next
    MsgBox(i) '3
    MsgBox(i > 3) 'False >> No problem

End Sub

This is about C# but, I guess, it the same thing for vb.net.




回答2:


This issue known as "Accuracy Problems (Wikipedia Link)"

The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations can not precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.



来源:https://stackoverflow.com/questions/30424579/vb-net-double-comparison-after-some-additions

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!