Arithmetic in ruby

后端 未结 6 1425
日久生厌
日久生厌 2020-12-16 06:25

Why this code 7.30 - 7.20 in ruby returns 0.0999999999999996, not 0.10?

But if i\'ll write 7.30 - 7.16, for examp

6条回答
  •  既然无缘
    2020-12-16 06:51

    Since you are doing floating point math then the number returned is what your computer uses for precision.

    If you want a closer answer, to a set precision, just multiple the float by that (such as by 100), convert it to an int, do the math, then divide.

    There are other solutions, but I find this to be the simplest since rounding always seems a bit iffy to me.

    This has been asked before here, you may want to look for some of the answers given before, such as this one: Dealing with accuracy problems in floating-point numbers

提交回复
热议问题