Ruby float precision

后端 未结 2 1032
孤街浪徒
孤街浪徒 2021-01-02 12:34

As I understand it, Ruby (1.9.2) floats have a precision of 15 decimal digits. Therefore, I would expect rounding float x to 15 decimal places would equal x

2条回答
  •  没有蜡笔的小新
    2021-01-02 13:13

    Well, though I'm not certain on the details of how Ruby handles floats internally, I do know why this particular bit of code is failing on my box:

     > x = (0.33 * 10)
    => 3.3000000000000003
     > x.round(15)
    => 3.300000000000001
    

    The first float keeps 16 decimal places for a total of 17 digits, for whatever reason. So, rounding to 15 discards those digits.

提交回复
热议问题