Float precision in ruby

前端 未结 2 1799
我在风中等你
我在风中等你 2020-12-11 13:01

I\'m writing a ruby program that uses floats. I\'m having trouble with the precision. For example

1.9.3p194 :013 > 113.0 * 0.01
# => 1.13000000000000         


        
2条回答
  •  轻奢々
    轻奢々 (楼主)
    2020-12-11 13:26

    In calculation with float you should use sigma method - it means not to compare two values, but compare absolute difference of them with a very little value - 1e-10, for example.

    ((113 * 0.01) - 1.13).abs<1e-10
    

提交回复
热议问题