IEEE-754 floating-point precision: How much error is allowed?

前端 未结 2 1357
说谎
说谎 2020-12-03 23:23

I\'m working on porting the sqrt function (for 64-bit doubles) from fdlibm to a model-checker tool I\'m using at the moment (cbmc).
As part of my doings, I

2条回答
  •  慢半拍i
    慢半拍i (楼主)
    2020-12-04 00:16

    In binary, the first 58 bits of the arbitrary precision answer is 1011111111111111111111110101010101111111111111111011010001...

    The 53-bit significand of the double value is

    10111111111111111111111101010101011111111111111110111

    Which means that the double value is correctly rounded to 53 significant bits, and is within 1/2 ULP. (That the error is "large" is only because the number itself is large).

提交回复
热议问题