Floats being Inexact

前端 未结 2 717
别跟我提以往
别跟我提以往 2021-01-25 11:30

I am puzzled. I have no explanation to why this test passes when using the double data type but fails when using the float data type. Consider the foll

2条回答
  •  心在旅途
    2021-01-25 11:50

    The value for 0.01 in decimal is expressed as the series: a1*(1/2) + a2*(1/2)^2 + a3*(1/2)^4 + etc. where aN is a zero or one.

    I leave it to you to figure out the specific values of a1, a2 and how many fractional bits (aN) are required. In some cases a decimal fraction cannot be represented by a finite series of (1/2)^n values.

    For this series to sum to 0.01 in decimal requires that aN go beyond the number of bits stored in a float (full word of bits minus the number of bits for a sign and exponent). But since double has more bits then 0.01 decimal can/might/maybe (you do the calculation) be precisely defined.

提交回复
热议问题