floating-accuracy

Possible Duplicates: Why is floating point arithmetic in C# imprecise? Why does ghci say that 1.1 + 1.1 + 1.1 > 3.3 is True? #!/usr/bin/perl $l1 = "0+0.590580+0.583742+0.579787+0.564928+0.504538+0.459805+0.433273+0.384211+0.3035810"; $l2 = "0+0.590580+0.583742+0.579788+0.564928+0.504538+0.459805+0.433272+0.384211+0.3035810"; $val1 = eval ($l1); $val2 = eval ($l2); $diff = (($val1 - $val2)/$val1)*100; print " (($val1 - $val2)/$val1)*100 ==> $diff\n"; Surprisingly the output ended up to be ((4.404445 - 4.404445)/4.404445)*100 ==> -2.01655014354845e-14. Is it not supposed to be a ZERO???? Can any

In a loop I am adding 0.10 till I reach the desired #, and getting the index. This is my code : private static int getIndexOfUnits(float units) { int index = -1; float addup = 0.10f; for(float i = 1.00f; i < units; i=(float)i+addup) { index++; System.out.println("I = " + i + " Index = " + index); } return index; } If the units passed is 5.7, the ourput I see is : I = 1.0 Index = 0 I = 1.1 Index = 1 I = 1.2 Index = 2 I = 1.3000001 Index = 3 I = 1.4000001 Index = 4 I = 1.5000001 Index = 5 I = 1.6000001 Index = 6 I = 1.7000002 Index = 7 I = 1.8000002 Index = 8 I = 1.9000002 Index = 9 I = 2

Why so? >>> max(2, 2.01) 2.0099999999999998 The number 2.01 represented in binary is: b10.00000010100011111100001010001111110000101000111111000010100011111100... The computer uses only a finite number of digits to store floating-point values, but the binary representation of 2.01 requires infinitely many digits; as a result, it is rounded to the closest representable value: b10.000000101000111111000010100011111100001010001111110 Expressed in decimal, this number is exactly: 2.0099999999999997868371792719699442386627197265625 When you print it out, it is rounded a second time to seventeen

Debugging some finance-related SQL code found a strange issue with numeric(24,8) mathematics precision. Running the following query on your MSSQL you would get A + B * C expression result to be 0.123457 SELECT A, B, C, A + B * C FROM ( SELECT CAST(0.12345678 AS NUMERIC(24,8)) AS A, CAST(0 AS NUMERIC(24,8)) AS B, CAST(500 AS NUMERIC(24,8)) AS C ) T So we have lost 2 significant symbols. Trying to get this fixed in different ways i got that conversion of the intermediate multiplication result (which is Zero!) to numeric (24,8) would work fine. And finally a have a solution. But still I hace a

ree-1.8.7-2010.02 :003 > (10015.8*100.0).to_i => 1001579 ree-1.8.7-2010.02 :004 > 10015.8*100.0 => 1001580.0 ree-1.8.7-2010.02 :005 > 1001580.0.to_i => 1001580 ruby 1.8.7 produces the same. Does anybody knows how to eradicate this heresy? =) Actually, all of this make sense. Because 0.8 cannot be represented exactly by any series of 1 / 2 ** x for various x , it must be represented approximately, and it happens that this is slightly less than 10015.8. So, when you just print it, it is rounded reasonably. When you convert it to an integer without adding 0.5, it truncates .79999999... to .7 When

I've been going through a Haskell tutorial recently and noticed this behaviour when trying some simple Haskell expressions in the interactive ghci shell: Prelude> 1.1 + 1.1 == 2.2 True Prelude> 1.1 + 1.1 + 1.1 == 3.3 False Prelude> 1.1 + 1.1 + 1.1 > 3.3 True Prelude> 1.1 + 1.1 + 1.1 3.3000000000000003 Does anybody know why that is? Brian Campbell Because 1.1 and 3.3 are floating point numbers . Decimal fractions, such as .1 or .3, are not exactly representable in a binary floating point number. .1 means 1/10. To represent that in binary, where each fractional digit represents 1/2 n (1/2, 1/4,

Is there a function for integer exponentiation in OCaml? ** is only for floats. Although it seems to be mostly accurate, isn't there a possibility of precision errors, something like 2. ** 3. = 8. returning false sometimes? Is there a library function for integer exponentiation? I could write my own, but efficiency concerns come into that, and also I'd be surprised if there isn't such a function already. Regarding the floating-point part of your question: OCaml calls the underlying system's pow() function. Floating-point exponentiation is a difficult function to implement, but it only needs to

I want to determine (in c++) if one float number is the multiplicative inverse of another float number. The problem is that i have to use a third variable to do it. For instance this code: float x=5,y=0.2; if(x==(1/y)) cout<<"They are the multiplicative inverse of eachother"<<endl; else cout<<"They are NOT the multiplicative inverse of eachother"<<endl; will output: "they are not..." which is wrong and this code: float x=5,y=0.2,z; z=1/y; if(x==z) cout<<"They are the multiplicative inverse of eachother"<<endl; else cout<<"They are NOT the multiplicative inverse of eachother"<<endl; will output