floating-accuracy

var x = dr["NationalTotal"].ToString(); gives me 333333333 var xxx = Convert.ToSingle(dr["NationalTotal"].ToString()); gives me 333333344 Any ideas why? The floating point specification says that the 32 bit representation of a floating point number is Thus, the largest integer that can be represented without loss of (integer) accuracy is 16777216 (0x1000000). A simple test program to convince you that this is so: #include <stdio.h> int main(void) { float x; unsigned long j; j = 0x00FFFFFC; int i; x = j; for(i=0; i<10;i++) printf("%ld + %d = %f\n", j, i, x+i); } Output: 16777212 + 0 = 16777212

Python (and almost anything else) has known limitations while working with floating point numbers (nice overview provided here ). While problem is described well in the documentation it avoids providing any approach to fixing it. And with this question I am seeking to find a more or less robust way to avoid situations like the following: print(math.floor(0.09/0.015)) # >> 6 print(math.floor(0.009/0.0015)) # >> 5 print(99.99-99.973) # >> 0.016999999999825377 print(.99-.973) # >> 0.017000000000000015 var = 0.009 step = 0.0015 print(var < math.floor(var/step)*step+step) # False print(var < (math

My sanity check fails because a double variable does not contain the expected result, it's really bizarre. double a = 1117.54 + 8561.64 + 13197.37; double b = 22876.55; Console.WriteLine("{0} == {1}: {2}", a, b, a == b); Gives us this output: 22876.55 == 22876.55: False Further inspection shows us that variable a, in fact, contains the value 22876.550000000003. This is reproducible in vb.net as well. Am I sane ? What is going on? Floating point types are not always capable of accurately representing their exact decimal values. It's either a "known bug" or "by design", depending on your

so i have 2 functions that should do the same thing float ver1(float a0, float a1) { float r0 = a0 - a1; if (abs(r0) > PI) { if (r0 > 0) { r0 -= PI2; } else { r0 += PI2; } } return r0; } float ver2(float a0, float a1) { float a2 = a1 - PI2; float r0 = a0 - a1; float r1 = a0 - a2; if (abs(r0) < abs(r1)) { return r0; } if (abs(r0) > abs(r1)) { return r1; } return 0; } note: PI and PI2 are float constants of pi and 2*pi The odd thing is that sometimes they produce different results, for example if you feed them 0.28605145 and 5.9433694 then the first one results in 0.62586737 and the second one

In F#. How to efficiently compare floats for equality that are almost equal? It should work for very large and very small values too. I am thinking of first comparing the Exponent and then the Significand (Mantissa) while ignoring the last 4 bits of the its 52 bits. Is that a good approach? How can I get the Exponent and Significand of a float? An F# float is just a shorthand for System.Double . That being the case, you can use the BitConverter.DoubleToInt64Bits method to efficiently (and safely! ) "cast" an F# float value to int64 ; this is useful because it avoids allocating a byte[] , as

I am trying to count the amount of dollar and coin denominations in a grand total by using a series of while loops. When I get down to the coins however, I am off by a penny. When I enter say 99.95, I get the output 3 quarters, 1 dime, 1 nickel, and 4 pennies. I've narrowed the problem down to a floating point accuracy issue. However all the solutions I've researched haven't been applicable in my situation. Any pointers? #include <iostream> using namespace std; int main() { float amount; cout<<"enter amount" << endl; cin>>amount; int pennies=0, nickels=0, dimes=0, quarters=0, ones=0, fives=0,