floating-accuracy

System.out.println(2.14656); 2.14656 System.out.println(2.14656%2); 0.14656000000000002 WTF? The do give the expected results. Your expectations are incorrect. When you type the double-precision literal 2.14656 , what you actually get is the closest double-precision value, which is: 2.14656000000000002359001882723532617092132568359375 the println happens to round this when it prints it out (to 17 significant digits), so you see the nice value that you expect. After the modulus operation (which is exact), the value is: 0.14656000000000002359001882723532617092132568359375 Again, this is rounded

Why is this C program giving the "wrong" output? #include<stdio.h> void main() { float f = 12345.054321; printf("%f", f); getch(); } Output: 12345.054688 But the output should be, 12345.054321 . I am using VC++ in VS2008. It's giving the "wrong" answer simply because not all real values are representable by floats (or doubles, for that matter). What you'll get is an approximation based on the underlying encoding. In order to represent every real value, even between 1.0x10 -100 and 1.1x10 -100 (a truly minuscule range), you still require an infinite number of bits. Single-precision IEEE754

Is there a way to set a Perl script's floating point precision (to 3 digits), without having to change it specifically for every variable? Something similar to TCL's: global tcl_precision set tcl_precision 3 Xetius There is no way to globally change this. If it is just for display purposes then use sprintf("%.3f", $value); . For mathematical purposes, use (int(($value * 1000.0) + 0.5) / 1000.0) . This would work for positive numbers. You would need to change it to work with negative numbers though. draegtun Use Math::BigFloat or bignum : use Math::BigFloat; Math::BigFloat->precision(-3); my $x

I came to know about the accuracy issues when I executed the following following program: public static void main(String args[]) { double table[][] = new double[5][4]; int i, j; for(i = 0, j = 0; i <= 90; i+= 15) { if(i == 15 || i == 75) continue; table[j][0] = i; double theta = StrictMath.toRadians((double)i); table[j][1] = StrictMath.sin(theta); table[j][2] = StrictMath.cos(theta); table[j++][3] = StrictMath.tan(theta); } System.out.println("angle#sin#cos#tan"); for(i = 0; i < table.length; i++){ for(j = 0; j < table[i].length; j++) System.out.print(table[i][j] + "\t"); System.out.println();

In the following query declare @a float(23) declare @b float(23) declare @c float(53) set @a = 123456789012.1234 set @b = 1234567.12345678 set @c = @a * @b select @c select LTRIM(STR((@c),32,12)) declare @x decimal(16,4) declare @y decimal(16,8) declare @z decimal (32,12) set @x = 123456789012.1234 set @y = 1234567.12345678 set @z = @x * @y select @z I get answers as 1.52415693411713E+17 152415693411713020.000000000000 152415692881907790.143935926652 From the above answers the third answer is the correct one. Is this the reason why float data type is called Approximate Numeric Data Type Or am

for example, 0 , 0.5, 0.15625 , 1 , 2 , 3... are values converted from IEEE 754. Are their hardcode version precise? for example: is float a=0; if(a==0){ return true; } always return true? other example: float a=0.5; float b=0.25; float c=0.125; is a * b always equal to 0.125 and a * b==c always true? And one more example: int a=123; float b=0.5; is a * b always be 61.5? or in general, is integer multiply by IEEE 754 binary float precise? Or a more general question: if the value is hardcode and both the value and result can be represented by binary format in IEEE 754 (e.g.:0.5 - 0.125), is the

Consider these two cases: fmt.Println(912 * 0.01) fmt.Println(float64(912) * 0.01) ( Go Playground link ) The second one prints 9.120000000000001, which is actually fine, I understand why that is happening . However, why does the first line print 9.12, without the …01 at the end? Does Go multiply the two untyped constants and simply replace them with a 9.12 literal when compiling? As per spec : Constant expressions are always evaluated exactly; intermediate values and the constants themselves may require precision significantly larger than supported by any predeclared type in the language.

Possible Duplicate: strange output in comparison of float with float literal When I am trying to compare 2 same float values it doesn't print "equal values" in the following code : void main() { float a = 0.7; clrscr(); if (a < 0.7) printf("value : %f",a); else if (a == 0.7) printf("equal values"); else printf("hello"); getch(); } Thanks in advance. While many people will tell you to always compare floating point numbers with an epsilon (and it's usually a good idea, though it should be a percentage of the values being compared rather than a fixed value), that's not actually necessary here

I've been waffling on how to deal with currency display and math in PHP, and for a long time have been storing it in MySQL using the DECIMAL type, and using money_format() to format it for display on the web page. However, today I looked at the actual prototype: string money_format ( string $format , float $number ) I'm a little confused now. All I've been told is, avoid floats for money! But here it is, the fundamental formatting function (say that five times fast), casting the input to a float. number_format() does the same. So my questions are: Unless I'm dealing with fractional cents or