floating-accuracy

In most cases, I understand that a floating point comparison test should be implemented using over a range of values (abs(x-y) < epsilon), but does self subtraction imply that the result will be zero? // can the assertion be triggered? float x = //?; assert( x-x == 0 ) My guess is that nan/inf might be special cases, but I'm more interested in what happens for simple values. edit: I'm happy to pick an answer if someone can cite a reference (IEEE floating point standard)? As you hinted, inf - inf is NaN , which is not equal to zero. Similarly, NaN - NaN is NaN . It is true, however, that for

I often need to floor or ceil a CGFloat to an int , for calculation of an array index. The problem I permanently see with floorf(theCGFloat) or ceilf(theCGFloat) is that there can be troubles with floating point inaccuracies. So what if my CGFloat is 2.0f but internally it is represented as 1.999999999999f or something like that. I do floorf and get 1.0f , which is a float again. And yet I must cast this beast to int which may introduce another problem. Is there a best practice how to floor or ceil a float to an int such that something like 2.0 would never accidentally get floored to 1 and

Is it possible to get division by 0 (or infinity) in the following example? public double calculation(double a, double b) { if (a == b) { return 0; } else { return 2 / (a - b); } } In normal cases it will not, of course. But what if a and b are very close, can (a-b) result in being 0 due to precision of the calculation? Note that this question is for Java, but I think it will apply to most programming languages. nwellnhof In Java, a - b is never equal to 0 if a != b . This is because Java mandates IEEE 754 floating point operations which support denormalized numbers. From the spec : In

In the following code, both coeff1 and coeff2 are Decimal objects. When i check their type using type(coeff1), i get (class 'decimal.Decimal') but when i made a test code and checked decimal objects i get decimal. Decimal, without the word class coeff1 = system[i].normal_vector.coordinates[i] coeff2 = system[m].normal_vector.coordinates[i] x = coeff2/coeff1 print(type(x)) system.xrow_add_to_row(x,i,m) another issue is when i change the first input to the function xrow_add_to_row to negative x: system.xrow_add_to_row(-x,i,m) I get invalid operation error at a line that is above the changed code

This question already has an answer here: Strange behaviour of the pow function 5 answers I am having some problems with pow() function in C. When ever run this code, 153 as input, the sum evaluates to 152 . However if I dont use pow() function and instead use a for loop to get the value of N n , the sum evaluates to 153 . Can anyone help please explain me this difference? #include <stdio.h> #include <string.h> #include <stdlib.h> #include <math.h> int main(void) { unsigned int i, n, sum = 0, N, a = 1, j; char num[100], x[2] = { 0 }; printf("Determining an armstrong number\n\n" "Enter a number

The following value gives me wrong precision. It is observed with only specific numbers. It might be a floating representation problem, but wanted to know the specific reason. String m = "154572.49"; //"154,572.49"; Float f = Float.parseFloat(m); System.out.println(f); The output it is printing is 154572.48 instead of 154572.49 . If you want decimal numbers to come out as exactly as you entered them in Java, use BigDecimal instead of float. Floating point numbers are inherently inaccurate for decimals because many numbers that terminate in decimal (e.g. 0.1) are recurring numbers in binary and

This question already has an answer here: Python 3.x rounding behavior 9 answers Looks like both 4.5 and 5.5 have exact float representations in Python 3.5: >>> from decimal import Decimal >>> Decimal(4.5) Decimal('4.5') >>> Decimal(5.5) Decimal('5.5') If this is the case, then why >>> round(4.5) 4 >>> round(5.5) 6 ? In Python 3, exact half way numbers are rounded to the nearest even result. This behavior changed in Python 3 The round() function rounding strategy and return type have changed. Exact halfway cases are now rounded to the nearest even result instead of away from zero. (For example