floating-accuracy | 易学教程

How to actually avoid floating point errors when you need to use float?

阅读更多关于 How to actually avoid floating point errors when you need to use float?

I am trying to affect the translation of a 3D model using some UI buttons to shift the position by 0.1 or -0.1. My model position is a three dimensional float so simply adding 0.1f to one of the values causes obvious rounding errors. While I can use something like BigDecimal to retain precision, I still have to convert it from a float and back to a float at the end and it always results in silly numbers that are making my UI look like a mess. I could just pretty the displayed values but the rounding errors will only get worse with more editing and they make my save files rather hard to read.

SQLite query where clause with floating point numbers fails?

阅读更多关于 SQLite query where clause with floating point numbers fails?

I'm putting a float in an Android based SQLite database, like so: private static final String DATABASE_CREATE = "create table " + DATABASE_TABLE + " (" + KEY_ID + " integer primary key autoincrement, " + KEY_FLOAT + " REAL, " + ... ... content.put(KEY_FLOAT, 37.3f); db.insert(DATABASE_TABLE, null, content); When I query the table: Cursor cursor = db.query(false, DATABASE_TABLE, new String[] { KEY_ID, KEY_FLOAT, ... }, KEY_LATITUDE + "=37.3", null, null, null, null, null); the cursor comes back empty. If I change the value of the float to 37.0 it works properly, returning the record in the

Rules-of-thumb for minimising floating-point errors in C?

阅读更多关于 Rules-of-thumb for minimising floating-point errors in C?

问题 Regarding minimising the error in floating-point operations, if I have an operation such as the following in C: float a = 123.456; float b = 456.789; float r = 0.12345; a = a - (r * b); Will the result of the calculation change if I split the multiplication and subtraction steps out, i.e.: float c = r * b; a = a - c; I am wondering whether a CPU would then treat these calculations differently and thereby the error may be smaller in one case? If not, which I presume anyway, are there any good

Result of casting double to int is wrong

阅读更多关于 Result of casting double to int is wrong

There seems to be some kind of obscure rounding error when I run the following code: int roundedTotal = (int)(PriorityJob * 100.0); Initially PriorityJob = 1.4 and roundedTotal is undefined. Evaluating PriorityJob * 100.0 at that point gives 140 . Afterwards roundedTotal = 139 . Apparently, 140.0 is being interpreted as 139.99999. Is this a deficiency in the floating point engine? I have never seen anything like it. Just about every modern computer uses a binary representation for floating-point numbers. Just as 1/3 = 0.33333333... can't be represented exactly as a decimal fraction, so 1/10

printing the integral part of a floating point number

阅读更多关于 printing the integral part of a floating point number

问题 I am trying to figure out how to print floating point numbers without using library functions. Printing the decimal part of a floating point number turned out to be quite easy. Printing the integral part is harder: static const int base = 2; static const char hex[] = "0123456789abcdef"; void print_integral_part(float value) { assert(value >= 0); char a[129]; // worst case is 128 digits for base 2 plus NUL char * p = a + 128; *p = 0; do { int digit = fmod(value, base); value /= base; assert(p

How to avoid less precise sum for numpy-arrays with multiple columns

阅读更多关于 How to avoid less precise sum for numpy-arrays with multiple columns

问题 I've always assumed, that numpy uses a kind of pairwise-summation, which ensures high precision also for float32 - operations: import numpy as np N=17*10**6 # float32-precision no longer enough to hold the whole sum print(np.ones((N,1),dtype=np.float32).sum(axis=0)) # [17000000.], kind of expected However, it looks as if a different algorithm is used if the matrix has more than one column: print(np.ones((N,2),dtype=np.float32).sum(axis=0)) # [16777216. 16777216.] the error is just to big

How does floating point error propagate when doing mathematical operations in C++?

阅读更多关于 How does floating point error propagate when doing mathematical operations in C++?

Let's say that we have declared the following variables float a = 1.2291; float b = 3.99; float variables have precision 6, which (if I understand correctly) means that the difference between the number that the computer actually stores and the actual number that you want will be less than 10^-6 that means that both a and b have some error that is less than 10^-6 so inside the computer a could actually be 1.229100000012123 and b could be 3.9900000191919 now let's say that you have the following code float c = 0; for(int i = 0; i < 1000; i++) c += a + b; my question is, will c 's final result

Guaranteed precision of sqrt function in C/C++

阅读更多关于 Guaranteed precision of sqrt function in C/C++

问题 Everyone knows sqrt function from math.h / cmath in C/C++ - it returns square root of its argument. Of course, it has to do it with some error, because not every number can be stored precisely. But am I guaranteed that the result has some precision? For example, 'it's the best approximation of square root that can be represented in the floating point type used or if you calculate square of the result, it will be as close to initial argument as possible using the floating point type given`?

Check if a number is rational in Python, for a given fp accuracy

阅读更多关于 Check if a number is rational in Python, for a given fp accuracy

I would like to know a good way of checking if a number x is a rational (two integers n,m exist so that x=n/m) in python. In Mathematica, this is done by the function Rationalize[6.75] : 27/4 I assume this question has an answer for a given accuracy. Is there a common algorithm of obtaining these two integers? SilentGhost In python >= 2.6 there is a as_integer_ratio method on floats: >>> a = 6.75 >>> a.as_integer_ratio() (27, 4) >>> import math >>> math.pi.as_integer_ratio() (884279719003555, 281474976710656) However, due to the way floats are defined in programming languages there are no

Result of casting double to int is wrong

阅读更多关于 Result of casting double to int is wrong

问题 There seems to be some kind of obscure rounding error when I run the following code: int roundedTotal = (int)(PriorityJob * 100.0); Initially PriorityJob = 1.4 and roundedTotal is undefined. Evaluating PriorityJob * 100.0 at that point gives 140 . Afterwards roundedTotal = 139 . Apparently, 140.0 is being interpreted as 139.99999. Is this a deficiency in the floating point engine? I have never seen anything like it. 回答1: Just about every modern computer uses a binary representation for