floating-accuracy

问题 I'm putting a float in an Android based SQLite database, like so: private static final String DATABASE_CREATE = "create table " + DATABASE_TABLE + " (" + KEY_ID + " integer primary key autoincrement, " + KEY_FLOAT + " REAL, " + ... ... content.put(KEY_FLOAT, 37.3f); db.insert(DATABASE_TABLE, null, content); When I query the table: Cursor cursor = db.query(false, DATABASE_TABLE, new String[] { KEY_ID, KEY_FLOAT, ... }, KEY_LATITUDE + "=37.3", null, null, null, null, null); the cursor comes

问题 I see that for the following code produces the result as below, any idea on why is the output like this? #include <stdio.h> #include <math.h> int main() { int i = 0; for(i = 0; i < 10; i++) { printf("%d\t\t\t%d\t\t\t", i, (int)pow(10, i)); printf("%f\n", pow(10, i)); } return 0; } Outputs: 0 1 1.000000 1 10 10.000000 2 99 100.000000 3 1000 1000.000000 4 9999 10000.000000 5 100000 100000.000000 6 1000000 1000000.000000 7 9999999 10000000.000000 8 99999999 100000000.000000 9 999999999

问题 I see that for the following code produces the result as below, any idea on why is the output like this? #include <stdio.h> #include <math.h> int main() { int i = 0; for(i = 0; i < 10; i++) { printf("%d\t\t\t%d\t\t\t", i, (int)pow(10, i)); printf("%f\n", pow(10, i)); } return 0; } Outputs: 0 1 1.000000 1 10 10.000000 2 99 100.000000 3 1000 1000.000000 4 9999 10000.000000 5 100000 100000.000000 6 1000000 1000000.000000 7 9999999 10000000.000000 8 99999999 100000000.000000 9 999999999

问题 I know the integer format would be different between big-endian machine and little-endian machine, is it the same for float point format (IEEE 754)? 回答1: The IEEE754 specification for floating point numbers simply doesn't cover the endianness problem. Floating point numbers therefore may use different representations on different machines and, in theory, it's even possible that for two processors integer endianness is the same and floating point is different or vice-versa. See this wikipedia

问题 When a float number needs to be truncated to a certain digit after the floating point, it turns out that it is not easy to be done. For example if the truncating has to be done to second digit after the point, the numbers should be 45.8976 => 45.89, 0.0185 => 0.01 ( second digit after the point is not rounded according to the third digit after the point ). Functions like round() , number_format() , sprintf() round the number and print out 45.8976 => 45.90, 0.0185 => 0.02 I have met two

问题 The select returns right at 23,000 rows The except will return between 60 to 200 rows (and not the same rows) The except should return 0 as it is select a except select a PK: [docSVenum1].[enumID], [docSVenum1].[valueID], [FTSindexWordOnce].[wordID] [tf] is a float and and I get float is not exact But I naively thought avg(float) would be repeatable Avg(float) does appear to be repeatable What is the solution? TF is between 0 and 1 and I only need like 5 significant digits I just need avg(TF)

问题 My question is not about floating precision. It is about why Equals() is different from == . I understand why .1f + .2f == .3f is false (while .1m + .2m == .3m is true ). I get that == is reference and .Equals() is value comparison. ( Edit : I know there is more to this.) But why is (.1f + .2f).Equals(.3f) true , while (.1d+.2d).Equals(.3d) is still false ? .1f + .2f == .3f; // false (.1f + .2f).Equals(.3f); // true (.1d + .2d).Equals(.3d); // false 回答1: The question is confusingly worded.

问题 I am trying to implement a program with floating point numbers, using two or more programming languages. The program does say 50k iterations to finally bring the error to very small value. To ensure that my results are comparable, I wanted to make sure I use data types of same precision in different languages. Would you please tell if there is correspondence between float/double of C/C++ to that in D and Go. I expect C/C++ and D to be quite close in this regard, but not sure. Thanks a lot.

问题 Consider the following example. There is an image where user can select rectangular area (part of it). The image is displayed with some scale. Then we change the scale and we need to recalculate the new coordinates of selection. Let's take width, newSelectionWidth = round(oldSelectionWidth / oldScale * newScale) where oldScale = oldDisplayImageWidth / realImageWidth , newScale = newDisplayImageWidth / realImageWidth , all the values except for scales are integers. The question is how to prove

问题 I receive a FIX message for a trade Allocation, where Price is given in cents (ZAR / 100), but commission is given in Rands . These values are represented by the constants. When I run this calculation, commPerc1 shows a value of 0.099999999999999978, and commPerc2 shows 0.1. These values look to differ by x10, yet with the check of calculating back to Rands , both commRands1 and commRands2 show very similar values of 336.4831554 and 336.48315540000004 respectively. private const double Price