floating-accuracy

问题 I'm having issues in my code due to the multiple representations that a same float number can have. For instance, these number are considered to be the same: 0.0299999400 0.0300000000 I don't care much for a big precision, I need to calculate CRC of those numbers and they should be the same, so my approach was to use this code: private static float Rounding(float v, float p) { return Mathf.Round (v * p) / p; } Where p is my precision. That seems to be working well, but in that case, if I used

问题 I'm wondering what kind of errors I can encounter if I set /fp:fast instead of fp:precise? I work under MSV10 I perform /,*,+,- operations on double with max digits equal to 8, like 1.4379294 (currency rate). 回答1: The same errors you always get with double calculations: you never get more than 15 significant digits after you convert the result to decimal, the rest are just random noise digits. The difference between /fp:fast and /fp:precise is what happens to those noise digits. The /fp

问题 This question already has answers here : strange output in comparison of float with float literal (8 answers) Closed 4 years ago . I was reading this. It contains following C program. #include<stdio.h> int main() { float x = 0.1; if (x == 0.1) printf("IF"); else if (x == 0.1f) printf("ELSE IF"); else printf("ELSE"); } The article says that The output of above program is “ELSE IF” which means the expression “x == 0.1″ returns false and expression “x == 0.1f” returns true. But I tried it on

问题 My question, in short is, why does rounding error in floats only show up after calculations and not for storage of literals? What I mean is this - I know about the issues that arise due to rounding error in floats when converting from decimal to binary and back. Eg, in Java: double a = 10.567; double b = 2.16; double c = a * b; c then stores the value 22.824720000000003, instead of 22.82472. This is because the result 22.82472 cannot be stored accurately in the finite binary digits of the

问题 If I have a YAML file like --- 2.1: my product version without any quotation marks, Ruby will treat the 2.1 as a Float (at least under Syck - I'm not sure about Psych). Are there any risks in using a Float as a hash's key, so long as all of the other occurences of the key are either from a YAML file or literals from Ruby? 来源： https://stackoverflow.com/questions/8708113/is-it-safe-to-use-floats-in-yaml-hashes-with-ruby

问题 I was trying to answer a question from here (Substracion of two items from two lists). The original problem has two different lists with float values and the goal is to zip them and substract This code works fine: Enter=[7.12, 7.14, 7.24, 7.45, 7.28, 7.31, 7.18, 7.25, 7.33, 7.38] Leave=[7.56, 7.24, 7.48, 7.52, 7.45, 7.57, 7.22, 7.31, 7.37, 7.41] intervals = map(lambda x, y: y-x, Enter, Leave) Then: print intervals Output: [0.4399999999999995, 0.10000000000000053, 0.24000000000000021, 0

问题 This is concerning Java. From what I've understood, 0.1 cannot be perfectly represented by Java because of binary representations. That makes 0.1 + 0.1 + 0.1 == 0.3 false. However, why does 0.1 + 0.1 == 0.2 gives true? 回答1: 0.1 cannot be perfectly represented by Java because of binary representations. That makes 0.1 + 0.1 + 0.1 == 0.3 false. That is not the entire reason why the equality is false, although it is part of it. 0.3 is not exactly 3/10 either. It so happens that 0.2 is exactly

问题 This question already has answers here : Python 3.x rounding behavior (8 answers) Closed 3 years ago . One half, i.e. 0.5 in decimal, has an exact binary representation: 0.1 Nevertheless, if I round it to integer, I get 0 instead of 1. I tried in Python and C, which behave the same. E.g. the python code: >>> a,b,c = 0.49, 0.5, 0.51 >>> [round(x) for x in (a,b,c)] [0, 0, 1] >>> "%.0f %.0f %.0f" % (a,b,c) '0 0 1' Interestingly, >>> a,b,c = 0.049, 0.05, 0.051 >>> [round(x,1) for x in (a,b,c)] [0

问题 I have the following line of code. hero->onBeingHit(ENEMY_ATTACK_POINT * (1.0 - hero->getDefensePercent())); void onBeingHit(int decHP) method accepts integer number and updates health points. float getDefensePercent() method is a getter method returning the defense percent of a hero. ENEMY_ATTACK_POINT is a macro constant factor defined as #define ENEMY_ATTACK_POINT 20 . Let's say hero->getDefensePercent() returns 0.1 . So the calculation is 20 * (1.0 - 0.1) = 20 * (0.9) = 18 Whenever I

问题 I am trying to convert a IEEE 754 32 bit single precision floating point value (standard c float variable) to an unsigned long variable in the format of MIL-STD-1750A. I have included the specification for both IEEE 754 and MIL-STD-1750A at the bottom of the post. Right now, I am having issues in my code with converting the exponent. I also see issues with converting the mantissa, but I haven't gotten to fixing those yet. I am using the examples listed in Table 3 in the link above to confirm