floating-accuracy

问题 I am trying to count the amount of dollar and coin denominations in a grand total by using a series of while loops. When I get down to the coins however, I am off by a penny. When I enter say 99.95, I get the output 3 quarters, 1 dime, 1 nickel, and 4 pennies. I've narrowed the problem down to a floating point accuracy issue. However all the solutions I've researched haven't been applicable in my situation. Any pointers? #include <iostream> using namespace std; int main() { float amount; cout

问题 I am trying to write a simple log base 2 method. I understand that representing something like std::log(8.0) and std::log(2.0) on a computer is difficult. I also understand std::log(8.0) / std::log(2.0) may result in a value very slightly lower than 3.0. What I do not understand is why putting the result of a the calculation below into a double and making it an lvalue then casting it to an unsigned int would change the result compared to casting the the formula directly. The following code

问题 Here's a little gem directly from my VBE (MS Excel 2007 VBA): ?clng(150*0.85) 127 x = 150*0.85 ?clng(x) 128 Can anybody explain this behaviour? IMHO the first expression should yield 128 (.5 rounded to nearest even), or at least should both results be equal. 回答1: I think wqw is right, but I'll give the details. In the statement clng(150 * 0.85) , 150 * 0.85 is calculated in extended-precision: 150 = 1.001011 x 2^7 0.85 in double precision = 1

问题 This question already has answers here : Is floating point math broken? (31 answers) Closed 4 years ago . today happened to me a strange thing, when I try to compile and execute the output of this code isn't what I expected. Here is the code that simply add floating values to an array of float and then print it out. The simple code: int main(){ float r[10]; int z; int i=34; for(z=0;z<10;z++){ i=z*z*z; r[z]=i; r[z]=r[z]+0.634; printf("%f\n",r[z]); } } the output: 0.634000 1.634000 8.634000 27

问题 I am having a problem with precision of a double after performing some operations on a converted string to double. #include <iostream> #include <sstream> #include <math.h> using namespace std; // conversion function void convert(const char * a, const int i, double &out) { double val; istringstream in(a); in >> val; cout << "char a -- " << a << endl; cout << "val ----- " << val << endl; val *= i; cout << "modified val --- " << val << endl; cout << "FMOD ----- " << fmod(val, 1) << endl; out =

问题 I am trying to affect the translation of a 3D model using some UI buttons to shift the position by 0.1 or -0.1. My model position is a three dimensional float so simply adding 0.1f to one of the values causes obvious rounding errors. While I can use something like BigDecimal to retain precision, I still have to convert it from a float and back to a float at the end and it always results in silly numbers that are making my UI look like a mess. I could just pretty the displayed values but the

问题 I need to represent an IEEE 754-1985 double (64-bit) floating point number in a human-readable textual form, with the condition that the textual form can be parsed back into exactly the same (bit-wise) number. Is this possible/practical to do without just printing the raw bytes? If yes, code to do this would be much appreciated. 回答1: Best option: Use the C99 hexadecimal floating point format: printf("%a", someDouble); Strings produced this way can be converted back into double with the C99

问题 I store some numbers in a MySQL using the ORM of SQLAlchemy. When I fetch them afterward, they are truncated such that only 6 significant digits are conserved, thus losing a lot of precision on my float numbers. I suppose there is an easy way to fix this but I can't find how. For example, the following code: import sqlalchemy as sa from sqlalchemy.pool import QueuePool import sqlalchemy.ext.declarative as sad Base = sad.declarative_base() Session = sa.orm.scoped_session(sa.orm.sessionmaker())

问题 I have a function that computes a point in 3d spaced based on a value in range [0, 1] . The problem I'm facing is, that a binary floating-point number cannot represent exactly 1. The mathematical expression that is evaluated in the function is able to compute a value for t=1.0 , but the value will never be accepted by the function because it checks if for the range before computing. curves_error curves_bezier(curves_PointList* list, curves_Point* dest, curves_float t) { /* ... */ if (t < 0 ||

问题 This question already has answers here : ruby: converting from float to integer in ruby produces strange results (3 answers) Closed last year . When I add 0.1+0.2 I am getting 0.30000000000000004 but when I add the same number in ruby 1.8.7 I am getting the correct answer 0.3 . I get 0.3 by rounding but I just want to get 0.3 on ruby 1.9.2 by adding 0.1 and 0.2 回答1: You need bigdecimal for this to make work. (BigDecimal('0.1') + BigDecimal("0.2")).to_f See below link: http://redmine.ruby-lang