rounding-error

问题 I am using Entity Framework 6.x using the Code First approach on an MVC 5 application. In this particular situation my model (among other things) contains two properties named Latitude and Longitude: [Required, Range(-90, +90)] public decimal Latitude { get; set; } [Required, Range(-180, +180)] public decimal Longitude { get; set; } And when I performed the migration I got something like this CreateTable("ResProperty"), c => new { : Latitude = c.Decimal(nullable: false, precision: 10, scale:

问题 The following program: #include <stdio.h> int main() { double val = 1.0; int i; for (i = 0; i < 10; i++) { val -= 0.2; printf("%g %s\n", val, (val == 0.0 ? "zero" : "non-zero")); } return 0; } Produces this output: 0.8 non-zero 0.6 non-zero 0.4 non-zero 0.2 non-zero 5.55112e-17 non-zero -0.2 non-zero -0.4 non-zero -0.6 non-zero -0.8 non-zero -1 non-zero Can anyone tell me what is causing the error when subtracting 0.2 from 0.2? Is this a rounding error or something else? Most importantly, how

问题 What class should I use for representation of money to avoid most rounding errors? Should I use Decimal , or a simple built-in number ? Is there any existing Money class with support for currency conversion that I could use? Any pitfalls that I should avoid? 回答1: I assume that you talking about Python. http://code.google.com/p/python-money/ "Primitives for working with money and currencies in Python" - the title is self explanatory :) 回答2: Never use a floating point number to represent money.

问题 What class should I use for representation of money to avoid most rounding errors? Should I use Decimal , or a simple built-in number ? Is there any existing Money class with support for currency conversion that I could use? Any pitfalls that I should avoid? 回答1: I assume that you talking about Python. http://code.google.com/p/python-money/ "Primitives for working with money and currencies in Python" - the title is self explanatory :) 回答2: Never use a floating point number to represent money.

问题 This question/issue is maybe related somehow to this topic. If you type: print log10(1e7) you will get 7.0 . print int(log10(1e7)) you will get 7 . However, if you type print log10(1e6) you will get 6.0 . print int(log10(1e6)) you will get 5 . These are probably rounding errors related to log10 and cannot(?) be avoided. Because if you type print sprintf("%.20e",log10(1e6)) gives 5.99999999999999911182e+00 print sprintf("%.20e",log10(1e7)) gives 7.00000000000000000000e+00 You can extend and

问题 Web app coded in PHP with a MySQL database. I have a system which calculates different costs for a number of people when splitting a cost. For example Person A buys something for 10 and Persons B, C, and D should split the cost. The system should therefor register a positive record for person A of 10 and negative records of 10/3 for B, C and D. However, when this is done; B, C and D all have -3.33 after rounding. Which of course doesn't add up to the total of 10. What's the best way of going

问题 I want to convert a time_duration to a DATE format, which is the number of days since 1899, 12, 30. DATE date_from_duration(time_duration td) { double days = td.hours()/24.+td.minutes()/(24.*60.)+td.seconds()/(24.*60.*60.); return days; } This code almost works but gives sometimes rounding errors, f.i the time_duration(1007645, 15, 0) should result in 2014-12-12 00:15:00, but is actually 2014-12-12 00:14:59. The check of DATE is done with this method, stolen from here: ptime pTime_from_DATE

问题 Because the float data type in PHP is inaccurate, and a FLOAT in MySQL takes up more space than an INT (and is inaccurate), I always store prices as INTs, multipling by 100 before storing to ensure we have exactly 2 decimal places of precision. However I believe PHP is misbehaving. Example code: echo "<pre>"; $price = "1.15"; echo "Price = "; var_dump($price); $price_corrected = $price*100; echo "Corrected price = "; var_dump($price_corrected); $price_int = intval(floor($price_corrected));

问题 In follow up to this question, it appears that some numbers cannot be represented by floating point at all, and instead are approximated. How are floating point numbers stored? Is there a common standard for the different sizes? What kind of gotchas do I need to watch out for if I use floating point? Are they cross-language compatible (ie, what conversions do I need to deal with to send a floating point number from a python program to a C program over TCP/IP)? 回答1: As mentioned, the Wikipedia

问题 I wonder if the following should be reported as a bug in gcc implementation of standard library. For all unsigned integers i , if we compare int(std::sqrt(i)) to the actual square root of the integer, the conversion always give the good result. If we do the same with std::cbrt it's not the case : // Problem of rounding of std::cbrt for i from 0 to 100 million // i, exact cbrt(i), result of int(std::cbrt(i)) 2197, 13, 12 17576, 26, 25 24389, 29, 28 140608, 52, 51 185193, 57, 56 195112, 58, 57