precision

问题 I am trying to determine what the maximum precision for a double is. In the comments for the accepted answer in this link Retain precision with double in Java @PeterLawrey states max precision in 15. How do you determine this ? 回答1: @PeterLawrey states max precision in 15. That's actually not what he stated at all. What he stated was: double has 15 decimal places of accuracy and he is wrong. They have 15 decimal digits of accuracy. The number of decimal digits in any number is given by its

问题 I am aware of how floating point precision works in the regular cases, but I stumbled on an odd situation in my C# code. Why aren't result1 and result2 the exact same floating point value here? const float A; // Arbitrary value const float B; // Arbitrary value float result1 = (A*B)*dt; float result2 = (A*B); result2 *= dt; From this page I figured float arithmetic was left-associative and that this means values are evaluated and calculated in a left-to-right manner. The full source code

问题 How is it that Java's BigDecimal can be this painful? Double d = 13.3D; BigDecimal bd1 = new BigDecimal(d); BigDecimal bd2 = new BigDecimal(String.valueOf(d)); System.out.println("RESULT 1: "+bd1.toString()); System.out.println("RESULT 2: "+bd2.toString()); RESULT 1: 13.300000000000000710542735760100185871124267578125 RESULT 2: 13.3 Is there any situation where Result 1 would be desired? I know that Java 1.5 changed the toString() method but was this the intended consequence? Also I realise

问题 I am confused about what max_digits10 represents. According to its documentation, it is 0 for all integral types. The formula for floating-point types for max_digits10 looks similar to int 's digits10 's. 回答1: To put it simple, digits10 is the number of decimal digits guaranteed to survive text → float → text round-trip. max_digits10 is the number of decimal digits needed to guarantee correct float → text → float round-trip. There will be exceptions to both but these values give the minimum

问题 In C#. I have a double (which I've extracted from a database) that has 20 digit precision. In Visual Studio (using QuickWatch) I can see the value of the double to be = 0.00034101243963859839. I want to display this value in a textbox and then have it be the same value when I take it out and convert it back into a double. But I always lose the last two digits I've tried the following: double d = 0.00034101243963859839; string s = d.ToString(); string s2 = d.ToString("F20"); string s3 = d

问题 Closed. This question is off-topic. It is not currently accepting answers. Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed last year . I saw on StackOverflow a "point in polygon" raytracing algorithm that I implemented in my PHP Code. Most of the time, it works well, but in some complicated cases, with complex polygons and vicious points, it fails and it says that point in not in polygon when it is. For example: You will find here my Polygon and

问题 Are there data types with better precision than float? 回答1: Decimal datatype Unlike hardware based binary floating point, the decimal module has a user alterable precision (defaulting to 28 places) which can be as large as needed for a given problem. If you are pressed by performance issuses, have a look at GMPY 回答2: Python's built-in float type has double precision (it's a C double in CPython, a Java double in Jython). If you need more precision, get NumPy and use its numpy.float128 . 回答3:

问题 It's important to note that I'm not looking for a rounding function. I am looking for a function that returns the number of decimal places in an arbitrary number's simplified decimal representation. That is, we have the following: decimalPlaces(5555.0); //=> 0 decimalPlaces(5555); //=> 0 decimalPlaces(555.5); //=> 1 decimalPlaces(555.50); //=> 1 decimalPlaces(0.0000005); //=> 7 decimalPlaces(5e-7); //=> 7 decimalPlaces(0.00000055); //=> 8 decimalPlaces(5.5e-7); //=> 8 My first instinct was to

问题 Recently I have had to serialize a double into text, and then get it back. The value seems to not be equivalent: double d1 = 0.84551240822557006; string s = d1.ToString("R"); double d2 = double.Parse(s); bool s1 = d1 == d2; // -> s1 is False But according to MSDN: Standard Numeric Format Strings, the "R" option is supposed to guarantee round-trip safety. The round-trip ("R") format specifier is used to ensure that a numeric value that is converted to a string will be parsed back into the same

问题 This question arose out of something strange that I noticed after investigating this question further... I always understood MATLAB variables to be double-precision by default. So, if I were to do something like declare a variable with 20 digits after the decimal point: >> num = 2.71828182845904553488; >> class(num) % Display the variable type ans = double I would expect the last 4 digits to be ignored, since the floating-point relative accuracy is on the order of 10 -16 : >> eps(num) ans = 4