precision

问题 Can somebody please explain me the following output. I know that it has something to do with floating point precision, but the order of magnitue (difference 1e308) surprises me. 0: high precision > 1e-324==0 [1] TRUE > 1e-323==0 [1] FALSE 1: very unprecise > 1 - 1e-16 == 1 [1] FALSE > 1 - 1e-17 == 1 [1] TRUE 回答1: R uses IEEE 754 double-precision floating-point numbers. Floating-point numbers are more dense near zero. This is a result of their being designed to compute accurately (the

问题 Can somebody please explain me the following output. I know that it has something to do with floating point precision, but the order of magnitue (difference 1e308) surprises me. 0: high precision > 1e-324==0 [1] TRUE > 1e-323==0 [1] FALSE 1: very unprecise > 1 - 1e-16 == 1 [1] FALSE > 1 - 1e-17 == 1 [1] TRUE 回答1: R uses IEEE 754 double-precision floating-point numbers. Floating-point numbers are more dense near zero. This is a result of their being designed to compute accurately (the

问题 The manual of a program written in Fortran 90 says, "All real variables and parameters are specified in 64-bit precision (i.e. real*8 )." According to Wikipedia, single precision corresponds to 32-bit precision, whereas double precision corresponds to 64-bit precision, so apparently the program uses double precision. But what does real*8 mean? I thought that the 8 meant that 8 digits follow the decimal point. However, Wikipedia seems to say that single precision typically provides 6-9 digits

问题 Can a double (of a given number of bytes, with a reasonable mantissa/exponent balance) always fully precisely hold the range of an unsigned integer of half that number of bytes? E.g. can an eight byte double fully precisely hold the range of numbers of a four byte unsigned int? What this will boil down to is if a two byte float can hold the range of a one byte unsigned int. A one byte unsigned int will of course be 0 -> 255. 回答1: An IEEE754 64-bit double can represent any 32-bit integer,

问题 Imagine you have a large array of floating point numbers, of all kinds of sizes. What is the most correct way to calculate the sum, with the least error? For example, when the array looks like this: [1.0, 1e-10, 1e-10, ... 1e-10.0] and you add up from left to right with a simple loop, like sum = 0 numbers.each do |val| sum += val end whenever you add up the smaller numbers might fall below the precision threshold so the error gets bigger and bigger. As far as I know the best way is to sort

问题 I am facing a strange behavior of the round() function: for i in range(1, 15, 2): n = i / 2 print(n, "=>", round(n)) This code prints: 0.5 => 0 1.5 => 2 2.5 => 2 3.5 => 4 4.5 => 4 5.5 => 6 6.5 => 6 I expected the floating values to be always rounded up, but instead, it is rounded to the nearest even number. Why such behavior, and what is the best way to get the correct result? I tried to use the fractions but the result is the same. 回答1: The Numeric Types section documents this behaviour

问题 This question already has answers here : Closed 8 years ago . Possible Duplicates: Big integers in C# C# unlimited significant decimal digits (arbitrary precision) without java I read the question at Arbitrary precision decimals in C#? but I don't have the J# library. I need a library for arbitrary precision decimals with C#. 回答1: Big Decimal: Install the J# runtime (it's free): http://www.microsoft.com/downloads/en/details.aspx?familyid=f72c74b3-ed0e-4af8-ae63-2f0e42501be1&displaylang=en Big

问题 I would like to change the precision in a calculation of R. For example I would like to calculate x^6 with x = c(-2.5e+59, -5.6e+60) . In order to calculate it I should change the precision in R, otherwise the result is Inf , and I don't know how to do it. 回答1: As Livius points out in his comment, this is an issue with R (and in fact, most programming language), with how numbers are represented in binary. To work with extremely large/small floating point numbers, you can use the Rmpfr library

问题 This question already has answers here : Closed 9 years ago . Possible Duplicate: Floating point inaccuracy examples double a = 0.3; std::cout.precision(20); std::cout << a << std::endl; result: 0.2999999999999999889 double a, b; a = 0.3; b = 0; for (char i = 1; i <= 50; i++) { b = b + a; }; std::cout.precision(20); std::cout << b << std::endl; result: 15.000000000000014211 So.. 'a' is smaller than it should be. But if we take 'a' 50 times - result will be bigger than it should be. Why is

问题 This question already has answers here : Is floating point math broken? (31 answers) Closed 2 years ago . $a = '35'; $b = '-34.99'; echo ($a + $b); Results in 0.009999999999998 What is up with that? I wondered why my program kept reporting odd results. Why doesn't PHP return the expected 0.01? 回答1: Because floating point arithmetic != real number arithmetic. An illustration of the difference due to imprecision is, for some floats a and b , (a+b)-b != a . This applies to any language using