floating-accuracy

问题 I am using this code that I have found on another thread which is working fine on mdpi screens: public static float convertDpToPixel(float dp,Context context){ Resources resources = context.getResources(); DisplayMetrics metrics = resources.getDisplayMetrics(); float px = dp * (metrics.densityDpi/160f); return px; } public static float convertPixelsToDp(float px,Context context){ Resources resources = context.getResources(); DisplayMetrics metrics = resources.getDisplayMetrics(); float dp =

问题 I know from previous threads on this topic that using float arithmetic causes precision anomalies. But Interestingly I observed that the same function is behaving in two different ways. Using COUT output is 4 but if I am saving the result into a variable, then result is 3 ! #include <iostream> #include <cmath> using namespace std; #define mod 1000000007 long long int fastPower(long long int a, int n){ long long int res = 1; while (n) { if (n & 1) res = (res * a) % mod; n >>= 1; a = (a * a) %

问题 I came across this following arithmetic problem. But the result is different from normal maths operation, Why is it so? double d1 = 1.000001; double d2 = 0.000001; Console.WriteLine((d1-d2)==1.0); 回答1: I presume you found the question on Jon Skeet's Brainteasers page? The answers are listed and explained here on the same website. For a matter of reference, here's the answer copied from that page. 3) Silly arithmetic Computers are meant to be good at arithmetic, aren't they? Why does this

问题 Why does this: int main(void) { short w = 30; return 1.2 * w; } return 35? 回答1: If you want to get more suitable result, try the following: return 12*w/10 回答2: 1.2 * w is 36.0 . It has the double type meaning it is not represented exactly. Likely it turns out to be slightly less than 36 , maybe 35.99999 so when you return it the fractional part is discarded and the integer part only is returned. That's how you get 35 . P.S. All operations with floating point are not precise. You should expect

问题 While trying to figure out how to round a float like 1.255 to the nearest hundredth, I found something interesting. I'm using gcc 4.4.5 on Debian 6. int x = (1.255 * 100) + 0.5; // gives me back 125 instead of 126. float y = (1.255 * 100) + 0.5; // gives me back 126.000000. Why is is that when I save to an int I get back 125 and not 126 ? In fedora when I save the above expression to an int I get back 126 . Is this a gcc bug in debian ? Any help would be greatly appreciated. Thanks. 回答1:

问题 What sort of stability issues arise or are resolved by using std::pow() ? Will it be more stable (or faster, or at all different) in general to implement a simple function to perform log(n) iterated multiplies if the exponent is known to be an integer? How does std::sqrt(x) compare, stability-wise, to something of the form std::pow(x, k/2) ? Would it make sense to choose the method preferred for the above to raise to an integer power, then multiply in a square root, or should I assume that

问题 I would like to know a good way of checking if a number x is a rational (two integers n,m exist so that x=n/m) in python. In Mathematica, this is done by the function Rationalize[6.75] : 27/4 I assume this question has an answer for a given accuracy. Is there a common algorithm of obtaining these two integers? 回答1: In python >= 2.6 there is a as_integer_ratio method on floats: >>> a = 6.75 >>> a.as_integer_ratio() (27, 4) >>> import math >>> math.pi.as_integer_ratio() (884279719003555,

问题 Imagine the following situation. Amounts are added to a total in a loop: float total = 0; float[] amounts = new float[]{...}; foreach(float amount in amounts) { total += amount; } total , as wel as all the amount s are written to a database. When I calculate SUM(amount) in SQL, it results in a value that differs from total . Moreover, when I do the same calculation in C#, but this time adding the amounts to a value of type double, double total = 0; //the rest of the code is the same as above

问题 MySQL documentation says : The DECIMAL and NUMERIC types store exact numeric data values. These types are used when it is important to preserve exact precision, for example with monetary data. I did this test on that column decimal_column DECIMAL(31,30) . insert into tests (decimal_column) values(1/3); then inspecting what has been stored gives this select * from tests ; > result : 0.333333333000000000000000000000 then reversing the math operation with this query gives this select decimal

问题 Consider the following code: 0.1 + 0.2 == 0.3 -> false 0.1 + 0.2 -> 0.30000000000000004 Why do these inaccuracies happen? 回答1: Binary floating point math is like this. In most programming languages, it is based on the IEEE 754 standard. The crux of the problem is that numbers are represented in this format as a whole number times a power of two; rational numbers (such as 0.1 , which is 1/10 ) whose denominator is not a power of two cannot be exactly represented. For 0.1 in the standard