precision

问题 In 2013 there was a question on converting a big working code from double to quadruple precision: "Converting a working code from double-precision to quadruple-precision: How to read quadruple-precision numbers in FORTRAN from an input file", and the consensus was to declare variables using an adjustable parameter "WP" that specifies the "working precision", instead of having a separate version of the program with variables declared using D+01, and another version using Q+01. This way we can

问题 After getting very excited by what seemed like excellent results from using the MLP within the Weka GUI on my pricing data, I've coded up a bit of Java that uses an MLP with the same parameters. Here is where the fun starts, the results are completely different, I've now found that this appears to be be due to rounding differences. The GUI rounds to 3 dp, my java code rounds to 5 dp. I've looked through the manuals but I can't seem to find an option to force the GUI to use 5dp precision on

问题 I am getting a weird problem while parsing a double value in managed C++. It may be that I am doing something wrong. When I do: double value = 0.006; result = Math::Parse( value) The output of result is 0.006000000000001 . Why it is appending a 1? Also when I go an round the value to 5 decimal places, it fails. I am doing: result2 = Math::Round(result, 5) But result2 is always 0.006000000000001 . What am I doing wrong? 回答1: It is normal. This problem caused by IEEE format of double - in real

问题 I am currently writing a calculator application. I know that a double is not the best choice for good math. Most of the functions in the application have great precision but the ones that don't get super ugly results. My solution is to show users only 12 decimals of precision. I chose 12 because my lowest precision comes from my numerical derive function. The issue I am having is that if I multiply it by a scaler then round then divide by the scaler the precision will most likely be thrown

问题 Here is a pseudocode of my problem. I have an array of IEEE 754 double precision positive numbers. The array can come in a random order but numbers are always the same, just scrambled in their positions. Also these numbers can vary in a very wide range in the valid IEEE range of the double representation. Once I have the list, I initialize a variable: double sum_result = 0.0; And I accumulate the sum on sum_result , in a loop over the whole array. At each step I do: sum_result += my_double

问题 I'm trying to change a format of pd data frame column without changing the type of data. Here is what I have: df = pd.DataFrame({'Age': [24.0, 32.0}]) I'd like to represent Age in 24 32 type or 24.00 32.00 and keep them as floats. Here is what I can do: df['Age'].map('{:,.2f}'.format) But this line changes the type of data to object. I was also trying to apply: ` df = df.style.format({'Age': '{:,.2f}'.format})` but there is something wrong in it. Please help to figure out the right way. 回答1:

问题 How to work with equivalent of __float128 in Python? What precision should I use for decimal.getcontext() ? I mean, is the precision specified in decimal places or bits? from decimal import * getcontext().prec = # 34 or 128 ? Is it possible to set the precision "locally" for a given operation, rather than setting it "globally" with getcontext().prec ? Per Simon Byrne comment, is it even possible to simulate __float128 as defined by IEEE 754 with Decimal ? What other options do I have in

问题 I have this: leg2=strcat('Max Degree :',num2str(adet(1,1)/ch(l))); leg3=strcat('Min Degree :',num2str(adet(1,2)/ch(l))); leg4=strcat('Max Request :',num2str(adet(1,3)/ch(l))); leg5=strcat('Min Request :',num2str(adet(1,4)/ch(l))); leg6=strcat('Max Priority :',num2str(adet(1,5)/ch(l))); leg7=strcat('Random :',num2str(adet(1,6)/ch(l))); leg8=strcat('AICS :',num2str(adet(1,7)/ch(l))); legend(leg2,leg3,leg4,leg5,leg6,leg7,leg8,'Location','SouthWest'); Here, adet(1,1)/ch(l) is a number between 0

问题 I'm in the process of converting a program to C++ from Scilab (similar to Matlab) and I'm required to maintain the same level of precision that is kept by the previous code. Note: Although maintaining the same level of precision would be ideal. It's acceptable if there is some error with the finished result. The problem I'm facing (as I'll show below) is due to looping, so the calculation error compounds rather quickly. But if the final result is only a thousandth or so off (e.g. 1/1000 vs 1

问题 C++11 introduced very useful math functions in the standard like erf and erfc. There are mentions about "guaranteed underflow" for inputs greater or smaller than certain values, but I don't know enough about floating point representation to understand clearly what this means in terms of precision. If this question makes sense; what precision (order of magnitude at least) can I expect from the approximation implemented by the standard library (if it is specified)? 回答1: This depends on the