precision

问题 I need to compare two datetime values to determine equality(exactly the same),using minute precision.Would this be the best way to do it? My dates could have seconds and milliseconds, but i want to consider only down till minutes. where (Math.Abs(datetime1.Subtract(datetime2).TotalMinutes) == 0) 回答1: Checking whether Math.Abs(diff.TotalMinutes) == 0 won't do it, no - that's checking whether they're exactly the same. Are you trying to check whether they have the same minute, or whether they're

问题 I'm trying to generate an array of integers contains randoms adding up to a particular value. Here's my code: private long[] getRandoms(long size , long sum) throws Exception { double iniSum = 0; System.out.println("sum = " + sum); long[] ret = new long[(int) size]; for (int i = 0 ; i < ret.length; i++) { ret[i] = randomInRange(1, sum); iniSum += ret[i]; } double finSum = 0; for (int i = 0 ; i < ret.length; i++) { ret[i] = Math.round((sum * ret[i]) / iniSum); System.out.println("ret[" + i +"]

问题 I was wondering whether, under specific conditions, it is possible to remove floating point errors without resorting to arbitrary-precision datatypes. The problem is the usual one. The language is Ruby, but it holds in any language: f = 1829.82 => 1829.82 f / 12.0 => 152.485 (f / 12.0).round(2) => 152.48 Why not 152.49? Because due to the finite precision of floating points: format("%18.14f", f) => "1829.81999999999994" format("%18.14f", f / 12.0) => "152.48499999999999" So the rounding is

问题 This question already has answers here : Closed 7 years ago . Possible Duplicate: Java += operator In Java, this is not valid (doesn't compile), as expected: long lng = 0xffffffffffffL; int i; i = 5 + lng; //"error: possible loss of magnitude" But this is perfectly fine (?!) long lng = 0xffffffffffffL; int i = 5; i += lng; //compiles just fine This is obviously a narrowing operation, that can possibly exceed the int range. So why doesn't the compiler complain? 回答1: i += lng; compound

问题 I have a function getSlope which takes as parameters 4 doubles and returns another double calculated using this given parameters in the following way: double QSweep::getSlope(double a, double b, double c, double d){ double slope; slope=(d-b)/(c-a); return slope; } The problem is that when calling this function with arguments for example: getSlope(2.71156, -1.64161, 2.70413, -1.72219); the returned result is: 10.8557 and this is not a good result for my computations. I have calculated the

问题 Erlang (and by extension Elixir) supports floating-point numbers. Some possible Floats: 1.2345 1.0e10 1.0e-42 Erlang supports NaN ( nan. in Erlang) (I am however yet to discover a method that outputs nan itself). However, Erlang does not have support for Infinity . While common standards like IEEE-754 state that one should return Infinity when doing things like 1.0/0.0 , instead, Erlang throws a bad arithmetic error . The same happens when attempting to make floats that are 'too large' like 1

问题 I did some testing with floating point calculations to minimize the precision loss. I stumbled across a phenomen I want to show here and hopefully get an explanation. When I write print 1.0 / (1.0 / 60.0) the result is 60.0024000960 When I write the same formula and do explicit casting to float print cast(1.0 as float) / (cast(1.0 as float) / cast(60.0 as float)) the result is 60 Until now I thought that numeric literals with decimal places are automatically treated as float values with the

问题 I have the following code in FORTRAN 77: REAL*8 :: dm dm=1.-1.E-12 write(6,*) 'dm: ', dm I get: dm: 1 Is this OK? I would like to get dm=0.999999999999 回答1: As stated in a comment, you need to specify the precision of the constants. Also, real*8 is obsolete. (Was it always an extension?) Here is a modern way to write this, using the ISO Fortran Environment to obtain a 64-bit real type and using that type both in the declaration and in the constants. use ISO_FORTRAN_ENV real (real64) :: dm dm

问题 Is there any way to generate pseudo-random numbers to less precision and thus speed the process up? Another thing is that I know it saves time if random numbers are generated all at once (e.g. rand(100,1000) ), instead of one by one. Could someone explain why this is true? 回答1: MATLAB actually implements more than one random number generator. They differ significantly in terms of execution time and in terms of "randomness" (I think, but I didn't verify). However, I understand from your

问题 Is there any way to generate pseudo-random numbers to less precision and thus speed the process up? Another thing is that I know it saves time if random numbers are generated all at once (e.g. rand(100,1000) ), instead of one by one. Could someone explain why this is true? 回答1: MATLAB actually implements more than one random number generator. They differ significantly in terms of execution time and in terms of "randomness" (I think, but I didn't verify). However, I understand from your