floating-accuracy

I've always assumed, that numpy uses a kind of pairwise-summation , which ensures high precision also for float32 - operations: import numpy as np N=17*10**6 # float32-precision no longer enough to hold the whole sum print(np.ones((N,1),dtype=np.float32).sum(axis=0)) # [17000000.], kind of expected However, it looks as if a different algorithm is used if the matrix has more than one column: print(np.ones((N,2),dtype=np.float32).sum(axis=0)) # [16777216. 16777216.] the error is just to big print(np.ones((2*N,2),dtype=np.float32).sum(axis=0)) # [16777216. 16777216.] error is bigger Probably sum

I have a small 3D vector class in C# 3.0 based on struct that uses double as basic unit. An example: One vector's y-value is -20.0 straight I subtract a vector with an y-value of 10.094999999999965 The value for y I would expect is -30.094999999999963 (1) Instead I get -30.094999313354492 (2) When I'm doing the whole computation in one single thread, I get (1). Also the debugger and VS quick-watch returns (1). But, when I run a few iterations in one thread and then call the function from a different thread, the result is (2). Now, the debugger returns (2) as well! We have to keep in mind the

Here's a little gem directly from my VBE (MS Excel 2007 VBA): ?clng(150*0.85) 127 x = 150*0.85 ?clng(x) 128 Can anybody explain this behaviour? IMHO the first expression should yield 128 (.5 rounded to nearest even), or at least should both results be equal. Rick Regan I think wqw is right, but I'll give the details. In the statement clng(150 * 0.85) , 150 * 0.85 is calculated in extended-precision: 150 = 1.001011 x 2^7 0.85 in double precision = 1.1011001100110011001100110011001100110011001100110011 x 2^-1 Multiply these by hand and you get 1

I'm using a float constant and setting a objects private float variable to the float constant below, but when the object outputs the value it was set to, it's rounding up the last digit in the float. private final float RF_FREQUENCY = 956.35625f; Object o = new Object(); o.setRFFrequency(RF_FREQUENCY); System.out.println(o.getRFFrequency); Output: 956.35626 The variable in the object is declared as protected float rfFrequency; and below are the getters and setters. public float getRFFrequency() { return rfFrequency; } public void setRFFrequency(float value) { this.rfFrequency = value; } Any