I've set up numpy.seterr as follows:
np.seterr(invalid='raise', over ='raise', under='raise') And I'm getting the following error:
c = beta[j,i] + oneminusbeta[j,i] FloatingPointError: overflow encountered in double_scalars I've checked what beta[j,i] and oneminusbeta[j,i] are at the point of crash, and these are their values:
beta: -131.340389182 oneminusbeta: 0.0 Please note, this line of addition (beta[j,i] + oneminusbeta[j,i]) has run for thousands of lines in a loop (that performs image classification) before crashing here at this point. How can I deal with this? Is it necessary to change the type of the numpy arrays? This is how I've initialized them:
beta = np.empty([m,n]) oneminusbeta = np.empty([m,n]) Is it possible to cast the individual values before adding them up? Rather than changing the entire array declarations? Or is this even a serious issue? Would it be safe to simply turn off the numpy.seterr configuration and let the calculations go ahead without raising the error?
Edit
Someone suggested below, and I suspected as well, that the values being added shouldn't cause an overflow. Then how can I find out where the overflow is really happening?
This is my code:
epthreshold = 709 enthreshold = -708 f.write("weights["+str(i)+", " + str(j)+"] = math.exp(beta: " +str(beta[j,i])+ " + oneminusbeta: " + str(oneminusbeta[j,i])+")\n" ) c = beta[j,i] + oneminusbeta[j,i] weights[i,j] = math.exp(np.clip(c, enthreshold, epthreshold)) And when I check my log file, this is the line I get:
weights[5550, 13] = math.exp(beta: -131.340389182 + oneminusbeta: 0.0) Edit 2
Here's the rest of my code, where variables n,m and H have already been initialized to integer values:
import numba import numpy as np import statsmodels.api as sm weights = np.empty([n,m]) for curr_n in range(n): for curr_m in range(m): weights[curr_n,curr_m] = 1.0/(n) beta = np.empty([m,n]) oneminusbeta = np.empty([m,n]) for curr_class in range(m): for curr_sample in range(n): beta[curr_class,curr_sample] = 1./m epthreshold = 709 # positive exponential threshold enthreshold = -708 for h in range(H): print "Boosting round %d ... " % h z = np.empty([n,m]) for j in range(m): # computing working responses and weights, Step 2(a)(i) for i in range(no_samples): i_class = y[i] #get the correct class for the current sample if h == 0: z[i,j] = (int(j==i_class) - beta[j,i])/((beta[j,i])*(1. - beta[j,i])) weights[i,j] = beta[j,i]*(1. - beta[j,i]) else: if j == i_class: z[i,j] = math.exp(np.clip(-beta[j,i],enthreshold, epthreshold)) else: z[i,j] = -math.exp(np.clip(oneminusbeta[j,i], enthreshold, epthreshold)) f.write("weights["+str(i)+", " + str(j)+"] = math.exp(beta: " +str(beta[j,i])+ " + oneminusbeta: " + str(oneminusbeta[j,i])+")\n" ) c = beta[j,i] + oneminusbeta[j,i] weights[i,j] = math.exp(np.clip(c, enthreshold, epthreshold)) g_h = np.zeros([1,1]) j = 0 # Calculating regression coefficients per class # building the parameters per j class for y1_w in zip(z.T, weights.T): y1, w = y1_w temp_g = sm.WLS(y1, X, w).fit() # Step 2(a)(ii) if np.allclose(g_h,0): g_h = temp_g.params else: g_h = np.c_[g_h, temp_g.params] j = j + 1 if np.allclose(g,0): g = g_h else: g = g + g_h # Step(2)(a)(iii) # now set g(x), function coefficients according to new formula, step (2)(b) sum_g = g.sum(axis=1) for j in range(m): diff = (g[:,j] - ((1./m) * sum_g)) g[:,j] = ((m-1.)/m) * diff g_per_round[h,:,j] = g[:,j] #Now computing beta, Step 2(c)...." Q = 0. e = 0. for j in range(m): # Calculating beta and oneminusbeta for class j aj = 0.0 for i in range(no_samples): i_class = y[i] X1 = X[i].reshape(1, no_features) g1 = g[:,j].reshape(no_features, 1) gc = g[:,i_class].reshape(no_features, 1) dot = 1. + float(np.dot(X1, g1)) - float(np.dot(X1,gc)) aj = dot sum_e = 0. a_q = [] a_q.append(0.) for j2 in range(m): # calculating sum of e's except for all j except where j=i_class if j2 != i_class: # g based on j2, not necessarily g1? g2 = g[:,j2].reshape(no_features, 1) dot1 = 1. + float(np.dot(X1, g2)) - float(np.dot(X1,gc)) e2 = math.exp(np.clip(dot1,enthreshold, epthreshold)) sum_e = sum_e + e2 a_q.append(dot1) if (int(j==i_class) == 1): a_q_arr = np.array(a_q) alpha = np.array(a_q_arr[1:]) Q = mylogsumexp(f,a_q_arr, 1, 0) sumalpha = mylogsumexp(f,alpha, 1, 0) beta[j,i] = -Q oneminusbeta[j,i] = sumalpha - Q else: alpha = a_q alpha = np.array(alpha[1:]) a_q_arr = np.array(a_q) Q = mylogsumexp(f,a_q_arr, 0, aj) sumalpha = log(math.exp(np.clip(Q, enthreshold, epthreshold)) - math.exp(np.clip(aj, enthreshold, epthreshold))) beta[j,i] = aj - Q oneminusbeta[j,i] = sumalpha - Q and the function mylogsumexp is:
def mylogsumexp(f, a, is_class, maxaj, axis=None, b=None): np.seterr(over="raise", under="raise", invalid="raise") threshold = -sys.float_info.max maxthreshold = sys.float_info.max epthreshold = 709 # positive exponential threshold enthreshold = -708 a = asarray(a) if axis is None: a = a.ravel() else: a = rollaxis(a, axis) if is_class == 1: a_max = a.max(axis=0) else: a_max = maxaj #bnone = " none " if b is not None: a_max = maxaj b = asarray(b) if axis is None: b = b.ravel() else: b = rollaxis(b, axis) a = np.clip(a - a_max, enthreshold, epthreshold) midout = np.sum(np.exp(a), axis=0) midout = 1.0 + np.clip(midout - math.exp(a_max), threshold, maxthreshold) out = np.log(midout) else: a = np.clip(a - a_max, enthreshold, epthreshold) out = np.log(np.sum(np.exp(a))) out += a_max if out == float("inf"): out = maxthreshold if out == float("-inf"): out = threshold return out