SciPy interpolation of large matrix

Question

I have a ndarray (Z) with some 500000 elements on a rectangular grid (X, Y).

Now I want to interpolate values at some 100 locations in x,y which are not necessarily

Answer 1

edit: Whoops. Just realized the OP suggested this solution in the question!

I don't know why the interpolation routines take so much time and memory to find the knots of structured data, but since you are only using a small parts of the full grid, you could break up your interpolation into patches to make things more efficient.

from scipy import interpolate
import numpy as np

def my_interp(X, Y, Z, x, y, spn=3):
    xs,ys = map(np.array,(x,y))
    z = np.zeros(xs.shape)
    for i,(x,y) in enumerate(zip(xs,ys)):
        # get the indices of the nearest x,y
        xi = np.argmin(np.abs(X[0,:]-x))
        yi = np.argmin(np.abs(Y[:,0]-y))
        xlo = max(xi-spn, 0)
        ylo = max(yi-spn, 0)
        xhi = min(xi+spn, X[0,:].size)
        yhi = min(yi+spn, Y[:,0].size)
        # make slices of X,Y,Z that are only a few items wide
        nX = X[xlo:xhi, ylo:yhi]
        nY = Y[xlo:xhi, ylo:yhi]
        nZ = Z[xlo:xhi, ylo:yhi]
        intp = interpolate.interp2d(nX, nY, nZ)
        z[i] = intp(x,y)[0]
    return z

N = 1000
X,Y = np.meshgrid(np.arange(N), np.arange(N))
Z = np.random.random((N, N))

print my_interp(X, Y, Z, [13.2, 999.9], [0.01, 45.3])

Answer 2

As your data is on a grid, you can use RectBivariateSpline.

To work with complex numbers, you can interpolate data.real and data.imag separately (the FITPACK routines IIRC don't handle complex data).