Large Dataset Polynomial Fitting Using Numpy

Question

I\'m trying to fit a second order polynomial to raw data and output the results using Matplotlib. There are about a million points in the data set that I\'m trying to fit.

Answer 1

The array to plot must be sorted. Here is a comparisson between plotting a sorted and an unsorted array. The plot in the unsorted case looks completely distorted, however, the fitted function is of course the same.

        2
-3.496 x + 2.18 x + 17.26

import matplotlib.pyplot as plt
import numpy as np; np.random.seed(0)

x = (np.random.normal(size=300)+1)
fo = lambda x: -3*x**2+ 1.*x +20. 
f = lambda x: fo(x) + (np.random.normal(size=len(x))-0.5)*4
y = f(x)

fig, (ax, ax2) = plt.subplots(1,2, figsize=(6,3))
ax.scatter(x,y)
ax2.scatter(x,y)

def fit(ax, x,y, sort=True):
    z = np.polyfit(x, y, 2)
    fit = np.poly1d(z)
    print(fit)
    ax.set_title("unsorted")
    if sort:
        x = np.sort(x)
        ax.set_title("sorted")
    ax.plot(x, fo(x), label="original func", color="k", alpha=0.6)
    ax.plot(x, fit(x), label="fit func", color="C3", alpha=1, lw=2.5  )  
    ax.legend()


fit(ax, x,y, sort=False)

fit(ax2, x,y, sort=True) 


plt.show()