问题
This is a follow-up to my previous couple of questions. Here's the code I'm playing with:
import pandas as pd
import matplotlib.pyplot as plt
import scipy.stats as stats
import numpy as np
dictOne = {'Name':['First', 'Second', 'Third', 'Fourth', 'Fifth', 'Sixth', 'Seventh', 'Eighth', 'Ninth'],
"A":[1, 2, -3, 4, 5, np.nan, 7, np.nan, 9],
"B":[4, 5, 6, 5, 3, np.nan, 2, 9, 5],
"C":[7, np.nan, 10, 5, 8, 6, 8, 2, 4]}
df2 = pd.DataFrame(dictOne)
column = 'B'
df2[df2[column] > -999].hist(column, alpha = 0.5)
param = stats.norm.fit(df2[column].dropna()) # Fit a normal distribution to the data
print(param)
pdf_fitted = stats.norm.pdf(df2[column], *param)
plt.plot(pdf_fitted, color = 'r')
I'm trying to make a histogram of the numbers in a single column in the dataframe -- I can do this -- but with an overlaid normal curve...something like the last graph on here. I'm trying to get it working on this toy example so that I can apply it to my much larger dataset for real. The code I've pasted above gives me this graph:
Why doesn't pdf_fitted match the data in this graph? How can I overlay the proper PDF?
回答1:
You should plot the histogram with density=True if you hope to compare it to a true PDF. Otherwise your normalization (amplitude) will be off.
Also, you need to specify the x-values (as an ordered array) when you plot the pdf:
fig, ax = plt.subplots()
df2[df2[column] > -999].hist(column, alpha = 0.5, density=True, ax=ax)
param = stats.norm.fit(df2[column].dropna())
x = np.linspace(*df2[column].agg([min, max]), 100) # x-values
plt.plot(x, stats.norm.pdf(x, *param), color = 'r')
plt.show()
As an aside, using a histogram to compare continuous variables with a distribution is isn't always the best. (Your sample data are discrete, but the link uses a continuous variable). The choice of bins can alias the shape of your histogram, which may lead to incorrect inference. Instead, the ECDF is a much better (choice-free) illustration of the distribution for a continuous variable:
def ECDF(data):
n = sum(data.notnull())
x = np.sort(data.dropna())
y = np.arange(1, n+1) / n
return x,y
fig, ax = plt.subplots()
plt.plot(*ECDF(df2.loc[df2[column] > -999, 'B']), marker='o')
param = stats.norm.fit(df2[column].dropna())
x = np.linspace(*df2[column].agg([min, max]), 100) # x-values
plt.plot(x, stats.norm.cdf(x, *param), color = 'r')
plt.show()
来源：https://stackoverflow.com/questions/56015457/plotting-a-histogram-with-overlaid-pdf