问题
Does the scikit-learn Ridge regression include the intercept coefficient in the regularization term, and if so, is there a way to run ridge regression without regularizing the intercept?
Suppose I fit a Ridge Regression:
from sklearn import linear_model
mymodel = linear_model.Ridge(alpha=0.1, fit_intercept=True).fit(X, y)
print mymodel.coef_
print mymodel.intercept_
for some data X, y where X does not include a column of 1's. fit_intercept=True will automatically add an intercept column, and the corresponding coefficient is given by mymodel.intercept_. What I'm unable to figure out is whether this intercept coefficient was part of the regularization summation in the optimization objective.
According to http://scikit-learn.org/stable/modules/linear_model.html, the optimization objective is to minimize with respect to w:
||X*w - y||**2 + alpha* ||w||**2
(using the L2 norm). The second term is the regularization term, and the question is whether it includes the intercept coefficient in the case where we set fit_intercept=True; and if so, how to disable this.
回答1:
The intercept is not penalized. Just try a simple 3 point example with a large intercept.
from sklearn import linear_model
import numpy as np
x=np.array([-1,0,1]).reshape((3,1))
y=np.array([1001,1002,1003])
fit=linear_model.Ridge(alpha=0.1,fit_intercept=True).fit(x,y)
print fit.intercept_
print fit.coef_
The intercept was set to the MLE intercept (1002), while the slope was penalized (.952 instead of 1).
来源：https://stackoverflow.com/questions/26126224/scikit-learn-ridge-regression-with-unregularized-intercept-term