python sklearn non linear svm penalty

问题

I am using Python 2.7 with sklearn and using sklearn.svm.SVC with rbf kernel and suffer from over fitting.

I tried using the C and Gamma as explained here and it did not do the trick

If I understand correctly C and gamma are not l1 and l2 penalty, because C is the penalty for classifying wrong and gamma is the generalization parameter with respect to the data samples. i am looking for something that will penalize the model for complexity like l1 and l2.

i want to use regularization and l1 or l2 penalty, I found some examples here but when i try to use the penalty parameter with the SVC it throws and error.

svr_rbf = sklearn.svm.SVC(kernel='rbf', cache_size=1,  class_weight={1:100}, penalty='l1')

and get the error

traceback (most recent call last):
  File "/home/thebeancounter/PycharmProjects/nlp4/try.py", line 235, in <module>
    svr_rbf = SVC(kernel='rbf', cache_size=1,  class_weight={1:100}, penalty='l1')
TypeError: __init__() got an unexpected keyword argument 'penalty'

I know that the example is not good for this case, and i am trying to use penalty that is absent in the SVC, i am looking for the right way to do it and this is the closest i found.

looking at the docs for the SVC class i see that it has no penalty attribute

how can i use l1 and l2 penalty with rbf kernel svm or if i can't how else can i try to prevent over fitting with that model?

回答1:

If i am not mistaken l1 and l2 regularization can only be used with linear systems i.e. the linear kernel, and thus the development of different regularization methods for non linear kernels. Furthermore as the number of dimensions increases the l1 and l2 regularizations, become less and less effective. It is a result of the Curse of Dimensionality and that is in part why different regularization methods for use in these kinds of feature spaces have been developed.

I have to agree with @Vivek Kumar that utilizing a grid search method to tune your gamma and C values is going to be the best approach to avoid overfitting of the SVM, given your kernel of choice.

来源：https://stackoverflow.com/questions/43407896/python-sklearn-non-linear-svm-penalty