What is a good range of values for the svm.SVC() hyperparameters to be explored via GridSearchCV()?

Question

I am running into the problem that the hyperparameters of my svm.SVC() are too wide such that the GridSearchCV() never gets completed! One idea is to u

Answer 1

To search for hyperparameters, it is always better to understand what each of them is doing...

C : float, optional (default=1.0)
    Penalty parameter C of the error term.

You should try to change it by order of magnitude (0, 0.1, 1, 10, 100) and maybe then reduce your search between magnitude but I don't think it will improve that much your model.

degree : int, optional (default=3)
   Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.

Here you should change the way you are doing your grid search because as the documentation suggests, degree is only used for polynomial kernel, so you will waste time looking for each degree when using the 'rbf' kernel. Other point is that using two many degrees will just overfit your data. Here use something like (1, 2, 3, 4, 5)

Same remark for coef0 because it is only used with 'poly' kernel

tol : float, optional (default=1e-3)
   Tolerance for stopping criterion.

I would not touch that, your range of value doesn't really make any sense.

I'm not that familiar with the gamma parameter.

So use this representation instead of yours (http://scikit-learn.org/stable/modules/grid_search.html#exhaustive-grid-search):

param_grid = [
 {'C': [1, 10, 100, 1000], 'kernel': ['linear']},
 {'C': [1, 10, 100, 1000], 'gamma': [0.001, 0.0001], 'kernel': ['rbf']},
]

And try to understand what each of those parameters mean:

http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html