Specificity in scikit learn

Question

I need specificity for my classification which is defined as : TN/(TN+FP)

I am writing a custom scorer function :

from sklearn.         


        

Answer 1

First of all you need to know that:

DummyClassifier(strategy='most_frequent'...

Will give you classifier which returns most frequent label from your training set. It doesn't even take into consideration samples in X. You can pass anything instead of ground_truth in this line:

clf_dummy = clf_dummy.fit(ground_truth, p)

result of training, and predictions will stay same, because majority of labels inside p is label "0".

Second thing that you need to know: make_scorer returns function with interface scorer(estimator, X, y) This function will call predict method of estimator on set X, and calculates your specificity function between predicted labels and y.

So it calls clf_dummy on any dataset (doesn't matter which one, it will always return 0), and returns vector of 0's, then it computes specificity loss between ground_truth and predictions. Your predictions is 0 because 0 was majority class in training set. Your score is equals 1 because there is no false positive predictions.

I corrected your code, to add more convenience.

from sklearn.dummy import DummyClassifier
clf_dummy = DummyClassifier(strategy='most_frequent', random_state=0)
X = [[0],[0],[1],[0],[1],[1],[1],[0],[0],[1],[0],[0],[1]]
p  = [0,0,0,1,0,1,1,1,1,0,0,1,0]
clf_dummy = clf_dummy.fit(X, p)
score(clf_dummy, X, p)