What is a threshold in a Precision-Recall curve?

和自甴很熟 提交于 2019-12-31 08:52:12

问题


I am aware of the concept of Precision as well as the concept of Recall. But I am finding it very hard to understand the idea of a 'threshold' which makes any P-R curve possible.

Imagine I have a model to build that predicts the re-occurrence (yes or no) of cancer in patients using some decent classification algorithm on relevant features. I split my data for training and testing. Lets say I trained the model using the train data and got my Precision and Recall metrics using the test data.

But HOW can I draw a P-R curve now? On what basis? I just have two values, one precision and one recall. I read that its the 'Threshold' that allows you to get several precision-recall pairs. But what is that threshold? I am still a beginner and I am unable to comprehend the very concept of the threshold.

I see in so many classification model comparisons like the one below. But how do they get those many pairs?

Model Comparison Using Precision-Recall Curve


回答1:


First of all you should remove the 'roc' and 'auc' tags as precision-recall curve is something different:

ROC Curves:

  • x-axis: False Positive Rate FPR = FP /(FP + TN) = FP / N
  • y-axis: True Positive Rate TPR = Recall = TP /(TP + FN) = TP / P

Precision-Recall Curves:

  • x-axis: Recall = TP / (TP + FN) = TP / P = TPR
  • y-axis: Precision = TP / (TP + FP) = TP / PP

Your cancer detection example is a binary classification problem. Your predictions are based on a probability. The probability of (not) having cancer.

In general, an instance would be classified as A, if P(A) > 0.5 (your threshold value). For this value, you get your Recall-Precision pair based on the True Positives, True Negatives, False Positives and False Negatives.

Now, as you change your 0.5 threshold, you get a different result (different pair). You can already classify a patient as 'has cancer' for P(A) > 0.3. This will decrease Precision and increase Recall. You would rather tell someone that he has cancer even though he has not, to make sure that patients with cancer are sure to get the treatment they need. This represents the intuitive trade-off between TPR and FPR or Precision and Recall or Sensitivity and Specificity.

Let's add these terms as you see them more often common in biostatistics.

  • Sensitivity = TP / P = Recall = TPR
  • Specificity = TN / N = (1 – FPR)

ROC-curves and Precision-Recall curves visualize all these possible thresholds of your classifier.

You should consider these metrics, if accuracy alone is not a suitable quality measure. Classifying all patients as 'does not have cancer' will give you the highest accuracy but the values of your ROC and Precision-Recall curves will be 1s and 0s.



来源:https://stackoverflow.com/questions/46224752/what-is-a-threshold-in-a-precision-recall-curve

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!