Multiclass ROC curves in R

问题

I'm new to the concept of ROC curves. I've tried to understand it by reading a few tutorials on the web. I found a really good example here in python which was helpful.

I want to plot a ROC curve for multiclass classifier that I built(in Python). However, Most of the solutions on the web are for 2 class problems and not multiclass.

However, I finally found "multiclass.roc" function in pROC package in R which does multiclass ROC curve plotting.

The following is a simple example:

library(pROC)
data(aSAH)
multiclass.roc(aSAH$gos6, aSAH$s100b)

However, I don't understand how to interpret it (as I don't know R).

Can anyone please point out what aSAH$s100b variable refers to? I can say aSAH$gos6 points to different classes of data in aSAH dataset.

回答1:

Additionally, you might want to look at Fawcett, 2006 for a very nice guide to understanding and implementing ROC plots. He also addresses multi-class ROC and AUC, as well as points to additional resources.

回答2:

I know this is an old question but for completeness, and for future visitors I will add a brief list of examples using the pROC::multiclass.roc() function. When it comes to the software requirements, the only one is to make sure that the names of the columns of the predictions matrix match the true classes (real_values).

The first example generates random predictions. The second one generates a better prediction. The third one generates the perfect prediction (i.e. always assigning the highest probability to the true class.)

I printed head(obj) the results, therefore no need for R understanding is required to follow up the examples.

Technical details of the computations used by pROC::multiclass.roc() can be found at Hand & Till (2001): A simple generalization of the area under the ROC curve for multiple class classification problems. Machine learning, 45(2), 171-186.

library(pROC)
set.seed(42)
head(real_values)
real_values <- matrix( c("class1", "class2", "class3"), nc=1 )

# [,1]    
# [1,] "class1"
# [2,] "class2"
# [3,] "class3"
############################
#### Random predictions ####
############################
random_preds <- matrix(rbeta(3*3,2,2), nc=3)
random_preds <- sweep(random_preds, 1, rowSums(a1), FUN="/")
colnames(random_preds) <- c("class1", "class2", "class3")


head(random_preds)

#       class1    class2    class3
# [1,] 0.3437916 0.6129104 0.4733117
# [2,] 0.6016169 0.4700832 0.9364681
# [3,] 0.6741742 0.8677781 0.4823129

multiclass.roc(real_values, random_preds)
#Multi-class area under the curve: 0.1667

############################
#### Better Predictions ####
############################

better_preds <- matrix(c(0.75,0.15,0.5,
                         0.15,0.5,0.75,
                         0.15,0.75,0.5), nc=3)
colnames(better_preds) <- c("class1", "class2", "class3")

head(better_preds)

#       class1 class2 class3
# [1,]   0.75   0.15   0.15
# [2,]   0.15   0.50   0.75
# [3,]   0.50   0.75   0.50

multiclass.roc(real_values, better_preds)
#Multi-class area under the curve: 0.6667


#############################
#### Perfect Predictions ####
#############################
perfect_preds <- matrix(c(0.75,0.15,0.5,
                          0.15,0.75,0.5,
                          0.15,0.5,0.75), nc=3)
colnames(perfect_preds) <- c("class1", "class2", "class3")
head(perfect_preds)

multiclass.roc(real_values, perfect_preds)
#Multi-class area under the curve: 1

来源：https://stackoverflow.com/questions/11424112/multiclass-roc-curves-in-r

标签

machine-learning

artificial-intelligence

classification