caret train method not working (something is wrong for all accuracy results) for outcomes with >2 categories

问题

Hi I know someone asked similar issues before but no clear answer yet (or I tried their solution without success: Caret error using GBM, but not without caret Caret train method complains Something is wrong; all the RMSE metric values are missing )

I tried to use caret training methods to predict the categorical outcomes (online data examples below)

library(mlbench)
data(Sonar)
str(Sonar[, 1:10])

library(caret)
set.seed(998)

Sonar$rand<-rnorm(nrow(Sonar))  ##to randomly create the new 3-category outcome
table(Sonar$rand)
Sonar$Class_new<-ifelse(Sonar$Class=="R","R",ifelse(Sonar$rand>0,"M","H"))
table(Sonar$Class_new)

fitControl <- trainControl(## 10-fold CV
                           method = "repeatedcv",
                           number = 10,
                           ## repeated ten times
                           repeats = 10)

inTraining <- createDataPartition(Sonar$Class_new, p = .75, list = FALSE)
training <- Sonar[ inTraining,]
testing  <- Sonar[-inTraining,]

gbmFit1 <- train(Class_new ~ ., data = training,
                 method = "gbm",
                 trControl = fitControl,
                 verbose = FALSE)

Whenever I used the new class variable (Class_new) which has 3 categories, rather than 2 categories in original Class variable, I got the warnings below. It runs fine with 2 category outcome variables. And it is the same case regardless of the train methods (I tried rf, gbm, svm, all the same)

Something is wrong; all the Accuracy metric values are missing:

    Accuracy       Kappa    
 Min.   : NA   Min.   : NA  
 1st Qu.: NA   1st Qu.: NA  
 Median : NA   Median : NA  
 Mean   :NaN   Mean   :NaN  
 3rd Qu.: NA   3rd Qu.: NA  
 Max.   : NA   Max.   : NA  
 NA's   :9     NA's   :9

Error in train.default(x, y, weights = w, ...) : Stopping
In addition: Warning messages:
1: In train.default(x, y, weights = w, ...) :
The metric "RMSE" was not in the result set. Accuracy will be used instead.
2: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.

Any help on this is greatly appreciated!

回答1:

You need to convert the newly created Class_new to a factor, as follows:

Sonar$Class_new<-ifelse(Sonar$Class=="R","R",ifelse(Sonar$rand>0,"M","H"))
Sonar$Class_new <- factor(Sonar$Class_new)

Also, you may want to remove the variables Class and rand from your training and testing data sets. You can do somthing like:

training <- Sonar[ inTraining, !(names(Sonar) %in% c("Class", "rand"))]
testing <- Sonar[-inTraining, !(names(Sonar) %in% c("Class", "rand"))]

回答2:

I had allowParallel = TRUE in the train function and the machine I was working on did not have multiple cores. After I commented that statement, I did not get the error.

回答3:

Thank howaj for your post. That did work for the data I posted but somehow did not work for another dataset, where everything seems to be the same. But I figured out finally:

Could be a syntax issue here. Instead of using train(y~., data=training, ...), I changed to the train(train$y,train$x, ...) without specifying data=.. explicitly:

train(training[,!names(training)%in%response], training$response ...)

This worked.

回答4:

Instead of passing the formula in the train function, pass values for parameters x, y, method etc

the old way:

modFit = train(data.df$Label ~ ., 
                 data = data.df, 
                method = "rpart", 
                trControl= cntr, 
                tuneLength = 7)

new way:

modFit = train(x = data.df.cols, 
                 y = data.df$Label,
                 method = "rpart",
                   trControl = cntrl, 
                   tuneLength = 7)

Note: x = data.df.cols has all columns except the label, data.df.cols = data.df[,2:ncol(data.df)]

来源：https://stackoverflow.com/questions/33308648/caret-train-method-not-working-something-is-wrong-for-all-accuracy-results-for

标签

r-caret