Create RMSLE metric in caret in r

Question

Could someone please help me with the following: I need to change my xgboost training model with caret package to an undefault metric RMSLE. By default caret and xgboost train and measure in RMSE.

Here are the lines of code:

create custom summary function in caret format

custom_summary = function(data, lev = NULL, model = NULL){
out = rmsle(data[, "obs"], data[, "pred"])
names(out) = c("rmsle")
out
}

create control object

control = trainControl(method = "cv",  
                   number = 2,     
                   summaryFunction = custom_summary)

create grid of tuning parameters

grid = expand.grid(nrounds = 100, 
               max_depth = 6,          
               eta = 0.075,     
               gamma = 0, 
               colsample_bytree = 0.4, 
               min_child_weight = 2.25,
               subsample = 1)

cl = makeCluster(3, type="SOCK") #make clusters

registerDoSNOW(cl)  #register clusters

set.seed(1)

train my model

caret4 =  train(price_doc~. - sub_area - id,
                    data=train.train,
                    method="xgbTree",
                    trControl=control, 
                    tuneGrid=grid, 
                    metric="rmsle",
                    maximize = FALSE)

and I keep getting an error: { :task 1 failed - "can not find function "rmsle""

Answer 1

I also encountered the same issue in my project. This is even after loading the Metrics package in memory using the below command.
library(Metrics)

If you see, rmsle function is being called from another function called custom_summary. It is not called directly. So I loaded the Metrics package from within the function custom_summary and it solved the issue for me.

so here, the custom_summary function should look like:

custom_summary = function(data, lev = NULL, model = NULL) {
library(Metrics)
out = rmsle(data[, "obs"], data[, "pred"])
names(out) = c("rmsle")
out
}