cross-validation

I am currently trying to evaluate a tslm-model using timeseries cross validation. I want to use a fixed model (without parameter reestimation) an look at the 1 to 3 step ahead horizon forecasts for the evaluation period of the last year. I have trouble to get tsCV and tslm from the forecast-library to work well together. What am I missing? library(forecast) library(ggfortify) AirPassengers_train <- head(AirPassengers, 100) AirPassengers_test <- tail(AirPassengers, 44) ## Holdout Evaluation n_train <- length(AirPassengers_train) n_test <- length(AirPassengers_test) pred_train <- ts(rnorm(n

I would like to build a GBM model with H2O. My data set is imbalanced, so I am using the balance_classes parameter. For grid search (parameter tuning) I would like to use 5-fold cross validation. I am wondering how H2O deals with class balancing in that case. Will only the training folds be rebalanced? I want to be sure the test-fold is not rebalanced. Thank you. In class imbalance settings, artificially balancing the test/validation set does not make any sense: these sets must remain realistic , i.e. you want to test your classifier performance in the real world setting, where, say, the

In MATLAB the method splitEachLabel of an imageDatastore object splits an image data store into proportions per category label. How can one split an image data store for training using cross-validation and using the trainImageCategoryCalssifier class? I.e. it's easy to split it in N partitions, but then some sort of _mergeEachLabel_ functionality is needed to be able to train a classifier using cross-validation. Or is there another way of achieving that? Regards, Elena I stumbled on the same issue recently. Not sure if there is anyone still looking for a possible solution to this. I ended up

I know that I can use a CrossValidator to tune a single model. But what is the suggested approach for evaluating different models against each other? For example, say that I wanted to evaluate a LogisticRegression classifier against a LinearSVC classifier using CrossValidator . After familiarizing myself a bit with the API, I solved this problem by implementing a custom Estimator that wraps two or more estimators it can delegate to, where the selected estimator is controlled by a single Param[Int] . Here is the actual code: import org.apache.spark.ml.Estimator import org.apache.spark.ml.Model