cross-validation

I'm trying to do a 10-fold cross validation for some glm models that I have built earlier in R. I'm a little confused about the cv.glm() function in the boot package, although I've read a lot of help files. When I provide the following formula: library(boot) cv.glm(data, glmfit, K=10) Does the "data" argument here refer to the whole dataset or only to the test set? The examples I have seen so far provide the "data" argument as the test set but that did not really make sense, such as why do 10-folds on the same test set? They are all going to give exactly the same result (I assume!).

I'm unable to find a way of performing cross validation on a regression random forest model that I'm trying to produce. So I have a dataset containing 1664 explanatory variables (different chemical properties), with one response variable (retention time). I'm trying to produce a regression random forest model in order to be able to predict the chemical properties of something given its retention time. ID RT (seconds) 1_MW 2_AMW 3_Sv 4_Se 4281 38 145.29 5.01 14.76 28.37 4952 40 132.19 6.29 11 21.28 4823 41 176.21 7.34 12.9 24.92 3840 41 174.24 6.7 13.99 26.48 3665 42 240.34 9.24 15.2 27.08 3591

I have created a custom model in python using scikit-learn, and I want to use cross validation. The class for the model is defined as follows: class MultiLabelEnsemble: ''' MultiLabelEnsemble(predictorInstance, balance=False) Like OneVsRestClassifier: Wrapping class to train multiple models when several objectives are given as target values. Its predictor may be an ensemble. This class can be used to create a one-vs-rest classifier from multiple 0/1 labels to treat a multi-label problem or to create a one-vs-rest classifier from a categorical target variable. Arguments: predictorInstance -- A

I am trying this training modeling using train_test_split and a decision tree regressor: import sklearn from sklearn.model_selection import train_test_split from sklearn.tree import DecisionTreeRegressor from sklearn.model_selection import cross_val_score # TODO: Make a copy of the DataFrame, using the 'drop' function to drop the given feature new_data = samples.drop('Fresh', 1) # TODO: Split the data into training and testing sets using the given feature as the target X_train, X_test, y_train, y_test = train_test_split(new_data, samples['Fresh'], test_size=0.25, random_state=0) # TODO: Create

I am using RandomForestClassifier() with 10 fold cross validation as follows. clf=RandomForestClassifier(random_state = 42, class_weight="balanced") k_fold = StratifiedKFold(n_splits=10, shuffle=True, random_state=42) accuracy = cross_val_score(clf, X, y, cv=k_fold, scoring = 'accuracy') print(accuracy.mean()) I want to identify the important features in my feature space. It seems to be straightforward to get the feature importance for single classification as follows. print("Features sorted by their score:") feature_importances = pd.DataFrame(clf.feature_importances_, index = X_train.columns,