cross-validation

I'm tinkering with some cross-validation code from the PySpark documentation, and trying to get PySpark to tell me what model was selected: from pyspark.ml.classification import LogisticRegression from pyspark.ml.evaluation import BinaryClassificationEvaluator from pyspark.mllib.linalg import Vectors from pyspark.ml.tuning import ParamGridBuilder, CrossValidator dataset = sqlContext.createDataFrame( [(Vectors.dense([0.0]), 0.0), (Vectors.dense([0.4]), 1.0), (Vectors.dense([0.5]), 0.0), (Vectors.dense([0.6]), 1.0), (Vectors.dense([1.0]), 1.0)] * 10, ["features", "label"]) lr =

I have a small corpus and I want to calculate the accuracy of naive Bayes classifier using 10-fold cross validation, how can do it. Your options are to either set this up yourself or use something like NLTK-Trainer since NLTK doesn't directly support cross-validation for machine learning algorithms . I'd recommend probably just using another module to do this for you but if you really want to write your own code you could do something like the following. Supposing you want 10-fold , you would have to partition your training set into 10 subsets, train on 9/10 , test on the remaining 1/10 , and

Let me start by saying that I have read many posts on Cross Validation and it seems there is much confusion out there. My understanding of that it is simply this: Perform k-fold Cross Validation i.e. 10 folds to understand the average error across the 10 folds. If acceptable then train the model on the complete data set. I am attempting to build a decision tree using rpart in R and taking advantage of the caret package. Below is the code I am using. # load libraries library(caret) library(rpart) # define training control train_control<- trainControl(method="cv", number=10) # train the model

I want to find the parameters of ParamGridBuilder that make the best model in CrossValidator in Spark 1.4.x, In Pipeline Example in Spark documentation, they add different parameters ( numFeatures , regParam ) by using ParamGridBuilder in the Pipeline. Then by the following line of code they make the best model: val cvModel = crossval.fit(training.toDF) Now, I want to know what are the parameters ( numFeatures , regParam ) from ParamGridBuilder that produces the best model. I already used the following commands without success: cvModel.bestModel.extractParamMap().toString() cvModel.params

I'm running GridSearch CV to optimize the parameters of a classifier in scikit. Once I'm done, I'd like to know which parameters were chosen as the best. Whenever I do so I get a AttributeError: 'RandomForestClassifier' object has no attribute 'best_estimator_' , and can't tell why, as it seems to be a legitimate attribute on the documentation . from sklearn.grid_search import GridSearchCV X = data[usable_columns] y = data[target] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0) rfc = RandomForestClassifier(n_jobs=-1,max_features= 'sqrt' ,n_estimators=50

I am trying to run the following code: from sklearn.model_selection import StratifiedKFold X = ["hey", "join now", "hello", "join today", "join us now", "not today", "join this trial", " hey hey", " no", "hola", "bye", "join today", "no","join join"] y = ["n", "r", "n", "r", "r", "n", "n", "n", "n", "r", "n", "n", "n", "r"] skf = StratifiedKFold(n_splits=10) for train, test in skf.split(X,y): print("%s %s" % (train,test)) But I get the following error: ValueError: n_splits=10 cannot be greater than the number of members in each class. I have looked here scikit-learn error: The least populated