Using CategoricalFeaturesInfo with DecisionTreeClassifier method in Spark

Question

I have to use this code:

val dt = new DecisionTreeClassifier().setLabelCol(\"indexedLabel\").setFeaturesCol(\"indexed         


        

Answer 1

You're mixing two different APIs which take different approach to categorical data:

• RDD based o.a.s.mllib which provides required metadata by passing categoricalFeaturesInfo map.
• Dataset (DataFrame) o.a.s.ml which is using column metadata to determine variable types. If you correctly use ML transformers to create features this should be handled automatically for you, otherwise you'll have to provide metadata manually.