Is there in PySpark a parameter equivalent to scikit-learn's sample_weight?

问题

I am currently using the SGDClassifier provided by the scikit-learn library. When I use the fit method I can set the sample_weight parameter:

Weights applied to individual samples. If not provided, uniform weights are assumed. These weights will be multiplied with class_weight (passed through the constructor) if class_weight is specified

I want to switch to PySpark and to use the LogisticRegression class. Anyway I cannot find a parameter similar to sample_weight. There is a weightCol parameter but I think it does something different.

Do you have any suggestion?

回答1:

There is a weightCol parameter but I think it does something different.

On the contrary, weightCol of Spark ML does exactly that; from the docs (emphasis added):

weightCol = Param(parent='undefined', name='weightCol', doc='weight column name. If this is not set or empty, we treat all instance weights as 1.0.')

来源：https://stackoverflow.com/questions/56949825/is-there-in-pyspark-a-parameter-equivalent-to-scikit-learns-sample-weight