We are using LogisticRegressionWithSGD and would like to figure out which of our variables predict and with what significance. Some stats packages (StatsModels) return p-values for each term. A low p-value (< 0.05) indicates a meaningful addition to the model.
How can we get/calculate p-values from LogisticRegressionWithSGD model?
Any help with this is appreciated.
This is a very old question, but some guidance for people coming to it late might be valuable.
LogisticRegressionWithSGD is deprecated. In that version, no true set of "summary" information was provided with the model itself. If you cannot get access to an up-to-date version of pyspark, you will have to calculate the P-values for each of your features yourself. Here is a nice intro to doing that by "hand".
If you can get the current version of pyspark, then you will want to use:
pyspark.mllib.classification.LogisticRegressionWithLBFGS
(docs here)
来源：https://stackoverflow.com/questions/36944121/how-to-calculate-p-values-in-sparks-logistic-regression