pickle.PicklingError: args[0] from __newobj__ args has the wrong class with hadoop python

Question

I am trying to I am tring to delete stop words via spark,the code is as follow

from nltk.corpus import stopwords
from pyspark.context import SparkContext
fro         


        

Answer 1

Probably, it's just because you are defining the stopwords.words('english') every time on the executor. Define it outside and this would work.