Pyspark .toPandas(): 'Unsupported class file major version 55'

问题

I have a Pyspark dataframe that I'm converting to a Pandas df so that I can use seaborn on it. And I get the error that: IllegalArgumentException: 'Unsupported class file major version 55'

I tried what was suggested here, so that my Windows JAVA_HOME now reads:

C:\Users\XXXX\Downloads\jdk-11.0.1 -v 1.8

and tried restarting Jupyter but to no avail.

Full output below:

IllegalArgumentException                  Traceback (most recent call last)
<ipython-input-6-ac86121ca6f3> in <module>
      1 sns.set(style="white")
----> 2 df3 =df2.toPandas()
      3 g = sns.lineplot(x="TransactionDate", y="Amount", data=df3)
      4 display(g.fig)

~\Documents\2_PersonalWork\myenv\lib\site-packages\pyspark\sql\dataframe.py in toPandas(self)
   2140 
   2141         # Below is toPandas without Arrow optimization.
-> 2142         pdf = pd.DataFrame.from_records(self.collect(), columns=self.columns)
   2143 
   2144         dtype = {}

~\Documents\2_PersonalWork\myenv\lib\site-packages\pyspark\sql\dataframe.py in collect(self)
    531         """
    532         with SCCallSiteSync(self._sc) as css:
--> 533             sock_info = self._jdf.collectToPython()
    534         return list(_load_from_socket(sock_info, BatchedSerializer(PickleSerializer())))
    535 

~\Documents\2_PersonalWork\myenv\lib\site-packages\py4j\java_gateway.py in __call__(self, *args)
   1255         answer = self.gateway_client.send_command(command)
   1256         return_value = get_return_value(
-> 1257             answer, self.gateway_client, self.target_id, self.name)
   1258 
   1259         for temp_arg in temp_args:

~\Documents\2_PersonalWork\myenv\lib\site-packages\pyspark\sql\utils.py in deco(*a, **kw)
     77                 raise QueryExecutionException(s.split(': ', 1)[1], stackTrace)
     78             if s.startswith('java.lang.IllegalArgumentException: '):
---> 79                 raise IllegalArgumentException(s.split(': ', 1)[1], stackTrace)
     80             raise
     81     return deco

IllegalArgumentException: 'Unsupported class file major version 55'

来源：https://stackoverflow.com/questions/53989432/pyspark-topandas-unsupported-class-file-major-version-55

标签

pyspark