toPandas() error using pyspark: 'int' object is not iterable

Question

I have a pyspark dataframe and I am trying to convert it to pandas using toPandas(), however I am running into below mentioned error.

I tried different options

Answer 1

Our custom repository of libraries had a package for pyspark which was clashing with the pyspark that is provided by the spark cluster and somehow having both works on Spark shell but does not work on a notebook.
So, renaming the pyspark library in the custom repository resolved the issue!