How to convert pyspark.rdd.PipelinedRDD to Data frame with out using collect() method in Pyspark?

Question

I have pyspark.rdd.PipelinedRDD (Rdd1). when I am doing Rdd1.collect(),it is giving result like below.

 [(10, {3: 3.616726727464709         


        

Answer 1

Ensure a spark session is created first:

sc = SparkContext()
spark = SparkSession(sc)

I found this answer when I was trying to solve this exact issue.
'PipelinedRDD' object has no attribute 'toDF' in PySpark