Create single row dataframe from list of list PySpark

Question

I have a data like this data = [[1.1, 1.2], [1.3, 1.4], [1.5, 1.6]] I want to create a PySpark dataframe

I already use

dataframe = SQLCo         


        

Answer 1

You should use the Vector Assembler function, from your code I guess you are doing this to train a machine learning model, and vector assembler works the best for that case. You can also add the assembler in the pipeline.

assemble_feature=VectorAssembler(inputCol=data.columns,outputCol='features')
pipeline=Pipeline(stages=[assemble_feature])
pipeline.fit(data).transform(data)