How to prepare data into a LibSVM format from DataFrame?

Question

I want to make libsvm format, so I made dataframe to the desired format, but I do not know how to convert to libsvm format. The format is as shown in the figure. I hope that

Answer 1

libsvm datatype features is a sparse vector, u can use pyspark.ml.linalg.SparseVector to solve the problem

a = SparseVector(4, [1, 3], [3.0, 4.0])

def sparsevecfuc(len,index,score):
    """
     args: len int, index array, score array
    """
    return SparseVector(len,index,score)
trans_sparse = udf(sparsevecfuc,VectorUDT())