Pandas sparse dataFrame to sparse matrix, without generating a dense matrix in memory

Question

Is there a way to convert from a pandas.SparseDataFrame to scipy.sparse.csr_matrix, without generating a dense matrix in memory?

sc         


        

Answer 1

As of Pandas version 0.25 SparseSeries and SparseDataFrame are deprecated. DataFrames now support Sparse Dtypes for columns with sparse data. Sparse methods are available through sparse accessor, so conversion one-liner now looks like this:

sparse_matrix = scipy.sparse.csr_matrix(df.sparse.to_coo())