sparse-matrix

问题 I'm trying to implement a matrix-vector Multiplication on GPU (using CUDA). In my C++ code (CPU), I load the matrix as a dense matrix, and then I perform the matrix-vector multiplication using CUDA. I'm also using shared memory to improve the performance. How can I load the matrix in an efficient way, knowing that my matrix is a sparse matrix? Below is my C++ function to load the matrix: int readMatrix( char* filename, float* &matrix, unsigned int *dim = NULL, int majority = ROW_MAJOR ) {

问题 I'm trying to implement a matrix-vector Multiplication on GPU (using CUDA). In my C++ code (CPU), I load the matrix as a dense matrix, and then I perform the matrix-vector multiplication using CUDA. I'm also using shared memory to improve the performance. How can I load the matrix in an efficient way, knowing that my matrix is a sparse matrix? Below is my C++ function to load the matrix: int readMatrix( char* filename, float* &matrix, unsigned int *dim = NULL, int majority = ROW_MAJOR ) {

问题 I have a huge sparse dataset in a dataframe and have been using df.to_sparse but it will be deprecated soon so wanted to switch to pd.Series(pd.SparseArray()) but not sure how to do that for an entire dataframe? My final df is 100K rows and 49K columns so need an automated way. 回答1: You could try something like this : dtype = {key: pd.SparseDtype(df.dtypes[key].type, fill_value=df[key].value_counts().argmax()) for key in df.dtypes.keys()} df = df.astype(dtype) And then check the density with

问题 I've got linear system to solve which consists of large, sparse matrices. I've been using the scipy.sparse library, and its linalg sub-library to do this, but I can't get some of the linear solvers to work. Here is a working example which reproduces the issue for me: from numpy.random import random from scipy.sparse import csc_matrix from scipy.sparse.linalg import spsolve, minres N = 10 A = csc_matrix( random(size = (N,N)) ) A = (A.T).dot(A) # force the matrix to be symmetric, as required by

问题 I am having 2RDD and I want to multiply element-wise between these 2 rdd. Lets say that I am having the following RDD(example): a = ((1,[0.28,1,0.55]),(2,[0.28,1,0.55]),(3,[0.28,1,0.55])) aRDD = sc.parallelize(a) b = ((1,[0.28,0,0]),(2,[0,0,0]),(3,[0,1,0])) bRDD = sc.parallelize(b) It can be seen that b is sparse and I want to avoid multiply a zero value with another value. I am doing the following: from pyspark.mllib.linalg import Vectors def create_sparce_matrix(a_list): length = len(a_list

问题 I am having 2RDD and I want to multiply element-wise between these 2 rdd. Lets say that I am having the following RDD(example): a = ((1,[0.28,1,0.55]),(2,[0.28,1,0.55]),(3,[0.28,1,0.55])) aRDD = sc.parallelize(a) b = ((1,[0.28,0,0]),(2,[0,0,0]),(3,[0,1,0])) bRDD = sc.parallelize(b) It can be seen that b is sparse and I want to avoid multiply a zero value with another value. I am doing the following: from pyspark.mllib.linalg import Vectors def create_sparce_matrix(a_list): length = len(a_list

问题 I have a very large and also sparse matrix (531K x 315K), the number of total cells is ~167 Billion. The non-zero values are only 1s. Total number of non-zero values are around 45K. Is there an efficient NMF package to solve my problem? I know there are couple of packages for that and they are working well only for small size of data matrix. Any idea helps. Thanks in advance. 回答1: scikit-learn will handle this easily ! Code: from time import perf_counter as pc import numpy as np import scipy

问题 I have a very large and also sparse matrix (531K x 315K), the number of total cells is ~167 Billion. The non-zero values are only 1s. Total number of non-zero values are around 45K. Is there an efficient NMF package to solve my problem? I know there are couple of packages for that and they are working well only for small size of data matrix. Any idea helps. Thanks in advance. 回答1: scikit-learn will handle this easily ! Code: from time import perf_counter as pc import numpy as np import scipy

问题 Say I would like to remove the diagonal from a scipy.sparse.csr_matrix . Is there an efficient way of doing so? I saw that in the sparsetools module there are C functions to return the diagonal. Based on other SO answers here and here my current approach is the following: def csr_setdiag_val(csr, value=0): """Set all diagonal nonzero elements (elements currently in the sparsity pattern) to the given value. Useful to set to 0 mostly. """ if csr.format != "csr": raise ValueError('Matrix given

问题 I am implementing SVR using sklearn svr package in python. My sparse matrix is of size 146860 x 10202. I have divided it into various sub-matrices of size 2500 x 10202. For each sub matrix, SVR fitting is taking about 10 mins. What could be the ways to speed up the process? Please suggest any different approach or different python package for the same. Thanks! 回答1: You can average the SVR sub-models predictions. Alternatively you can try to fit a linear regression model on the output of