sparse-matrix

问题 Is there a way to convert from a pandas.SparseDataFrame to scipy.sparse.csr_matrix , without generating a dense matrix in memory? scipy.sparse.csr_matrix(df.values) doesn't work as it generates a dense matrix which is cast to the csr_matrix . Thanks in advance! 回答1: Pandas docs talks about an experimental conversion to scipy sparse, SparseSeries.to_coo: http://pandas-docs.github.io/pandas-docs-travis/sparse.html#interaction-with-scipy-sparse ================ edit - this is a special function

问题 I noticed Pandas now has support for Sparse Matrices and Arrays. Currently, I create DataFrame() s like this: return DataFrame(matrix.toarray(), columns=features, index=observations) Is there a way to create a SparseDataFrame() with a scipy.sparse.csc_matrix() or csr_matrix() ? Converting to dense format kills RAM badly. Thanks! 回答1: A direct conversion is not supported ATM. Contributions are welcome! Try this, should be ok on memory as the SpareSeries is much like a csc_matrix (for 1 column)

问题 I noticed Pandas now has support for Sparse Matrices and Arrays. Currently, I create DataFrame() s like this: return DataFrame(matrix.toarray(), columns=features, index=observations) Is there a way to create a SparseDataFrame() with a scipy.sparse.csc_matrix() or csr_matrix() ? Converting to dense format kills RAM badly. Thanks! 回答1: A direct conversion is not supported ATM. Contributions are welcome! Try this, should be ok on memory as the SpareSeries is much like a csc_matrix (for 1 column)

问题 I am currently trying to speed up my large sparse (scipy) matrix multiplications. I have successfully linked my numpy installation with OpenBLAS and henceforth, also scipy. I have run these tests with success. When I use numpy.dot(X,Y) I can clearly see performance boosts and also that multiple cores are used simultaneously. However, when I use scipy's dot functionality, no such performance boosts can be seen and still one one core is used. For example: x = scipy.sparse.csr_matrix(numpy

问题 I initialize an empty sparse matrix using S = scipy.sparse.lil_matrix((n,n),dtype=int) As expected print S doesn't show anything, since nothing has been assigned. Yet if I test: print S[0,0]==0 I receive true . Is there a way to test if a value has been set before? E.g. along the lines of ifempty ? 回答1: You can check for stored values with def get_items(s): s_coo = s.tocoo() return set(zip(s_coo.row, s_coo.col)) Demo: >>> n = 100 >>> s = scipy.sparse.lil_matrix((n,n),dtype=int) >>> s[10, 12]

问题 I have a sparse 5018x5018 matrix in MATLAB, which has about 100k values set to 1 (i.e., about 99.6% empty). I'm trying to flip roughly 5% of those zeros to ones (i.e., about 1.25m entries). I have the x and y indices in the matrix I want to flip. Here is what I have done: sizeMat=size(network); idxToReplace=sub2ind(sizeMat,x_idx, y_idx); network(idxToReplace) = 1; This is incredibly slow, in particular the last line. Is there any way to make this operation run noticeably faster, preferably

问题 I have a sparse matrix that is not symmetric I.E. the sparsity is somewhat random, and I can't count on all the values being a set distance away from the diagonal. However, it is still sparse, and I want to reduce the storage requirement on the matrix. Therefore, I am trying to figure out how to store each row starting at the first non-zero, in order, until I get to the last non-zero. That is, if the first non-zero of row m occurs at column 2, and the last non-zero is at column 89, I want to

问题 I get a weird effect when trying to overload the + operator and using the Matrix package with sparse matrices. I first define a very simple class that does not use the Matrix package but has a + operator. I then sum two sparse matrices. The first M+M addition delivers the expected result but the second throws an error. Here is a very simple code that generates the error: require(Matrix) setClass("TestM",representation(M='numeric')) setMethod("initialize", "TestM", function(.Object,x) {

问题 I have the following R matrix that is a combination of 2x3 and 3x3 submatrices and it can be more than 2 submatrices with different dimension (e.g. m1xp and m2xp and m3xp where each of m1,m2,m3 <= p) A2 <- list(rbind(c(1,1,1),c(-1,1,-1)), rbind(c(-1,1,1),c(1,-1,2),c(2,-1,2))) library(Matrix) A2 <- as.matrix(Matrix::bdiag(A2)) Rhs <- matrix(c(0,5,0.5,4),nrow = 4) beta <- c(rep(1.2,3),c(0.5,0.2,0.1)) > A2 [,1] [,2] [,3] [,4] [,5] [,6] [1,] 1 1 1 0 0 0 [2,] -1 1 -1 0 0 0 [3,] 0 0 0 -1 1 1 [4,] 0

问题 Is there a way in Python to have an efficient incremental update of sparse matrix? H = lil_matrix((n,m)) for (i,j) in zip(A,B): h(i,j) += compute_something It seems that such a way to build a sparse matrix is quite slow ( lil_matrix is the fastest sparse matrix type for that). Is there a way (like using dict of dict or other kind of approaches) to efficiently build the sparse matrix H? 回答1: In https://stackoverflow.com/a/27771335/901925 I explore incremental matrix assignment. lol and dok are