sparse-matrix

Does scipy support multithreading for sparse matrix multiplication when using MKL BLAS?

萝らか妹 提交于 2019-12-03 14:15:44
According to MKL BLAS documentation "All matrix-matrix operations (level 3) are threaded for both dense and sparse BLAS." http://software.intel.com/en-us/articles/parallelism-in-the-intel-math-kernel-library I have built Scipy with MKL BLAS. Using the test code below, I see the expected multithreaded speedup for dense, but not sparse, matrix multiplication. Are there any changes to Scipy to enable multithreaded sparse operations? # test dense matrix multiplication from numpy import * import time x = random.random((10000,10000)) t1 = time.time() foo = dot(x.T, x) print time.time() - t1 # test

Multidimensional scaling with missing values in dissimilarity matrix

偶尔善良 提交于 2019-12-03 13:40:40
问题 I have a dissimilarity matrix on which I would like to perform multidimensional scaling (MDS) using the sklearn.manifold.MDS function. The dissimilarity between some elements in this matrix is not meaningful and I am thus wondering if there is a way to run MDS on a sparse matrix or on a matrix with missing values? According to this question, dissimilarities with 0 are considered as missing values, but I was unable to find this statement in the official documentation. Isn't a dissimilarity

Improving Performance of Multiplication of Scipy Sparse Matrices

◇◆丶佛笑我妖孽 提交于 2019-12-03 13:36:30
问题 Given a Scipy CSC Sparse matrix "sm" with dimensions (170k x 170k) with 440 million non-null points and a sparse CSC vector "v" (170k x 1) with a few non-null points, is there anything that can be done to improve the performance of the operation: resul = sm.dot(v) ? Currently it's taking roughly 1 second. Initializing the matrices as CSR increased the time up to 3 seconds, so CSC performed better. SM is a matrix of similarities between products and V is the vector that represents which

Scipy sparse matrix multiplication

守給你的承諾、 提交于 2019-12-03 12:54:50
I have this example of matrix by matrix multiplication using numpy arrays: import numpy as np m = np.array([[1,2,3],[4,5,6],[7,8,9]]) c = np.array([0,1,2]) m * c array([[ 0, 2, 6], [ 0, 5, 12], [ 0, 8, 18]]) How can i do the same thing if m is scipy sparse CSR matrix? This gives dimension mismatch: sp.sparse.csr_matrix(m)*sp.sparse.csr_matrix(c) You can call the multiply method of csr_matrix to do pointwise multiplication. sparse.csr_matrix(m).multiply(sparse.csr_matrix(c)).todense() # matrix([[ 0, 2, 6], # [ 0, 5, 12], # [ 0, 8, 18]], dtype=int64) When m and c are numpy arrays, then m * c is

Best way to calculate the fundamental matrix of an absorbing Markov Chain?

我与影子孤独终老i 提交于 2019-12-03 11:43:30
I have a very large absorbing Markov chain (scales to problem size -- from 10 states to millions) that is very sparse (most states can react to only 4 or 5 other states). I need to calculate one row of the fundamental matrix of this chain (the average frequency of each state given one starting state). Normally, I'd do this by calculating (I - Q)^(-1) , but I haven't been able to find a good library that implements a sparse matrix inverse algorithm! I've seen a few papers on it, most of them P.h.D. level work. Most of my Google results point me to posts talking about how one shouldn't use a

Correlation coefficients for sparse matrix in python?

ⅰ亾dé卋堺 提交于 2019-12-03 11:36:23
Does anyone know how to compute a correlation matrix from a very large sparse matrix in python? Basically, I am looking for something like numpy.corrcoef that will work on a scipy sparse matrix. You can compute the correlation coefficients fairly straightforwardly from the covariance matrix like this: import numpy as np from scipy import sparse def sparse_corrcoef(A, B=None): if B is not None: A = sparse.vstack((A, B), format='csr') A = A.astype(np.float64) n = A.shape[1] # Compute the covariance matrix rowsum = A.sum(1) centering = rowsum.dot(rowsum.T.conjugate()) / n C = (A.dot(A.T.conjugate

Local maxima in a point cloud

我怕爱的太早我们不能终老 提交于 2019-12-03 08:58:25
I have a point cloud C, where each point has an associated value. Lets say the points are in 2-d space, so each point can be represented with the triplet (x, y, v). I'd like to find the subset of points which are local maxima. That is, for some radius R, I would like to find the subset of points S in C such that for any point Pi (with value vi) in S, there is no point Pj in C within R distance of Pi whose value vj is greater that vi. I see how I could do this in O(N^2) time, but that seems wasteful. Is there an efficient way to do this? Side Notes: The source of this problem is that I'm trying

How to change elements in sparse matrix in Python's SciPy?

≯℡__Kan透↙ 提交于 2019-12-03 06:59:33
问题 I have built a small code that I want to use for solving eigenvalue problems involving large sparse matrices. It's working fine, all I want to do now is to set some elements in the sparse matrix to zero, i.e. the ones in the very top row (which corresponds to implementing boundary conditions). I can just adjust the column vectors (C0, C1, and C2) below to achieve that. However, I wondered if there is a more direct way. Evidently, NumPy indexing does not work with SciPy's sparse package.

Using pytables, which is more efficient: scipy.sparse or numpy dense matrix?

柔情痞子 提交于 2019-12-03 06:23:47
When using pytables , there's no support (as far as I can tell) for the scipy.sparse matrix formats, so to store a matrix I have to do some conversion, e.g. def store_sparse_matrix(self): grp1 = self.getFileHandle().createGroup(self.getGroup(), 'M') self.getFileHandle().createArray(grp1, 'data', M.tocsr().data) self.getFileHandle().createArray(grp1, 'indptr', M.tocsr().indptr) self.getFileHandle().createArray(grp1, 'indices', M.tocsr().indices) def get_sparse_matrix(self): return sparse.csr_matrix((self.getGroup().M.data, self.getGroup().M.indices, self.getGroup().M.indptr)) The trouble is

slicing sparse (scipy) matrix

倾然丶 夕夏残阳落幕 提交于 2019-12-03 05:12:02
问题 I would appreciate any help, to understand following behavior when slicing a lil_matrix (A) from the scipy.sparse package. Actually, I would like to extract a submatrix based on an arbitrary index list for both rows and columns. When I used this two lines of code: x1 = A[list 1,:] x2 = x1[:,list 2] Everything was fine and I could extract the right submatrix. When I tried to do this in one line, it failed (The returning matrix was empty) x=A[list 1,list 2] Why is this so? Overall, I have used