sparse-matrix

Suppose I have a 2d sparse array. In my real usecase both the number of rows and columns are much bigger (say 20000 and 50000) hence it cannot fit in memory when a dense representation is used: >>> import numpy as np >>> import scipy.sparse as ssp >>> a = ssp.lil_matrix((5, 3)) >>> a[1, 2] = -1 >>> a[4, 1] = 2 >>> a.todense() matrix([[ 0., 0., 0.], [ 0., 0., -1.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 2., 0.]]) Now suppose I have a dense 1d array with all non-zeros components with size 3 (or 50000 in my real life case): >>> d = np.ones(3) * 3 >>> d array([ 3., 3., 3.]) I would like to compute

I'm writing a sparse matrix solver using the Gauss-Seidel method. By profiling, I've determined that about half of my program's time is spent inside the solver. The performance-critical part is as follows: size_t ic = d_ny + 1, iw = d_ny, ie = d_ny + 2, is = 1, in = 2 * d_ny + 1; for (size_t y = 1; y < d_ny - 1; ++y) { for (size_t x = 1; x < d_nx - 1; ++x) { d_x[ic] = d_b[ic] - d_w[ic] * d_x[iw] - d_e[ic] * d_x[ie] - d_s[ic] * d_x[is] - d_n[ic] * d_x[in]; ++ic; ++iw; ++ie; ++is; ++in; } ic += 2; iw += 2; ie += 2; is += 2; in += 2; } All arrays involved are of float type. Actually, they are not

Multiplication of sparse tensors with themselves or with dense tensors does not seem to work in TensorFlow. The following example from __future__ import print_function import tensorflow as tf x = tf.constant([[1.0,2.0], [3.0,4.0]]) y = tf.SparseTensor(indices=[[0,0],[1,1]], values=[1.0,1.0], shape=[2,2]) z = tf.matmul(x,y) sess = tf.Session() sess.run(tf.initialize_all_variables()) print(sess.run([x, y, z])) fails with the error message TypeError: Input 'b' of 'MatMul' Op has type string that does not match type float32 of argument 'a' Both tensors have values of type float32 as seen by

There are at least two sparse matrix packages for R. I'm looking into these because I'm working with datasets that are too big and sparse to fit in memory with a dense representation. I want basic linear algebra routines, plus the ability to easily write C code to operate on them. Which library is the most mature and best to use? So far I've found Matrix which has many reverse dependencies, implying it's the most used one. SparseM which doesn't have as many reverse deps. Various graph libraries probably have their own (implicit) versions of this; e.g. igraph and network (the latter is part of

Suppose I have a NxN matrix M (lil_matrix or csr_matrix) from scipy.sparse, and I want to make it (N+1)xN where M_modified[i,j] = M[i,j] for 0 <= i < N (and all j) and M[N,j] = 0 for all j. Basically, I want to add a row of zeros to the bottom of M and preserve the remainder of the matrix. Is there a way to do this without copying the data? I don't think that there is any way to really escape from doing the copying. Both of those types of sparse matrices store their data as Numpy arrays (in the data and indices attributes for csr and in the data and rows attributes for lil) internally and

So, I'm doing some Kmeans classification using numpy arrays that are quite sparse-- lots and lots of zeroes. I figured that I'd use scipy's 'sparse' package to reduce the storage overhead, but I'm a little confused about how to create arrays, not matrices. I've gone through this tutorial on how to create sparse matrices: http://www.scipy.org/SciPy_Tutorial#head-c60163f2fd2bab79edd94be43682414f18b90df7 To mimic an array, I just create a 1xN matrix, but as you may guess, Asp.dot(Bsp) doesn't quite work because you can't multiply two 1xN matrices. I'd have to transpose each array to Nx1, and that

I have a scipy.sparse matrix with 45671x45671 elements. In this matrix, some rows contain only '0' value. My question is, how to divide each row values by the row sum. Obviously, with for loop it's work, but I look for an efficient method... I already tried : matrix / matrix.sum(1) but I have MemoryError issue. matrix / scs.csc_matrix((matrix.sum(axis=1))) but ValueError: inconsistent shapes Other wacky things... Moreover, I want to skip rows with only '0' values. So, if you have any solution... Thank you in advance ! I have an M hanging around: In [241]: M Out[241]: <6x3 sparse matrix of type

I am trying to find out a way to do a matrix power for a sparse matrix M: M^k = M*...*M k times where * is the matrix multiplication (numpy.dot), and not element-wise multiplication . I know how to do it for a normal matrix: import numpy as np import scipy as sp N=100 k=3 M=(sp.sparse.spdiags(np.ones(N), 0, N, N)-sp.sparse.spdiags(np.ones(N), 2, N, N)).toarray() np.matrix_power(M,k) How can I do it for sparse M: M=(sp.sparse.spdiags(np.ones(N), 0, N, N)-sp.sparse.spdiags(np.ones(N), 2, N, N)) Of course, I can do this by recursive multiplications, but I am wondering if there is a functionality