sparse-matrix | 易学教程

Directly use Intel mkl library on Scipy sparse matrix to calculate A dot A.T with less memory

阅读更多关于 Directly use Intel mkl library on Scipy sparse matrix to calculate A dot A.T with less memory

I want to call mkl.mkl_scsrmultcsr from python. The goal is to calculate a sparse matrix C in compressed sparse row format. Sparse matrix C is the matrix product between A and transpose of A, where A is also a sparse matrix in csr format. When calculating C = A dot (A.T) with scipy, scipy seems (?) to allocate new memory for holding transpose of A (A.T), and definitely allocates memory for a new C matrix (This means I can't use an existing C matrix). So, I want to try to use the mkl c function directly to decrease memory usage. Here is an answer that works for another mkl function. In that

How to elementwise-multiply a scipy.sparse matrix by a broadcasted dense 1d array?

阅读更多关于 How to elementwise-multiply a scipy.sparse matrix by a broadcasted dense 1d array?

问题 Suppose I have a 2d sparse array. In my real usecase both the number of rows and columns are much bigger (say 20000 and 50000) hence it cannot fit in memory when a dense representation is used: >>> import numpy as np >>> import scipy.sparse as ssp >>> a = ssp.lil_matrix((5, 3)) >>> a[1, 2] = -1 >>> a[4, 1] = 2 >>> a.todense() matrix([[ 0., 0., 0.], [ 0., 0., -1.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 2., 0.]]) Now suppose I have a dense 1d array with all non-zeros components with size 3 (or

sparse 3d matrix/array in Python?

阅读更多关于 sparse 3d matrix/array in Python?

In scipy, we can construct a sparse matrix using scipy.sparse.lil_matrix() etc. But the matrix is in 2d. I am wondering if there is an existing data structure for sparse 3d matrix / array (tensor) in Python? p.s. I have lots of sparse data in 3d and need a tensor to store / perform multiplication. Any suggestions to implement such a tensor if there's no existing data structure? Happy to suggest a (possibly obvious) implementation of this, which could be made in pure Python or C/Cython if you've got time and space for new dependencies, and need it to be faster. A sparse matrix in N dimensions

Most mature sparse matrix package for R?

阅读更多关于 Most mature sparse matrix package for R?

问题 There are at least two sparse matrix packages for R. I'm looking into these because I'm working with datasets that are too big and sparse to fit in memory with a dense representation. I want basic linear algebra routines, plus the ability to easily write C code to operate on them. Which library is the most mature and best to use? So far I've found Matrix which has many reverse dependencies, implying it's the most used one. SparseM which doesn't have as many reverse deps. Various graph

Is sparse tensor multiplication implemented in TensorFlow?

阅读更多关于 Is sparse tensor multiplication implemented in TensorFlow?

问题 Multiplication of sparse tensors with themselves or with dense tensors does not seem to work in TensorFlow. The following example from __future__ import print_function import tensorflow as tf x = tf.constant([[1.0,2.0], [3.0,4.0]]) y = tf.SparseTensor(indices=[[0,0],[1,1]], values=[1.0,1.0], shape=[2,2]) z = tf.matmul(x,y) sess = tf.Session() sess.run(tf.initialize_all_variables()) print(sess.run([x, y, z])) fails with the error message TypeError: Input 'b' of 'MatMul' Op has type string that

expanding (adding a row or column) a scipy.sparse matrix

阅读更多关于 expanding (adding a row or column) a scipy.sparse matrix

问题 Suppose I have a NxN matrix M (lil_matrix or csr_matrix) from scipy.sparse, and I want to make it (N+1)xN where M_modified[i,j] = M[i,j] for 0 <= i < N (and all j) and M[N,j] = 0 for all j. Basically, I want to add a row of zeros to the bottom of M and preserve the remainder of the matrix. Is there a way to do this without copying the data? 回答1: I don't think that there is any way to really escape from doing the copying. Both of those types of sparse matrices store their data as Numpy arrays

Matrix Library for .NET [closed]

阅读更多关于 Matrix Library for .NET [closed]

问题 As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 6 years ago . I'm looking for a good (well-tested, fully-featured, and ideally with a nice interface) matrix library for .NET/C#. My main

Block operations on Sparse Matrices- Eigen Toolbox- C++

阅读更多关于 Block operations on Sparse Matrices- Eigen Toolbox- C++

问题 Block operations for sparse matrices - Eigen Toolbox - C++ #include "Eigen/Dense" #include "Eigen/Sparse" #include <iostream> using namespace std; using namespace Eigen; int main() { MatrixXd silly(6, 3); silly << 0, 1, 2, 0, 3, 0, 2, 0, 0, 3, 2, 1, 0, 1, 0, 2, 0, 0; SparseMatrix<double> sparse_silly,temp; sparse_silly= Eigen::SparseMatrix<double>(6, 3); temp = Eigen::SparseMatrix<double>(6, 3); sparse_silly = silly.sparseView(); std::cout << "Whole Matrix" << std::endl; std::cout << sparse

Create Sparse Matrix from a data frame

阅读更多关于 Create Sparse Matrix from a data frame

I m doing an assignment where I am trying to build a collaborative filtering model for the Netflix prize data. The data that I am using is in a CSV file which I easily imported into a data frame. Now what I need to do is create a sparse matrix consisting of the Users as the rows and Movies as the columns and each cell is filled up by the corresponding rating value. When I try to map out the values in the data frame I need to run a loop for each row in the data frame, which is taking a lot of time in R, please can anyone suggest a better approach. Here is the sample code and data:

Sparse matrix to a data frame in R

阅读更多关于 Sparse matrix to a data frame in R

I have a sparse matrix Formal class 'dgCMatrix' [package "Matrix"] with 6 slots ..@ i : int [1:37674] 1836 2297 108 472 1735 1899 2129 2131 5 67 ... ..@ p : int [1:3417] 0 2 8 22 25 35 44 45 45 47 ... ..@ Dim : int [1:2] 3416 3416 ..@ Dimnames:List of 2 .. ..$ : chr [1:3416] "AAA" "AAE" "AAL" "AAN" ... .. ..$ : chr [1:3416] "AAA" "AAE" "AAL" "AAN" ... ..@ x : num [1:37674] 1 1 1 1 1 1 1 1 1 1 ... ..@ factors : list() What is a fast way to convert this matrix to a list as (except for a for loop ): Origin Destination Weight AAA AAE 4 AAL AAN 5 Note: I only need to get the Origin and Destination