sparse-matrix | 易学教程

Best C++ Matrix Library for sparse unitary matrices

阅读更多关于 Best C++ Matrix Library for sparse unitary matrices

问题 I am looking for a good (in the best case actively maintained) C++ matrix library. Thereby it should be templated, because I want to use a complex of rationals as numerical type. The matrices what I am dealing with are mainly sparse and unitary. Can you please suggest libraries and also give a small explaination why to use them, because I know how to find them, but I cannot really decide what is suitable for me because I am missing the experience with them. EDIT: The main operations I am

Populate a Pandas SparseDataFrame from a SciPy Sparse Matrix

阅读更多关于 Populate a Pandas SparseDataFrame from a SciPy Sparse Matrix

I noticed Pandas now has support for Sparse Matrices and Arrays . Currently, I create DataFrame() s like this: return DataFrame(matrix.toarray(), columns=features, index=observations) Is there a way to create a SparseDataFrame() with a scipy.sparse.csc_matrix() or csr_matrix() ? Converting to dense format kills RAM badly. Thanks! Jeff A direct conversion is not supported ATM. Contributions are welcome! Try this, should be ok on memory as the SpareSeries is much like a csc_matrix (for 1 column) and pretty space efficient In [37]: col = np.array([0,0,1,2,2,2]) In [38]: data = np.array([1,2,3,4,5

sparse 3d matrix/array in Python?

阅读更多关于 sparse 3d matrix/array in Python?

问题 In scipy, we can construct a sparse matrix using scipy.sparse.lil_matrix() etc. But the matrix is in 2d. I am wondering if there is an existing data structure for sparse 3d matrix / array (tensor) in Python? p.s. I have lots of sparse data in 3d and need a tensor to store / perform multiplication. Any suggestions to implement such a tensor if there's no existing data structure? 回答1: Happy to suggest a (possibly obvious) implementation of this, which could be made in pure Python or C/Cython if

Directly use Intel mkl library on Scipy sparse matrix to calculate A dot A.T with less memory

阅读更多关于 Directly use Intel mkl library on Scipy sparse matrix to calculate A dot A.T with less memory

问题 I want to call mkl.mkl_scsrmultcsr from python. The goal is to calculate a sparse matrix C in compressed sparse row format. Sparse matrix C is the matrix product between A and transpose of A, where A is also a sparse matrix in csr format. When calculating C = A dot (A.T) with scipy, scipy seems (?) to allocate new memory for holding transpose of A (A.T), and definitely allocates memory for a new C matrix (This means I can't use an existing C matrix). So, I want to try to use the mkl c

Sparse matrix slicing using list of int

阅读更多关于 Sparse matrix slicing using list of int

I'm writing a machine learning algorithm on huge & sparse data (my matrix is of shape (347, 5 416 812 801) but very sparse, only 0.13% of the data is non zero. My sparse matrix's size is 105 000 bytes (<1Mbytes) and is of csr type. I'm trying to separate train/test sets by choosing a list of examples indices for each. So I want to split my dataset in two using : training_set = matrix[train_indices] of shape (len(training_indices), 5 416 812 801) , still sparse testing_set = matrix[test_indices] of shape (347-len(training_indices), 5 416 812 801) also sparse With training_indices and testing

Create Sparse Matrix from a data frame

阅读更多关于 Create Sparse Matrix from a data frame

问题 I m doing an assignment where I am trying to build a collaborative filtering model for the Netflix prize data. The data that I am using is in a CSV file which I easily imported into a data frame. Now what I need to do is create a sparse matrix consisting of the Users as the rows and Movies as the columns and each cell is filled up by the corresponding rating value. When I try to map out the values in the data frame I need to run a loop for each row in the data frame, which is taking a lot of

Sparse matrix to a data frame in R

阅读更多关于 Sparse matrix to a data frame in R

问题 I have a sparse matrix Formal class 'dgCMatrix' [package "Matrix"] with 6 slots ..@ i : int [1:37674] 1836 2297 108 472 1735 1899 2129 2131 5 67 ... ..@ p : int [1:3417] 0 2 8 22 25 35 44 45 45 47 ... ..@ Dim : int [1:2] 3416 3416 ..@ Dimnames:List of 2 .. ..$ : chr [1:3416] "AAA" "AAE" "AAL" "AAN" ... .. ..$ : chr [1:3416] "AAA" "AAE" "AAL" "AAN" ... ..@ x : num [1:37674] 1 1 1 1 1 1 1 1 1 1 ... ..@ factors : list() What is a fast way to convert this matrix to a list as (except for a for

Creating (and Accessing) a Sparse Matrix with NA default entries

阅读更多关于 Creating (and Accessing) a Sparse Matrix with NA default entries

问题 After learning about the options for working with sparse matrices in R, I want to use the Matrix package to create a sparse matrix from the following data frame and have all other elements be NA . s r d 1 1089 3772 1 2 1109 190 1 3 1109 2460 1 4 1109 3071 2 5 1109 3618 1 6 1109 38 7 I know I can create a sparse matrix with the following, accessing elements as usual: > library(Matrix) > Y <- sparseMatrix(s,r,x=d) > Y[1089,3772] [1] 1 > Y[1,1] [1] 0 but if I want to have the default value to be

Efficiently create sparse pivot tables in pandas?

阅读更多关于 Efficiently create sparse pivot tables in pandas?

问题 I\'m working turning a list of records with two columns (A and B) into a matrix representation. I have been using the pivot function within pandas, but the result ends up being fairly large. Does pandas support pivoting into a sparse format? I know I can pivot it and then turn it into some kind of sparse representation, but isn\'t as elegant as I would like. My end goal is to use it as the input for a predictive model. Alternatively, is there some kind of sparse pivot capability outside of

Directly creating dummy variable set in a sparse matrix in R

阅读更多关于 Directly creating dummy variable set in a sparse matrix in R

问题 Suppose you have a data frame with a high number of columns(1000 factors, each with 15 levels). You\'d like to create a dummy variable data set, but since it would be too sparse, you would like to keep dummies in sparse matrix format. My data set is quite big and the less steps there are, the better for me. I know how to do above steps; but I couldn\'t get my head around directly creating that sparse matrix from the initial data set, i.e. having one step instead of two. Any ideas? EDIT: Some