sparse-matrix

问题 consider the the pandas series s n = 1000 s = pd.Series([0] * n + [1] * n, dtype=int) s.memory_usage() 8080 I can "sparsify" this by using to_sparse s.to_sparse(fill_value=0).memory_usage() 4080 But I only have 2 types of integers. I'd think I could sparsify twice. Is there a way to do this? 回答1: Since you tagged this with scipy , I'll show you what a scipy.sparse matrix is like: In [31]: n=100 In [32]: arr=np.array([[0]*n+[1]*n],int) In [33]: M=sparse.csr_matrix(arr) In [34]: M.data Out[34]:

问题 I have a sparse matrix saved in a mat file. I want to process it in C++ but my library expects the sparse matrix to be represented as three vectors: colIdx (vector of indices of nonzero columns), rowIdx (vector of indices of nonzero rows), and val (vector of values). How should I go about this? Are there any built-in commands doing similar operations? 回答1: find is what you are looking for [rowIdx colIdx val] = find( myHugeSparseMatrix ); 来源： https://stackoverflow.com/questions/32177489/save

问题 I have a large sparse matrix, call it P: > str(P) Formal class 'dgCMatrix' [package "Matrix"] with 6 slots ..@ i : int [1:7868093] 4221 6098 8780 10313 11102 14243 20570 22145 24468 24977 ... ..@ p : int [1:7357] 0 0 269 388 692 2434 3662 4179 4205 4256 ... ..@ Dim : int [1:2] 1303967 7356 ..@ Dimnames:List of 2 .. ..$ : NULL .. ..$ : NULL ..@ x : num [1:7868093] 1 1 1 1 1 1 1 1 1 1 ... ..@ factors : list() I'd like to row-normalize (say, with the L-2 norm)... (taking advantage of vector

问题 I am trying to create a sparse matrix with numerical and categorical data which will be used as an input to cv.glmnet. When only numerical data is involved, I can create a sparseMatrix using the following syntax sparseMatrix(i=c(1,3,5,2), j=c(1,1,1,2), x=c(1,2,4,3), dims=c(5,2)) For categorical variables, the following approach seems to work: sparse.model.matrix(~-1+automobile, data.frame(automobile=c("sedan","suv","minivan","truck","sedan"))) My VERY sparse instance has 1,000,000

问题 I know that Intel Fortran has libraries with functions and subroutines for working with sparse matricies, but I'm wondering if there is also some sort of data type or automated method for creating the sparse matricies in the first place. BACKGROUND: I have a program that uses some 3 & 4 dimensional arrays that can be very large in the first 2 dimensions (~10k to ~100k elements in each dimension, maybe more). In the first 2 dimensions, each array is mostly (95% or so) populated w/ zeroes. To

问题 I have to handle sparse matrix that can occasionally be very big, nearing or exceeding RAM capacity. I also need to support mat*vec and mat*mat operations. Since internally a csr_matrix is 3 arrays data , indices and indptr is it possible to create a csr matrix from numpy memmap. 回答1: This works without any problems. 来源： https://stackoverflow.com/questions/49835342/using-memory-mapped-buffers-for-scipy-sparse

问题 I have a large sparse matrix A , and I would like to create a sparse matrix of the 3X3 block diagonals of A. How would I do this? keep in mind that A is very large and sparse, so any methods that use iteration will be slow, and any methods that use some methods that creates full (as opposed to sparse) matrices will take up too much memory. 回答1: If I understand correctly, here is some code (see the portions between the %%%%%%%%%%% lines. Below are timing results, which seem reasonable to me,

问题 I have a purchase data ( df_temp ). I managed to replace using Pandas Dataframe to using a sparse csr_matrix because I have lots of products (89000) which I have to get their user-item information (purchased or not purchased) and then calculate the similarities between products. First, I converted Pandas DataFrame to Numpy array: df_user_product = df_temp[['user_id','product_id']].copy() ar1 = np.array(df_user_product.to_records(index=False)) Second, created a coo_matrix because it's known

问题 I'm trying to multiply a sparse real matrix with a complex vector but the program does not compile. If I change the vector to real or the matrix to dense, then everything goes through. A sample code is: #define ARMA_64BIT_WORD #include <armadillo> #include <iostream> #include <stdio.h> #include <math.h> using namespace arma; int main(){ size_t n(5); vec vR(randu<vec>(n)), vI(randu<vec>(n)); //Create random complex vector 'v' cx_vec v(vR, vI); std::cout<<"\n\tMultiplying real matrix with

问题 I have to solve a large amount of linear matrix equations of the type "Ax=B" for x where A is a sparse matrix with mainly the main diagonal populated and B is a vector. My first approach was to use dense numpy arrays for this purpose with numpy.linalg.solve, and it works fine with a (N,n,n)-dimensional array with N being the number of linear matrix equations and n the square matrix dimension. I first used it with a for loop iterating through all equations, which in fact is rather slow. But