sparse-matrix

Use Redis to generate unique ID from a limited range

生来就可爱ヽ(ⅴ<●) 提交于 2019-12-06 04:14:54
I have database items that, in addition to their primary key, need an index unique to the group in which the items belong. Let's call the property nbr , and the property that groups items together and defines the scope of unique nbr :s we'll call group . This nbr must be in the [1-N] range, and may be set when items are imported from an external source. Since all items must have a nbr , the task then becomes how to track which values are used, to enable picking a free nbr for new items that are added manually. I'm using DynamoDB and Redis. I cannot have a DynamoDB index on nbr . The idea I

How to build a sparse matrix in PySpark?

穿精又带淫゛_ 提交于 2019-12-06 03:51:22
问题 I am new to Spark. I would like to make a sparse matrix a user-id item-id matrix specifically for a recommendation engine. I know how I would do this in python. How does one do this in PySpark? Here is how I would have done it in matrix. The table looks like this now. Session ID| Item ID | Rating 1 2 1 1 3 5 import numpy as np data=df[['session_id','item_id','rating']].values data rows, row_pos = np.unique(data[:, 0], return_inverse=True) cols, col_pos = np.unique(data[:, 1], return_inverse

Getting different colors for different numbers using `spy` in Matlab

╄→гoц情女王★ 提交于 2019-12-06 03:34:22
When I use spy to check a sparsity pattern, it doesn't distinguish certain elements from others. Is there any way to do this? Say, for example, elements that are equal to 10 are red and all elements equal to 9 are blue. Can I get this in one spy plot? I've only been able to change the size and style of the plot points. Here is how you can do it: spy(a,'k') hold on spy(a==10,'r') spy(a==9,'b') hold off Another way is to use scatter instead of spy : [x,y] = find(a); clr = a(a~=0); scatter(x,y,[],clr) set(gca,'YDir','rev') In this case the points will be colored by a values according to current

How to create a binary matrix of inventory per row? (R)

寵の児 提交于 2019-12-06 02:55:09
问题 I have a dataframe of 9 columns consisting of an inventory of factors. Each row can have all 9 columns filled (as in that row is holding 9 "things"), but most don't (most have between 3-4). The columns aren't specific either, as in if item 200 shows up in columns 1 and 3, it's the same thing. I'd like to create a matrix that is binary for each row that includes all factors. Ex (shortened to 4 columns just to get point across) R1 3 4 5 8 R2 4 6 7 NA R3 1 5 NA NA R4 2 6 8 9 Should turn into 1 2

In R, when using named rows, can a sparse matrix column be added (concatenated) to another sparse matrix?

≯℡__Kan透↙ 提交于 2019-12-06 02:44:22
问题 I have two sparse matrices, m1 and m2 : > m1 <- Matrix(data=0,nrow=2, ncol=1, sparse=TRUE, dimnames=list(c("b","d"),NULL)) > m2 <- Matrix(data=0,nrow=2, ncol=1, sparse=TRUE, dimnames=list(c("a","b"),NULL)) > m1["b",1]<- 4 > m2["a",1]<- 5 > m1 2 x 1 sparse Matrix of class "dgCMatrix" b 4 d . > m2 2 x 1 sparse Matrix of class "dgCMatrix" a 5 b . > and I want to cbind() them to make a sparse matrix like: [,1] [,2] a . 5 b 4 . d . . however cbind() ignores the named rows: > cbind(m1[,1],m2[,1]) [

Random binary matrix with row and column sum constraints

限于喜欢 提交于 2019-12-06 02:25:57
My objective is to create: a randomly populated matrix with entries either 0 or 1 . In this particular case, the matrix is 4x24 . The row sum of each of the 4 rows is exactly 6 . The column sum of each of the 24 columns is exactly 1 Call the desired matrix M . Another way of looking at M : There are exactly 24 entries equal to 1 . No column has more than one 1 entry. Progress: There are 6 spots on each row with a 1 entry. The rest are zero, the matrix is sparse. With 4 rows, this means that M can be uniquely determined by a matrix of indices that stores the locations of the 1 entries. Call

How to visualize a sparse matrix in MATLAB?

扶醉桌前 提交于 2019-12-06 01:59:00
So I have this matrix here , and it is of size 13 x 8198. (I have called it 'blah'). This is a sparse matrix, in that, most of its entries are 0. When I do an imagesc(blah), I get the following image: Clearly this is worthless because I cannot clearly see the non-zero elements. I have tried playing around with the color scaling, but to no avail. Anyway, I was wondering if there might be a nicer way to be able to visualize this matrix in MATLAB somehow? I am designing an algorithm and would like to be able to see certain things int teh matrix. Thanks! Try spy ; it's intended for exactly that.

The fastest way to calculate eigenvalues of large matrices

折月煮酒 提交于 2019-12-06 01:17:24
Until now I used numpy.linalg.eigvals to calculate the eigenvalues of quadratic matrices with at least 1000 rows/columns and, for most cases, about a fifth of its entries non-zero (I don't know if that should be considered a sparse matrix). I found another topic indicating that scipy can possibly do a better job. However, since I have to calculate the eigenvalues for hundreds of thousands of large matrices of increasing size (possibly up to 20000 rows/columns and yes, I need ALL of their eigenvalues), this will always take awfully long. If I can speed things up, even just the tiniest bit, it

How to create a diagonal sparse matrix in SciPy

时间秒杀一切 提交于 2019-12-06 00:43:33
I am trying to create a sparse matrix which has a 2D pattern run down the diagonal. This is probably easiest to explain with a quick example. Say my pattern is: [1,0,2,0,1]... I want to create a sparse matrix: [[2,0,1,0,0,0,0...0], [0,2,0,1,0,0,0...0], [1,0,2,0,1,0,0...0], [0,1,0,2,0,1,0...0], [0,0,1,0,2,0,1...0], [...]] The scipy.sparse.dia_matrix seems like a good candidate, however, I simply cannot figure out how to accomplish what I want from the documentation available. Thank you in advance N = 10 diag = np.zeros(N) + 2 udiag = np.zeros(N) + 1 ldiag = np.zeros(N) + 1 mat = scipy.sparse

Sparse Matrix as input to Hierarchical clustering in R

◇◆丶佛笑我妖孽 提交于 2019-12-05 23:47:34
I have a question about clustering using a distance matrix, but sparse. Is there a sparse distance object format that does not expand the matrix and can work with the sparse representation? Currently I'm doing the following # read sparse matrix sparse <- readMM('sparse-matrix') distance <- as.dist(sparse) sparse-matrix is already the correct distance matrix, which has NA's for entries that are not connected. >sparse [1,] . . . [2,] 1 . . [3,] 1 . . > as.dist(sparse) 1 2 2 1 3 1 0 But converting it with as.dist fails with Error in asMethod(object) : negative length vectors are not allowed