numpy

How to select some rows from sparse matrix then use them form a new sparse matrix

左心房为你撑大大i 提交于 2021-01-29 05:34:32
问题 I have a very large sparse matrix(100000 column and 100000 rows). I want to select some of the rows of this sparse matrix and then use them to form a new sparse matrix. I tried to do it by first converting them to dense matrix and then convert them to sparse matrix again. But when I do this python raise a 'Memory error'. Then I tried another method, which is I select the rows of sparse matrix and then put them into a array, but when I try to convert this array to sparse matrix, it says:

Python (numpy) crashes system with large number of array elements

这一生的挚爱 提交于 2021-01-29 05:30:29
问题 I'm trying to build a basic character recognition model using the many classifiers that scikit provides. The dataset being used is a standard handwritten set of alphanumeric samples (Chars74K image dataset taken from this source: EnglishHnd.tgz). There are 55 samples of each character (62 alphanumeric characters in all), each being 900x1200 pixels. I'm flattening the matrix (first converting to grayscale) into a 1x1080000 array (each representing a feature). for sample in sample_images: #

pandas dataframe fails to assign value to slice subset

强颜欢笑 提交于 2021-01-29 05:24:59
问题 I'm trying to change all values in the slice except the first one but it does not work... what am i doing wrong ? print(test) test.loc[(test.col_1==-5)&(test.index>'2018-07-17 13:00:00')&(test.index<'2018-07-17 14:00:00'),['col_1']][1:]=-1 print(test) provides the below output 17/07/2018 13:51:00 -5 17/07/2018 13:52:00 -1 17/07/2018 13:53:00 -5 17/07/2018 13:54:00 -5 17/07/2018 13:55:00 -5 17/07/2018 13:56:00 -5 17/07/2018 13:57:00 -5 17/07/2018 13:58:00 -5 17/07/2018 13:59:00 -5 17/07/2018

Finding Non-Zero Values/Indexes in Numpy

风流意气都作罢 提交于 2021-01-29 05:23:31
问题 I have a quite big numpy array with the shape of (12388, 4). The first two values are coordiantes and the second two key values. Some of the are zero. I want to filter through the array and find all indexes where both of the second two values are non-zeros. My code looks like this: slice_index,_ = np.where((slice[:,2:4]!=0)) slice_nonzero_values = slice[slice_index] The shape of the resulting array slice_nonzero_values is (18550,4). Thus something must have gone wrong, as the resulting array

Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

不想你离开。 提交于 2021-01-29 05:23:01
问题 I have added a large-scale key values pairs in python file which is about 20000 and i got below error after running a code. Trace (236167, 3) Process finished with exit code 137 (interrupted by signal 9: SIGKILL) File.py import pandas as pd import cupy.cuda as np import itertools from sklearn import metrics from sklearn.metrics import confusion_matrix, accuracy_score, roc_curve, auc import matplotlib.pyplot as plt from tqdm import tqdm np.Device(0).use() # -------------------------- # Data

Python numpy 2D array sum over certain indices

筅森魡賤 提交于 2021-01-29 05:21:09
问题 There is a 2-d array like this: img = [ [[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[2, 2, 2], [3, 2, 3], [6, 7, 6]], [[9, 8, 1], [9, 8, 3], [9, 8, 5]] ] And i just want to get the sum of certain indices which are like this: indices = [[0, 0], [0, 1]] # which means img[0][0] and img[0][1] # means here is represents There was a similar ask about 1-d array in stackoverflow in this link, but it got a error when I tried to use print(img[indices]) . Because I want to make it clear that the element of img

numba vstack doesn't work on list of arrays

半腔热情 提交于 2021-01-29 05:20:28
问题 It's weird for me vstack doesn't work with Numba when input is a list of array, it only works when the input is a tuple of array. Example code: @nb.jit(nopython=True) def stack(items): return np.vstack(items) stack((np.array([1,2,3]), np.array([4,5,6]))) returns array([[1, 2, 3], [4, 5, 6]]) but stack([np.array([1,2,3]), np.array([4,5,6])]) Throws an error TypingError: No implementation of function Function(<function vstack at 0x0000027271963488>) found for signature: >>>vstack(reflected list

How to find number of self intersection points on 2D plot?

风格不统一 提交于 2021-01-29 05:03:01
问题 I have two numpy arrays x and y : x = [-256.70946838 -188.26946838 -83.86946838 29.81053162 131.89053162 213.67053162 271.09053162 315.17053162 310.53053162 296.03053162 252.53053162 184.67053162 82.59053162 -33.40946838 -139.54946838 -213.78946838 -271.20946838 -317.02946838 -310.64946838 -298.46946838 -256.70946838] y = [ 9.71224758e-02 -3.19097822e-02 -4.80388145e-02 6.48644113e-02 -3.19097822e-02 9.71224758e-02 -1.57807500e-02 6.48644113e-02 -4.02877524e-01 -1.93200105e-01 6.48644113e-02

Create a range of decimal number using numpy does not work

为君一笑 提交于 2021-01-29 05:01:48
问题 I want to create a range of values between 521 and 522 with step 0.1. This is my code: ICD9CD1 = np.arange(521, 522, 0.1) The result is: array([521. , 521.1, 521.2, 521.3, 521.4, 521.5, 521.6, 521.7, 521.8, 521.9]) but when I want to covert it to a list, this is the result: np.arange(521, 522, 0.1).tolist() [521.0, 521.1, 521.2, 521.3000000000001, 521.4000000000001, 521.5000000000001, 521.6000000000001, 521.7000000000002, 521.8000000000002, 521.9000000000002] What part of my code is wrong? I

how to construct an index from percentage change time series?

心已入冬 提交于 2021-01-29 04:49:42
问题 consider the values below array1 = np.array([526.59, 528.88, 536.19, 536.18, 536.18, 534.14, 538.14, 535.44,532.21, 531.94, 531.89, 531.89, 531.23, 529.41, 526.31, 523.67]) I convert these into a pandas Series object import numpy as np import pandas as pd df = pd.Series(array1) And compute the percentage change as df = (1+df.pct_change(periods=1)) from here, how do i construct an index (base=100)? My desired output should be: 0 100.00 1 100.43 2 101.82 3 101.82 4 101.82 5 101.43 6 102.19 7