numpy

How in python 3.6 to get data array from hdf5 file if dtype is “<u4”?

会有一股神秘感。 提交于 2021-01-29 16:21:29
问题 I want to get dataset with format {N, 16, 512, 128} as 4D numpy array from hdf5 file. N is a number of 3D arrays with {16, 512, 128} format. I try to do this: import os import sys import h5py as h5 import numpy as np import subprocess import re file_name = sys.argv[1] path = sys.argv[2] f = h5.File(file_name, 'r') data = f[path] print(data.shape) #{27270, 16, 512, 128} print(data.dtype) #"<u4" data = np.array(data, dtype=np.uint32) print(data.shape) Unfortunately, after data = np.array(data,

Why is np.linalg.norm(x,2) slower than solving it directly?

ⅰ亾dé卋堺 提交于 2021-01-29 16:09:26
问题 Example code: import numpy as np import math import time x=np.ones((2000,2000)) start = time.time() print(np.linalg.norm(x, 2)) end = time.time() print("time 1: " + str(end - start)) start = time.time() print(math.sqrt(np.sum(x*x))) end = time.time() print("time 2: " + str(end - start)) The output (on my machine) is: 1999.999999999991 time 1: 3.216777801513672 2000.0 time 2: 0.015042781829833984 It shows that np.linalg.norm() takes more than 3s to solve it, while the direct solution takes

Add to numpy 3D array

旧时模样 提交于 2021-01-29 15:51:16
问题 I have a numpy array of shape (100, 100, 6) . For the purposes of my work, I have to pad this array to have the 3rd dimension of 8 instead of 6, so the final shape should be (100, 100, 8) . I want to pad using zeros, something like this. np.zeros((100, 100, 2)) I have tried both numpy append and concatenate along axes 0 and 1, but it's not resulting in the way I want the zero padding to be appended. Could someone please forward me to the right direction? Thank you. 回答1: You can use np.dstack(

Pythonic and efficient way to find all the different intersections between two partitions of the same set

元气小坏坏 提交于 2021-01-29 15:20:33
问题 I need to find all the different intersections between two partitions of the same set. For example, if we have the following two partitions of the same set x = [[1, 2], [3, 4, 5], [6, 7, 8, 9, 10]] y = [[1, 3, 6, 7], [2, 4, 5, 8, 9, 10]] the required result is [[1], [2], [3], [4, 5], [6, 7], [8, 9, 10]]. In detail, we calculate the cartesian product between every subset of x and y, and for each of these products, we classify the elements in new subsets accordingly if they belong to the

numpy structured array sorting by multiple columns

时间秒杀一切 提交于 2021-01-29 15:20:14
问题 A minimal numpy structured array generator: import numpy as np index = np.arange(4) A = np.stack((np.sin(index), np.cos(index)),axis=1) B = np.eye(4).astype(int) C = np.array([1, 0, 1, 0], dtype=bool) goodies = [(a, b, c, d) for a, b, c, d in zip(index, A, B, C)] dt = [('index', 'int'), ('two_floats', 'float', 2), ('four_ints', 'int', 4), ('and_a_bool', 'bool')] s = np.array(goodies, dtype=dt) generates the minimal numpy structured array: array([(0, [ 0. , 1. ], [1, 0, 0, 0], True), (1, [ 0

Optimizing Calculations with numpy and numba Python

不打扰是莪最后的温柔 提交于 2021-01-29 15:08:13
问题 I am trying to make python run standard deviation functions faster with numba and numpy. However the problem is the for loop is very slow and I need alternatives so that I could make the code much faster. I iterated numba to the already existing numpy version however there is not much of a gain in performance. My original list_ has million of values within it thus it is taking a very long time to compute the standard deviation function. The list_ function down below is a very short numpy

Python find size of each sublist in a list

两盒软妹~` 提交于 2021-01-29 15:05:03
问题 I have a big list of floats and integers as given below. I want to find the length of each sublist by neglecting empty or single elements. big_list = [[137.83,81.80,198.56],0.0,[200.37,151.55,165.26, 211.84], 0.0,[1,2,3],4,[5,6,0,5,7,8],0,[2,1,4,5],[9,1,-2]] My present code: list_len = [] for i in big_list: list_len.append(len(i)) Present output: TypeError: object of type 'numpy.float64' has no len() Expected output: list_len = [3,4,3,6,4,3] # list_len should neglect elements like 0, 4 in big

numpy function to use for mathematical dot product to produce scalar

北慕城南 提交于 2021-01-29 15:03:38
问题 Question What numpy function to use for mathematical dot product in the case below? Backpropagation for a Linear Layer 回答1: Define sample (2,3) array: In [299]: dldx = np.arange(6).reshape(2,3) In [300]: w Out[300]: array([[0.1, 0.2, 0.3], [0. , 0. , 0. ]]) Element wise multiplication: In [301]: dldx*w Out[301]: array([[0. , 0.2, 0.6], [0. , 0. , 0. ]]) and summing on the last axis (size 3) produces a 2 element array: In [302]: (dldx*w).sum(axis=1) Out[302]: array([0.8, 0. ]) Your (6) is the

How to stream MP3 chunks given a NumPy array in Python?

让人想犯罪 __ 提交于 2021-01-29 14:41:44
问题 I'm struggling to find a solution for streaming synthesized audio from a Python server. The synthesized audio is incrementally generated and returned as a np.float32 NumPy array. It then needs to be transformed from a NumPy array into an MP3 chunk. Finally, the MP3 chunk is served via flask . Here is some pseudo-code: import numpy from flask import Flask from flask import Response app = Flask(__name__) sample_rate = 24000 def pcm_to_mp3(): raise NotImplementedError() def get_synthetic_audio()

Dataframe add element from a column based on values contiguity from another columns

戏子无情 提交于 2021-01-29 14:24:17
问题 I have a df like this: a=[1,2,10,11,15,16,17,18,30] b=[5,6,7,8,9,1,2,3,4] df=pd.DataFrame(list(zip(a,b)),columns=['s','i']) Using a I need to add elements of b. Result I would like: (1-2)=5+6=11 (10-11)=7+8=15 (15-18)=9+1+2+3=15 (30)=4 My idea was to create a list of values that are continuous, take the difference(+1) and use it to calculate the sum of the corresponding b elements. #find continuous integer def r (nums): nums= list(df['s']) gaps = [[s, e] for s, e in zip(nums, nums[1:]) if s+1