genfromtxt

Despite the advice from the previous questions: -9999 as missing value with numpy.genfromtxt() Using genfromtxt to import csv data with missing values in numpy I still am unable to process a text file that ends with a missing value, a.txt: 1 2 3 4 5 6 7 8 I've tried multiple arrangements of options of missing_values , filling_values and can not get this to work: import numpy as np sol = np.genfromtxt("a.txt", dtype=float, invalid_raise=False, missing_values=None, usemask=True, filling_values=0.0) print sol What I would like to get is: [[1.0 2.0 3.0] [4.0 5.0 6.0] [7.0 8.0 0.0]] but instead I

I want to read in a standard-ascii csv file into numpy, which consists of floats and strings. E.g., ZINC00043096,C.3,C1,-0.1540,methyl ZINC00043096,C.3,C2,0.0638,methylene ZINC00043096,C.3,C4,0.0669,methylene ZINC00090377,C.3,C7,0.2070,methylene ... Whatever I tried, the resulting array would look like E.g., all_data = np.genfromtxt(csv_file, dtype=None, delimiter=',') [(b'ZINC00043096', b'C.3', b'C1', -0.154, b'methyl') (b'ZINC00043096', b'C.3', b'C2', 0.0638, b'methylene') (b'ZINC00043096', b'C.3', b'C4', 0.0669, b'methylene') However, I want to save a step for the byte-string conversion and

I have a text file containing simulation data (60 columns, 100k rows): a b c 1 11 111 2 22 222 3 33 333 4 44 444 ... where in the first row are variable names, and beneath (in columns) is the corresponding data (float type). I need to use all these variables with their data in Python for further calculations. For example, when I insert: print(b) I need to receive the values from the second column. I know how to import data: data=np.genfromtxt("1.txt", unpack=True, skiprows = 1) Assign variables "manually": a,b,c=np.genfromtxt("1.txt", unpack=True, skiprows = 1) But I'm having trouble with

I'm trying to use genfromtxt with Python3 to read a simple csv file containing strings and numbers. For example, something like (hereinafter "test.csv"): 1,a 2,b 3,c with Python2, the following works well: import numpy data=numpy.genfromtxt("test.csv", delimiter=",", dtype=None) # Now data is something like [(1, 'a') (2, 'b') (3, 'c')] in Python3 the same code returns [(1, b'a') (2, b'b') (3, b'c')] . This is somehow expected due to the different way Python3 reads the files. Therefore I use a converter to decode the strings: decodef = lambda x: x.decode("utf-8") data=numpy.genfromtxt("test.csv

I have a file with some metadata, and then some actual data consisting of 2 columns with headings. Do I need to separate the two types of data before using genfromtxt in numpy? Or can I somehow split the data maybe? What about placing the file pointer to the end of the line just above the headers, and then trying genfromtxt from there? Thanks The format of the file is shown below: &SRS <MetaDataAtStart> multiple=True Wavelength (Angstrom)=0.97587 mode=assessment background=True issid=py11n2g noisy=True </MetaDataAtStart> &END Two Theta(deg) Counts(sec^-1) 10.0 41.0 10.1 39.0 10.2 38.0 10.3 38

I have a csv file that looks something like this (actual file has many more columns and rows): 1,2,3,4,5 6,7,8,9,10 11,12,13,14,15 16 Say the name of the file is info.csv If I try to import this using data = numpy.genfromtxt('info.csv', delimiter = ',') then I get the following error: ValueError: Some errors were detected ! Line #4 (got 1 columns instead of 5) If I use, data = numpy.genfromtxt('info.csv', delimiter = ',', skip_footer = 1) both lines with data 16 and with data 11, 12, 13, 14, 15 are skipped. I don't understand why the line with 11, 12, 13, 14, 15 is being skipped. I would

I'm running genfromtxt like below: date_conv = lambda x: str(x).replace(":", "/") time_conv = lambda x: str(x) a = np.genfromtxt(input.txt, delimiter=',', skip_header=4, usecols=[0, 1] + radii_indices, converters={0: date_conv, 1: time_conv}) Where input.txt is from this gist . When I look at the results, it is a 1D array not a 2D array: >>> np.shape(a) (918,) It seems to be an array of tuples instead: >>> a[0] ('06/03/2006', '08:27:23', 6.4e-05, 0.000336, 0.001168, 0.002716, 0.004274, 0.004658, 0.003756, 0.002697, 0.002257, 0.002566, 0.003522, 0.004471, 0.00492, 0.005602, 0.006956, 0.008442,