Using genfromtxt to import csv data with missing values in numpy

Question

I have a csv file that looks something like this (actual file has many more columns and rows):

1,2,3,4,5
6,7,8,9,10
11,12,13,14,15
16

Say the name of the file is info.csv If I try to import this using

data = numpy.genfromtxt('info.csv', delimiter = ',')

then I get the following error:

ValueError: Some errors were detected ! Line #4 (got 1 columns instead of 5)

If I use,

data = numpy.genfromtxt('info.csv', delimiter = ',', skip_footer = 1) 

both lines with data 16 and with data 11, 12, 13, 14, 15 are skipped. I don't understand why the line with 11, 12, 13, 14, 15 is being skipped. I would appreciate any help on how I can appropriately use the genfromtxt to import first three lines in the above file.

Thanks

Answer 1

if you can ignore the 16 at the end of the file try using the

invalid_raise (bool, optional) parameter if set to False it ignores all incomplete lines without throwing an exception

see here (its the last parameter before the examples) http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html

Answer 2

The command filling_values also helped me. I set it to zero. Thus each empty value is set to zero. It probably doesn't always make sense, but maybe it will help you.