Reading data from text file with missing values

Question

I want to read data from a file that has many missing values, as in this example:

1,2,3,4,5
6,,,7,8
,,9,10,11

I am using the numpy.loadtxt

Answer 1

I'd probably use genfromtxt:

>>> from numpy import genfromtxt
>>> genfromtxt("missing1.dat", delimiter=",")
array([[  1.,   2.,   3.,   4.,   5.],
       [  6.,  nan,  nan,   7.,   8.],
       [ nan,  nan,   9.,  10.,  11.]])

and then do whatever with the nans (change them to something, use a mask instead, etc.) Some of this could be done inline:

>>> genfromtxt("missing1.dat", delimiter=",", filling_values=99)
array([[  1.,   2.,   3.,   4.,   5.],
       [  6.,  99.,  99.,   7.,   8.],
       [ 99.,  99.,   9.,  10.,  11.]])