NumPy mmap: “ValueError: Size of available data is not a multiple of data-type size.”

问题

I'm trying to get data from "data.txt" into a numpy array and plot it with matplotlib. This is what each line of the data looks like:

"1" 11.658870417634 4.8159509459201

with about ten million lines.

I'm trying to get it into a memory map, but keep getting this error:

ValueError: Size of available data is not a multiple of data-type size.

Here is the code I am using:

import numpy
import matplotlib

matplotlib.use('Agg')
import matplotlib.pyplot as plt

datatype=[('index',numpy.int), ('floati',numpy.float32), ('floatq',numpy.float32)]
filename='data.txt'

def main():
    data = numpy.memmap(filename, datatype, 'r') 
    plt.plot(data['floati'],data['floatq'],'r,')
    plt.grid(True)
    plt.title("Signal-Diagram")
    plt.xlabel("Sample")
    plt.ylabel("In-Phase")
    plt.savefig('foo2.png')

if __name__ == "__main__":
    main()  

If you could please help me figure out where I'm going wrong, I would greatly appreciate it.

回答1:

That's a text file, not a binary file. memmap only works if the data is stored as binary (i.e. stored as the array would be stored in memory).

You can convert the file to binary by reading it in, line-by-line, and storing each converted line in a writable memmap.

回答2:

You're better off using numpy.genfromtxt or numpy.loadtxt. For example:

datatype=[('index',numpy.int), ('floati',numpy.float32), ('floatq',numpy.float32)]
filename='data.txt'

def main():
    data = numpy.genfromtxt(filename, dtype=datatype) 
    ...

来源：https://stackoverflow.com/questions/15303087/numpy-mmap-valueerror-size-of-available-data-is-not-a-multiple-of-data-type-s