问题:

I'm using python to analyse some large files and I'm running into memory issues, so I've been using sys.getsizeof() to try and keep track of the usage, but it's behaviour with numpy arrays is bizarre. Here's an example involving a map of albedos that I'm having to open:

>>> import numpy as np >>> import struct >>> from sys import getsizeof >>> f = open('Albedo_map.assoc', 'rb') >>> getsizeof(f) 144 >>> albedo = struct.unpack('%df' % (7200*3600), f.read(7200*3600*4)) >>> getsizeof(albedo) 207360056 >>> albedo = np.array(albedo).reshape(3600,7200) >>> getsizeof(albedo) 80

Well the data's still there, but the size of the object, a 3600x7200 pixel map, has gone from ~200 Mb to 80 bytes. I'd like to hope that my memory issues are over and just convert everything to numpy arrays, but I feel that this behaviour, if true, would in some way violate some law of information theory or thermodynamics, or something, so I'm inclined to believe that getsizeof() doesn't work with numpy arrays. Any ideas?

回答1:

You can use array.nbytes for numpy arrays, for example:

>>> import numpy as np >>> from sys import getsizeof >>> a = [0] * 1024 >>> b = np.array(a) >>> getsizeof(a) 8264 >>> b.nbytes 8192

回答2:

The following works without the need to create a new array:

import numpy as np a = np.arange(81).reshape(9,9) a.size * a.dtype.itemsize

转载请标明出处:Python memory usage of numpy arrays

文章来源: Python memory usage of numpy arrays

标签

memory

python

array