Upper memory limit?

Question

Is there a limit to memory for python? I\'ve been using a python script to calculate the average values from a file which is a minimum of 150mb big.

Depending on the

Answer 1

Python can use all memory available to its environment. My simple "memory test" crashes on ActiveState Python 2.6 after using about

1959167 [MiB]

On jython 2.5 it crashes earlier:

 239000 [MiB]

probably I can configure Jython to use more memory (it uses limits from JVM)

Test app:

import sys

sl = []
i = 0
# some magic 1024 - overhead of string object
fill_size = 1024
if sys.version.startswith('2.7'):
    fill_size = 1003
if sys.version.startswith('3'):
    fill_size = 497
print(fill_size)
MiB = 0
while True:
    s = str(i).zfill(fill_size)
    sl.append(s)
    if i == 0:
        try:
            sys.stderr.write('size of one string %d\n' % (sys.getsizeof(s)))
        except AttributeError:
            pass
    i += 1
    if i % 1024 == 0:
        MiB += 1
        if MiB % 25 == 0:
            sys.stderr.write('%d [MiB]\n' % (MiB))

---

In your app you read whole file at once. For such big files you should read the line by line.