Is it possible to speed-up python IO?

Question

Consider this python program:

import sys

lc = 0
for line in open(sys.argv[1]):
    lc = lc + 1

print lc, sys.argv[1]

Running it on my 6GB

Answer 1

You can't get any faster than the maximum disk read speed.

In order to reach the maximum disk speed you can use the following two tips:

1. Read the file in with a big buffer. This can either be coded "manually" or simply by using io.BufferedReader ( available in python2.6+ ).
2. Do the newline counting in another thread, in parallel.