Python: find regexp in a file

Question

Have:

f = open(...)  
r = re.compile(...)

Need:
Find the position (start and end) of a first matching regexp in a big file?
(star

Answer 1

One way to search through big files is to use the mmap library to map the file into a big memory chunk. Then you can search through it without having to explicitly read it.

For example, something like:

size = os.stat(fn).st_size
f = open(fn)
data = mmap.mmap(f.fileno(), size, access=mmap.ACCESS_READ)

m = re.search(r"867-?5309", data)

This works well for very big files (I've done it for a file 30+ GB in size, but you'll need a 64-bit OS if your file is more than a GB or two).