How to read the last MB of a very large text file

Question

I am trying to find a string near the end of a text file. The problem is that the text file can vary greatly in size. From 3MB to 4GB. But everytime I try to run a script to

Answer 1

The proposed answer using seek is a correct answer to your question, but I think it's not what you really want to do. Your solution loads the whole file into memory, just to get the last 20 lines. That's the main cause of your problem. The following would solve your memory issue:

for line in file(file_directory):
    if find_str in line:
        error = True

This will iterate over all lines in the file, but releasing the lines after they have been processed. I would guess, that this solution is already much faster than yours so no further optimization is needed. But if you really want to have just the last 20 lines, but the lines in a deque with a max length of 20.