How can I read large text files in Python, line by line, without loading it into memory?

前端 未结 15 1567
臣服心动
臣服心动 2020-11-22 03:32

I need to read a large file, line by line. Lets say that file has more than 5GB and I need to read each line, but obviously I do not want to use readlines() bec

15条回答
  •  借酒劲吻你
    2020-11-22 04:16

    I demonstrated a parallel byte level random access approach here in this other question:

    Getting number of lines in a text file without readlines

    Some of the answers already provided are nice and concise. I like some of them. But it really depends what you want to do with the data that's in the file. In my case I just wanted to count lines, as fast as possible on big text files. My code can be modified to do other things too of course, like any code.

提交回复
热议问题