How can I read large text files in Python, line by line, without loading it into memory?

前端 未结 15 1499
臣服心动
臣服心动 2020-11-22 03:32

I need to read a large file, line by line. Lets say that file has more than 5GB and I need to read each line, but obviously I do not want to use readlines() bec

15条回答
  •  北恋
    北恋 (楼主)
    2020-11-22 04:10

    I couldn't believe that it could be as easy as @john-la-rooy's answer made it seem. So, I recreated the cp command using line by line reading and writing. It's CRAZY FAST.

    #!/usr/bin/env python3.6
    
    import sys
    
    with open(sys.argv[2], 'w') as outfile:
        with open(sys.argv[1]) as infile:
            for line in infile:
                outfile.write(line)
    

提交回复
热议问题