Python file open/close every time vs keeping it open until the process is finished

Question

I have about 50 GB of text file and I am checking the first few characters each line and writing those to other files specified for that beginning text.

For example.

Answer 1

You should definitely try to open/close the file as little as possible

Because even comparing with file read/write, file open/close is far more expensive

Consider two code blocks:

f=open('test1.txt', 'w')
for i in range(1000):
    f.write('\n')
f.close()

and

for i in range(1000):
    f=open('test2.txt', 'a')
    f.write('\n')
    f.close()

The first one takes 0.025s while the second one takes 0.309s