Splitting large text file into smaller text files by line numbers using Python

Question

I have a text file say really_big_file.txt that contains:

line 1
line 2
line 3
line 4
...
line 99999
line 100000

I would like to write a Py

Answer 1

I had to do the same with 650000 line files.

Use the enumerate index and integer div it (//) with the chunk size

When that number changes close the current file and open a new one

This is a python3 solution using format strings.

chunk = 50000  # number of lines from the big file to put in small file
this_small_file = open('./a_folder/0', 'a')

with open('massive_web_log_file') as file_to_read:
    for i, line in enumerate(file_to_read.readlines()):
        file_name = f'./a_folder/{i // chunk}'
        print(i, file_name)  # a bit of feedback that slows the process down a

        if file_name == this_small_file.name:
            this_small_file.write(line)

        else:
            this_small_file.write(line)
            this_small_file.close()
            this_small_file = open(f'{file_name}', 'a')