question about splitting a large file

Question

Hey I need to split a large file in python into smaller files that contain only specific lines. How do I do this?

Answer 1

You're probably going to want to do something like this:

big_file = open('big_file', 'r')
small_file1 = open('small_file1', 'w')
small_file2 = open('small_file2', 'w')

for line in big_file:
    if 'Charlie' in line: small_file1.write(line)
    if 'Mark' in line: small_file2.write(line)

big_file.close()
small_file1.close()
small_file2.close()

Opening a file for reading returns an object that allows you to iterate over the lines. You can then check each line (which is just a string of whatever that line contains) for whatever condition you want, then write it to the appropriate file that you opened for writing. It is worth noting that when you open a file with 'w' it will overwrite anything already written to that file. If you want to simply add to the end, you should open it with 'a', to append.

Additionally, if you expect there to be some possibility of error in your reading/writing code, and want to make sure the files are closed, you can use:

with open('big_file', 'r') as big_file: