How can I split a large file csv file (7GB) in Python

Question

I have a 7GB csv file which I\'d like to split into smaller chunks, so it is readable and faster for analysis in Python on a notebook. I would like to grab a sm

Answer 1

Maybe something like this?

#!/usr/local/cpython-3.3/bin/python

import csv

divisor = 10

outfileno = 1
outfile = None

with open('big.csv', 'r') as infile:
    for index, row in enumerate(csv.reader(infile)):
        if index % divisor == 0:
            if outfile is not None:
                outfile.close()
            outfilename = 'big-{}.csv'.format(outfileno)
            outfile = open(outfilename, 'w')
            outfileno += 1
            writer = csv.writer(outfile)
        writer.writerow(row)