Writing large Pandas Dataframes to CSV file in chunks

Question

How do I write out a large data file to a CSV file in chunks?

I have a set of large data files (1M rows x 20 cols). However, only 5 or so columns of that data is of

Answer 1

Check out the chunksize argument in the to_csv method. Here are the docs.

Writing to file would look like:

df.to_csv("path/to/save/file.csv", chunksize=1000, cols=['TIME','STUFF'])