Error tokenizing data. C error: out of memory pandas python, large file csv

Question

I have a large csv file of 3.5 go and I want to read it using pandas.

This is my code:

import pandas as pd
tp = pd.read_csv(\'train_2011_2012_2013.csv\',         


        

Answer 1

You may try setting error_bad_lines = False when calling the csv file i.e.

import pandas as pd
df = pd.read_csv('my_big_file.csv', error_bad_lines = False)