Reading multiple JSON records into a Pandas dataframe

Question

I\'d like to know if there is a memory efficient way of reading multi record JSON file ( each line is a JSON dict) into a pandas dataframe. Below is a 2 line example with wo

Answer 1

If you are trying to save memory, then reading the file a line at a time will be much more memory efficient:

with open('test.json') as f:
    data = pd.DataFrame(json.loads(line) for line in f)

Also, if you import simplejson as json, the compiled C extensions included with simplejson are much faster than the pure-Python json module.