I want to flatten JSON column in a Pandas DataFrame

Question

I have an input dataframe df which is as follows:

id  e
1   {\"k1\":\"v1\",\"k2\":\"v2\"}
2   {\"k1\":\"v3\",\"k2\":\"v4\"}
3   {\"k1\":\"v5\",\"k2\":\"v6\"}         


        

Answer 1

If your column is not already a dictionary, you could use map(json.loads) and apply pd.Series:

s = df['e'].map(json.loads).apply(pd.Series).add_prefix('e.')

Or if it is already a dictionary, you can apply pd.Series directly:

s = df['e'].apply(pd.Series).add_prefix('e.')

Finally use pd.concat to join back the other columns:

>>> pd.concat([df.drop(['e'], axis=1), s], axis=1).set_index('id')    
id e.k1 e.k2
1    v1   v2
2    v3   v4
3    v5   v6