Creating dataframe from a dictionary where entries have different lengths

Question

Say I have a dictionary with 10 key-value pairs. Each entry holds a numpy array. However, the length of the array is not the same for all of them.

How can I create a

Answer 1

Use pandas.DataFrame and pandas.concat

• The following code will create a list of DataFrames with pandas.DataFrame, from a dict of uneven arrays, and then concat the arrays together in a list-comprehension.
  • This is a way to create a DataFrame of arrays, that are not equal in length.
  • For equal length arrays, use df = pd.DataFrame({'x1': x1, 'x2': x2, 'x3': x3})

import pandas as pd
import numpy as np


# create the uneven arrays
mu, sigma = 200, 25
np.random.seed(365)
x1 = mu + sigma * np.random.randn(10, 1)
x2 = mu + sigma * np.random.randn(15, 1)
x3 = mu + sigma * np.random.randn(20, 1)

data = {'x1': x1, 'x2': x2, 'x3': x3}

# create the dataframe
df = pd.concat([pd.DataFrame(v, columns=[k]) for k, v in data.items()], axis=1)

Use pandas.DataFrame and itertools.zip_longest

• For iterables of uneven length, zip_longest fills missing values with the fillvalue.
• The zip generator needs to be unpacked, because the DataFrame constructor won't unpack it.

from itertools import zip_longest

# zip all the values together
zl = list(zip_longest(*data.values()))

# create dataframe
df = pd.DataFrame(zl, columns=data.keys())

plot

df.plot(marker='o', figsize=[10, 5])

dataframe

           x1         x2         x3
0   232.06900  235.92577  173.19476
1   176.94349  209.26802  186.09590
2   194.18474  168.36006  194.36712
3   196.55705  238.79899  218.33316
4   249.25695  167.91326  191.62559
5   215.25377  214.85430  230.95119
6   232.68784  240.30358  196.72593
7   212.43409  201.15896  187.96484
8   188.97014  187.59007  164.78436
9   196.82937  252.67682  196.47132
10        NaN  223.32571  208.43823
11        NaN  209.50658  209.83761
12        NaN  215.27461  249.06087
13        NaN  210.52486  158.65781
14        NaN  193.53504  199.10456
15        NaN        NaN  186.19700
16        NaN        NaN  223.02479
17        NaN        NaN  185.68525
18        NaN        NaN  213.41414
19        NaN        NaN  271.75376