Constructing 3D Pandas DataFrame

Question

I\'m having difficulty constructing a 3D DataFrame in Pandas. I want something like this

A               B               C
start    end    start    end    st         


        

Answer 1

As @Aaron mentioned in a comment above, panels have been deprecated. Also, @tlnagy mentioned his dataset would be likely to expand to more than 3 dimensions in the future.

This sounds like a good use-case for the xarray package, which provides semantically labelled arrays of arbitrarily many dimensions. Pandas and xarray have strong conversion support, and panels have been deprecated in favour of using xarray.

Initial setup of the problem.

import numpy as np

A = np.array([[7,11,56,45], [20,21,74,12]]).T
B = np.array([[42], [52]]).T
C = np.array([[90,213,9], [101, 34, 45]]).T

You can then create a three dimensional xarray.DataArray object like so:

import xarray

output_as_dataarray = xarray.concat(
    [
        xarray.DataArray(
            X,
            dims=["record", "edge"],
            coords={"record": range(X.shape[0]), "edge": ["start", "end"]},
        )
        for X in (A, B, C)
    ],
    dim="descriptor",
).assign_coords(descriptor=["A", "B", "C"])

We turn our three 2D numpy arrays into xarray.DataArray objects, and then concatenate them together along a new dimension.

Our output looks like so:

array([[[  7.,  20.],
        [ 11.,  21.],
        [ 56.,  74.],
        [ 45.,  12.]],

       [[ 42.,  52.],
        [ nan,  nan],
        [ nan,  nan],
        [ nan,  nan]],

       [[ 90., 101.],
        [213.,  34.],
        [  9.,  45.],
        [ nan,  nan]]])
Coordinates:
  * record      (record) int64 0 1 2 3
  * edge        (edge)