You can use tf.estimator.inputs.pandas_input_fn in your make_input_fn(X, y, num_epochs) function. I've not managed to get it to work with a multi-index, however. I fixed this issue by turning it into a standard integer index, using df.reset_index(drop=True)