How to give a list of integers as input in Tensorflow Dataset?

Question

We are trying to finetune/train a pretrained RoBERTa model using tensorflow. For this we have to create a tf.data.Dataset from our dataframe.

The dataframe looks like