I get some solutions by searching here, which gave me tf.data.Dataset.from_generator. However my data generator process is quite complex and might require multiprocessing or