How to create a tensorflow input pipeline for local dataset available as JSON in this format?

风格不统一 提交于 2021-01-29 12:01:08

问题


examples, metadata = tfds.load('ted_hrlr_translate/pt_to_en', with_info=True,
                               as_supervised=True)
train_examples, val_examples = examples['train'], examples['validation']


type(train_examples)

#Output
tensorflow.python.data.ops.dataset_ops._OptionsDataset
for i in train_examples:
    print(type(i), i)
<class 'tuple'>
(<tf.Tensor: shape=(), dtype=string, numpy=b'os astr\xc3\xb3nomos acreditam que cada estrela da gal\xc3\xa1xia tem um planeta , e especulam que at\xc3\xa9 um quinto deles tem um planeta do tipo da terra que poder\xc3\xa1 ter vida , mas ainda n\xc3\xa3o vimos nenhum deles .'>, <tf.Tensor: shape=(), dtype=string, numpy=b"astronomers now believe that every star in the galaxy has a planet , and they speculate that up to one fifth of them have an earth-like planet that might be able to harbor life , but we have n't seen any of them .">)


This code was taken from https://www.tensorflow.org/tutorials/text/transformer

来源:https://stackoverflow.com/questions/59948631/how-to-create-a-tensorflow-input-pipeline-for-local-dataset-available-as-json-in

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!