tensorflow-datasets

I am reading Ganegedara‘s NLP with Tensorflow. The introduction to input pipieline has the following example import tensorflow as tf import numpy as np import os # Defining the graph and session graph = tf.Graph() # Creates a graph session = tf.InteractiveSession(graph=graph) # Creates a session # The filename queue filenames = ['test%d.txt'%i for i in range(1,4)] filename_queue = tf.train.string_input_producer(filenames, capacity=3, shuffle=True,name='string_input_producer') # check if all files are there for f in filenames: if not tf.gfile.Exists(f): raise ValueError('Failed to find file: '

I want to use in Jupyter (version 6.0.0) with Python3 tensorflow_datasets. Doing that results in an error message I cannot seem to fathom what the problem is. I made a new kernel for Python which should utilize the tensorflow_datasets. The following steps were taken (In anaconda using my administrator option). 1. conda info --envs 2. conda create --name py3-TF2.0 python=3 3. conda activate py3-TF2.0 4. pip install matplotlib 5. pip install tensorflow==2.0.0-alpha0 6. pip install ipykernel 7. conda install nb_conda_kernels 8. pip install tensorflow-datasets Upon closing I restarted my laptop.

I have 5 tfrecords files, one for each object. While training I want to read data equally from all the 5 tfrecords i.e. if my batch size is 50, I should get 10 samples from 1st tfrecord file, 10 samples from the second tfrecord file and so on. Currently, it just reads sequentially from all the three files i.e. I get 50 samples from the same record. Is there a way to sample from differnt tfrecords files? I advise you to read the tutorial by @mrry on tf.data . On slide 42 he explains how to use tf.data.Dataset.interleave() to read multiple tfrecord files at the same time. For instance if you

I have .txt files in a directory of format <text>\t<label> . I am using the TextLineDataset API to consume these text records: filenames = ["/var/data/file1.txt", "/var/data/file2.txt"] dataset = tf.contrib.data.Dataset.from_tensor_slices(filenames) dataset = dataset.flat_map( lambda filename: ( tf.contrib.data.TextLineDataset(filename) .map(_parse_data))) def _parse_data(line): line_split = tf.string_split([line], '\t') features = {"raw_text": tf.string(line_split.values[0].strip().lower()), "label": tf.string_to_number(line_split.values[1], out_type=tf.int32)} parsed_features = tf.parse

I have trained a model using the tf.data.Dataset API, so my training code looks something like this with graph.as_default(): dataset = tf.data.TFRecordDataset(tfrecord_path) dataset = dataset.map(scale_features, num_parallel_calls=n_workers) dataset = dataset.shuffle(10000) dataset = dataset.padded_batch(batch_size, padded_shapes={...}) handle = tf.placeholder(tf.string, shape=[]) iterator = tf.data.Iterator.from_string_handle(handle, train_dataset.output_types, train_dataset.output_shapes) batch = iterator.get_next() ... # Model code ... iterator = dataset.make_initializable_iterator() with

I am converting some legacy code to use the Dataset API - this code uses feed_dict to feed one batch to the train operation (actually three times) and then recalculates the losses for display using the same batch . So I need to have an iterator that returns the exact same batch two (or several) times. Unfortunately, I can't seem to find a way of doing it with tensorflow datasets - is it possible? You can repeat individual elements of a Dataset using Dataset.flat_map() , Dataset.from_tensors() and Dataset.repeat() together. For example, to repeat elements twice: NUM_REPEATS = 2 dataset = tf