问题
I am new to tensorflow, i am now a little confused about the meaning of batch_size. As commonly known that the meaning of batch_size is the number of samples for each batch, but according to the code in ptb_word_lm, it seems not:
reader.py:
data_len = tf.size(raw_data) #the number of words in dataset
batch_len = data_len // batch_size
What does batch_len mean? The number of batches?
ptb_word_lm.py:
self.epoch_size = ((len(data) // batch_size) - 1) // num_steps
What does epoch_size mean? The number of sequences in each batch?
But if batch_size means the number of batches, then everything make sense. have I misunderstood something?
回答1:
There are a few different concepts here: epoch, step, batch, and unroll steps for LSTM.
At the highest level, you train a network with multiple epochs. In each epoch, you will go through and use all training data (usually in a random order) by steps; In each step, you train a batch of samples.
I think the confusion here added by LSTM is that: each step, you will train a sequence of batches, instead of a single batch. The length of sequence is the number of unroll steps (num_steps).
来源：https://stackoverflow.com/questions/41338553/the-meaning-of-batch-size-in-ptb-word-lm-lstm-model-of-tensorflow