In MNIST LSTM examples, I don\'t understand what \"hidden layer\" means. Is it the imaginary-layer formed when you represent an unrolled RNN over time?
Why is the <
I think this is a correctly answer for your question. LSTM always make confusion.
You can refer this blog for more detail Animated RNN, LSTM and GRU