In the previous videos, you experimented with
using RNNs to predict values in your sequence.
The results were good but they did need a bit of
improvement as you were hitting
strange plateaus in your predictions.
You experimented with using
different hyperparameters and you saw some improvement,
but perhaps a better approach would be to use
LSTMs instead of RNNs to see the impact.
We'll explore that in this video.
If you remember when you looked at RNNs,
they looked a little bit like this.
They had cells that took patches as inputs or X,
and they calculated a Y output
as well as the state vector,
that fed into the cell along with
the next X which then resulted in the Y,
and the state vector and so on.
The impact of this is that while state
is a factor in subsequent calculations,
its impacts can diminish greatly over timestamps.
LSTMs are the cell state
to this that keep a state throughout
the life of the training so that
the state is passed from cell to cell,
timestamp to timestamp, and it can be better maintained.
This means that the data
from earlier in the window can have
a greater impact on
the overall projection than in the case of RNNs.
The state can also be bidirectional
so that state moves forwards and backwards.
In the case of texts, this was really powerful.
Within the prediction of numeric sequences,
it may or may not be, and it'll
be interesting to experiment with.
I'm not going to go into a lot of detail here but
hundreds videos around LSTM are terrific.
From there, you can really
understand how they work under the hood.