In this video, we will be doing code preparation for Arnon's in order to preview the code that we will
write next.
As before, the main purpose of this lecture is to show you the syntax so that you are not surprised
when we use it later.
Now, whether or not you understand what this code is doing will be dependent on whether you watch the
previous lectures.
If you did, then you will know what's going on under the hood.
If not, that's OK too.
You can simply treat this like an API.
OK, so the first thing we will discuss is that the structure of this code is no different than what
we had for Anan's and CNN's, you'll find that the setup for this video, meaning the previous conceptual
lectures, is way more meaty than the actual implementation.
Thanks to our hard work in the previous section, we already know most of what to do.
The new parts are really just learning about the syntax for the new layers.
So without further ado, let's review what we've already learned.
We know that the first step is to build our model.
The syntax for that is precisely what we will learn very shortly.
The next step is to call the compile function.
The arguments to this, such as the loss, will be dependent on what task you are doing, but it works
the same way as before.
The next step is to call the fifth function.
Notice that like CNN's X Traina, next test should have the shape.
And by TBD after training the model, we would like to use it to make predictions.
Thus we use the predictive function parsing in the input data.
We would like to make predictions for OK, so this is exactly the same as before as promised.
Now, at this point, since we've only discussed the simple arnet, that's what we will look at how
to implement first.
But what you will learn is that this is no different than implementing LSD or Giuse.
You'll see that these other Ahren and units can be switched in and out with ease.
By the way, remember that I'm not saying simple RNA to mean that simple as an adjective to describe
the complexity of this kind of arnet.
Rather, this is the actual name we use for this kind of arnet.
In other words, we call this a simple ornon because that is its name.
And I'm not saying it's simple as opposed to complex.
So as before, the main thing we need to learn is how to build a model, since all the other steps remain
in this case, it's helpful to think about how engines are built and ends are just a series of dense
layers.
So in the simple case, we have input dense, dense with an iron in there is a self loop in the hidden
layer.
Thus we simply switch the dense with the simple on an object.
And yes, it's just that simple self loop means simple iron and no loop means dense.
One small note to make is that for Arnon's, the default activation is a hyperbolic tangent, although
you can use other activations if you like.
This is unlike the dense layer or the default activation as identity.
Now, again, it's always good to think about the shape of the data as it passes through the neural
network.
So in this case, we start with a multivariate time series of shape TBD.
This then goes through a simple Arnon layer, as you recall, the simple ernan, it takes in a time
series of vectors X one, x two all the way up to exploit T, it converts them into a time series of
hidden vectors, each one, each to all the way up to each big T recall that each head and vector depends
on the current input X and the past hidden vector.
By default, we only get to keep the final each of Big T, which includes all the information from the
past.
In other words, the output of the simple answer by default is an M size vector H of Big T.
From this, we pass this through one of our final dense layers, which is like a regular.
And so supposing we have K outputs, our output will be a vector of size K.
Now, as you recall, it's also possible to use an R and such that it not only returns the final hit
and vector of Big T, but all the hidden vectors of the Time series, in order to do this intenser flow,
we just need to pass in a single argument.
Return sequences equals true.
By doing this, we get back all the hidden vectors in a single array of size t biem.
So you can imagine this holding each of one, each of two all the way up to big T..
From here we have several options.
So suppose this is a many to many task where we want to have one prediction for every timestep.
So we want y one y to all the way up to y big T in this case we can just pass the output of the RNA
through a dense layer and this is what we automatically get.
So in this case, the output of the neural network is T by K note that there is no special syntax you
need to use.
Tensor flow automatically knows that the dense layer should be dealing with a time series.
In other words, the dense layer works for both single vectors and the time series of vectors.
If you pass in a single vector, you get back a single vector of size K.
If you pass in a series of vectors, then you get back a series of vectors.
Each of these K.
The next scenario to consider is when you're dealing with the many to one task in this case, we want
an output only for the final timestep.
Now, of course, the easy thing to do would be to not use return sequences at all and just keep your
big T..
But let's suppose you want to keep all the hidden vectors and take the maximum value over time.
In this case, you can apply global max pooling, just as you would with coins.
As you recall, this a time series of size T Biem into a single vector of size m completely eliminating
the time dimension.
Once we have a single vector of size m, we're back to the usual situation and we can pass this through
a final dense layer to get back a single vector of size k.
The last scenario I want to cover is when you want to stack multiple Afnan layers together, as you
recall, the input of an aunt and there must be a time series and therefore the previous Afnan layer
must output a time series.
Of course, you already know that this can be done by setting return sequences equal to true.
So here's an example of a many to one Arnim where we've stacked multiple iron and layers.
In this case, we start with an input time series of sighs TBD, then we pass through one Arnold and
layer with thirty two hidden units we set return sequence is equal to true so that the output has the
shape t by thirty two.
Of course this is still a multivariate time series which can be passed through more often and layers.
So the next line and layer also has thirty two hidden units.
But return sequences is now set to force, which is the default.
So the output of this layer will just be a vector of size 30 to the next step is to pass this through
one final dense layer where we get an output of size k.
OK, so I hope this was pretty simple.
You've now seen how to stack of multiple art in layers together by using return sequences.
Now, just as a little preview, I want to show you how easy it is to use Lithium's and G.R. use, even
if you don't know anything about how they work.
Observe that it's simply a matter of changing the type of object that's being used.
So if I want to use ANTM, then I simply type LSM instead of simple arnet.
If I want to use a jianhu, then I use Jianhu instead of Ellerston.
So in fact, you don't even have to understand how these Arnon units work in order to use them.
So if that's your preferred approach, then by all means use that approach.