In this video, we will be doing code preparation for our own ends in order to preview the code that
we will right next as before.
The main purpose of this lecture is to show you the syntax so that you are not surprised when we use
it later.
Now, whether or not you understand what this code is doing will be dependent on whether you watch the
previous lectures.
If you did, then you will know what's going on under the hood.
If not, that's OK, too.
You can simply treat this like an API.
OK, so the first thing we will discuss is that the structure of this code is no different than what
we had for Anand's and CNN's.
You'll find that the setup for this video, meaning the previous conceptual lectures, is way more meaty
than the actual implementation.
Thanks to our hard work in the previous section, we already know most of what to do with the new parts
are really just learning about the syntax for the new layers.
So without further ado, let's review what we've already learned.
We know that the first step is to build our model.
The syntax for that is precisely what we will learn very shortly.
The next step is to call the compile function.
The arguments to this, such as the loss, will be dependent on what task you are doing, but it works
the same way as before.
The next step is to call the fit function.
Notice that, like CNN's extra and next test, should have the shape and bite C by D.
After training the model, we would like to use it to make predictions.
Thus, we use the predict function passing in the input data.
We would like to make predictions for OK, so this is exactly the same as before as promised.
Now, at this point since we've only discussed the simple Arnette, that's what we will look at how
to implement first.
But what you will learn is that this is no different than implementing LSD, Yams argues.
You'll see that these other Arnon units can be switched in and out with ease.
By the way, remember that I'm not saying simple Arnon to mean that simple as an adjective to describe
the complexity of this kind of Arnon.
Rather, this is the actual name we use for this kind of Arnette.
In other words, we call this a simple Arnon, because that is its name.
And I'm not saying it's simple as opposed to complex.
So as before, the main thing we need to learn is how to build a model, since all the other steps remain.
In this case, it's helpful to think about how aliens are built and ends are just a series of dense
layers.
So in the simple case, we have input dense, dense with an aunt in there is a self loop in the hidden
layer.
Thus, we simply switch the dense with the simple art and objects.
And yes, it's just that simple.
Self loop means simple arnon, and no loop means dense.
One small note to make is that for aunt ends, the default activation is a hyperbolic tension, although
you can use other activations if you like.
This is unlike the dense layer or the default activation as identity.
Now again, it's always good to think about the shape of the data as it passes through the neural network.
So in this case, we start with a multivariate time series of shape activity.
This then goes through a simple Arnon layer as you recall the simple answer, and it takes in a time
series of vectors x one x two all the way up to X Big T.
It converts them into a time series of hidden vectors, each one each to all the way up to Big T.
Recall that each head and vector depends on the current input X and the past hidden vector.
By default, we only get to keep the final each of the eighty, which includes all the information from
the past.
In other words, the output of the simple R9 by default is an m size vector h of Big T.
From this, we pass this through one of our final dense layers, which is like a regular A9.
So supposing we have key outputs, are output will be a vector of size K.
Now, as you recall, it's also possible to use an acronym such that it not only returns the final hidden
vector of Big T, but all the hidden vectors of the time series in order to do this in TensorFlow.
We just need to pass in a single argument.
Return sequences equals true.
By doing this, we get back all the hidden vectors in a single array of size T by em.
So you can imagine this holding each of one, each of two all the way up to big.
From here, we have several options.
So suppose this is a many to many task where we want to have one prediction for every time step.
So we want y one y two all the way up to y big T.
In this case, we can just pass the output of the Arnon through a dense layer.
And this is what we automatically get.
So in this case, the output of the neural network is T by K.
Note that there is no special syntax you need to use.
TensorFlow automatically knows that the dense layer should be dealing with a time series.
In other words, the dense layer works for both single vectors and the time series of vectors.
If you pass in a single vector, you get back a single vector of size K if you pass in a series of vectors.
Then you get back a series of vectors each of size K.
The next scenario to consider is when you're dealing with the many to one task, in this case, we want
an output only for the final time step.
Now, of course, the easy thing to do would be to not use return sequences at all and just keep your
big T.
But let's suppose you want to keep all the hidden vectors and take the maximum value over time.
In this case, you can apply global max pooling just as you would with scenes, as you recall.
This converts a time series of size T by M into a single vector of size m, completely eliminating the
time dimension.
Once we have a single vector of size M, we're back to the usual situation and we can pass this through
a final dense layer to get back a single vector of size K.
The last scenario I want to cover is when you want to stack multiple Arnon layers together, as you
recall, the input of an iron and there must be a time series.
And therefore the previous Arnon layer must output a time series.
Of course, you already know that this can be done by setting return sequences equal to true.
So here's an example of a many to one hour and then where we've stacked multiple aren't in layers.
In this case, we start with an input time series of size TBD.
Then we pass this through one Arnold layer with 32 hidden units.
We set return sequences equal to true so that the output has the shape T by 32.
Of course, this is still a multivariate time series which can be passed through moran and layers.
So the next start in layer also has 32 hidden units.
But return sequences is now set to false, which is the default.
So the output of this layer will just be a vector of size 32.
The next step is to pass this through one final dense layer where we get an output of size K.
OK, so I hope this was pretty simple.
You've now seen how to stack multiple aren't layers together by using return sequences.
Now, just as a little preview, I want to show you how easy it is to use Elysium's and use, even if
you don't know anything about how they were observed, that it's simply a matter of changing the type
of object that's being used.
So if I want to use in less time, then I simply type SDM instead of simple n.
If I want to use a giu, then I use giu instead of lithium.
So in fact, you don't even have to understand how these Arnon units work in order to use them.
So if that's your preferred approach, then by all means use that approach.
So as with our previous study on CNN's, it is helpful to think about Arnold's in terms of a more natural
kind of input, such as a time series with length T and D input features.
This is the same as when we studied scenes where we considered the input to be 2D images, even though
we eventually use them for one D signals, which we eventually thought of as word embeddings.
Note that the same situation applies in this case.
Initially, it's easier to think of inputs which have the shape T by D.
This is just a time series of length T and D different measurement sensors.
Of course, in practice, this won't be a time series we recorded using sensors, but word embeddings.
As you recall, this is what we get when we have an input with sequence length T, followed by an embedding
layer with embedding Dimension D.
The output of this is also T by D.
Therefore, without loss of generality, we can always think of Arnaz as having an input of shape t
by D.