-: Hello, and welcome to this Python tutorial.
Alright, so we just built the architecture
of our neural network
with the init function of our Network class.
And now we're gonna make a second function
which is going to be the Forward function,
and that's the function that will activate the neurons.
That is, it's the function
that will perform Forward Propagation.
So let's do this.
Let's make this function.
Let's call it Forward, as we just said.
And this function is going to take two arguments.
First is, as usual, Self, you know
to be able to use the variables of the object,
because we're gonna use fc1 and fc2.
So, we need this Self to be able to use these variables.
And then, we're gonna need a second argument,
which is our input.
And we're gonna call it State
because State is exactly the input of our neural networks.
You know, that's the State.
There are the inputs entering the neural network.
And then as output, we will have the Q-values
of the three possible actions;
go left, go straight or go right.
But, we don't need to input it as an argument here
because, that's exactly what we want to return.
So this Forward function is not only going to
activate the neurons, but also
and mostly, it'll return the Q-values
for each possible action, depending on the input state here.
Alright.
So that's the two arguments we need.
And now let's go inside the function
and let's specify what we want it to do.
Okay, So the first thing we're gonna do is
activate the hidden neurons
and we're gonna call the hidden neurons by the variable X.
So X represents the hidden neurons.
And so how are we going to activate them?
Well, of course, we're gonna take our input neurons.
We're gonna use our first full connection fc1 to
get the hidden neurons, and then we're going to
apply an activation function on them, which will be
rectifier function.
So how are we going to do that?
Remember? we imported the torch.nn.functional module
that contains all the functions in PyTorch to
implement a neural network, and we gave it the shortcut F.
So actually what we're gonna do now is we're going to
use one of these functions from the functional module.
And this function is the ReLU function.
So what is ReLU?
Relu is the rectifier function that you saw
in the intuition lectures.
That's just the name given to the rectifier function.
But since this function is taken from an INDOT functional
which was given the shortcut F; we need to type here
first F., and then that's where we can take
this function.
And actually, if I type Re, here we go,
we have the ReLU function. So that's the rectifier function
that will activate the hidden neurons, that is, X.
So, in this ReLU function, now we understand perfectly
what we have to input.
That's the neurons that we want to activate
that is, the hidden neurons.
And so to get these hidden neurons, we're gonna
take our first full connection fc1, which we
will apply to our input neurons to go from the
input neurons to the hidden neurons.
So let's take our first full connection, fc1.
But our first full connection is a variable of our object.
Therefore, we need to type here first self.fc1.
Here we go.
That's the first full connection of our neural network.
And, inside this first full connection
we are gonna input our input states to go
from the input neurons to the hidden neurons.
And so since we gave it the name state
well, here we have to input state.
And there we go.
We now get activated hidden neurons.
Alright? And now that we have the hidden neurons
we are going to return the output neurons.
So, next line.
And as you understood, the output neurons correspond
to our actions, but these are not the actions directly.
These are the Q-values
because we're building a deep Q-learning model
that combines a deep learning model to Q-learning.
And therefore, we use Q-learning here to get the Q-values
for each of our actions.
And then later using a softmax or an argmax,
we will get the final action.
So here, the variable I'm
about to introduce will correspond to the output neurons.
And since the output neurons are the Q-values, well
I'm gonna call this variable Q-values.
There we go.
So Q-values.
And now, we directly take our full connection,
which is the variable fc2
but the variable from our object.
So we take here self.fc2.
And of course, here we input the neurons
of the left side of this full connection.
That is what we got from the first line, which is X.
So, X, there we go.
We now get our Q-values.
That's the output neurons of our neural network.
Okay? And then last line of codes.
Of course
this forward function is used to return these Q-values.
So we just have to add a return and simply Q-values.
And that will return the Q-values
for each possible action.
Go left, go straight or go right.
Alright, so congratulations.
We are done with our first class, and actually,
we are done making the architecture of the neural network.
Remember, this is not a finished job.
You can always improve the architecture
of the neural network by trying different ones.
So feel free to do that by adding more neurons here.
For example, if you want to add 50 hidden neurons,
you can just replace the 30 here and the 30 here by 50,
50 and 50, and then you can add some more hidden layers
by making some new full connections.
Well, that's really the job of an artist.
There is no general rule
of what would be the best architecture in each situation.
So that's why we have to experiment.
But, let's already try with that. You will see
that we'll get eventually a pretty good self-driving car.
Alright, and now we're going to make the next class
which is about experience replay,
and we will be making that in the next three tutorials.
Until then, enjoy AI.