Tutor: Hello and welcome to this tutorial.
All right, so now we are going
to implement our artificial intelligence from scratch.
We're going to code it line by line,
and in this first code section we're going
to import the libraries.
But before we start with this first code section,
I would like to explain the connection between the AI
and our map.py file.
That is, why are we implementing this for the map?
What is the purpose of our AI
and where will we be using it?
So it's actually very simple.
We are only making our AI
to select the right action at each time.
So, okay, we import the DQN class from our AI file.
So we will be making this DQN class in this file,
but then we import it only to select the right action
to play at each time.
And we select this action exactly at this line.
Action equals brain update, last reward, last signal.
Last signal will be the input of the neural network.
You know, it's composed of the three signals
of the sensors plus the orientation
and minus orientation.
So that's the input,
but then the output is the action to play.
And that's only what we'll be taking from our AI file
that we're about to make.
So keep that in mind, it's very simple.
We first import the DQN class from the AI
then we create the object brain from the DQN class
which takes as input the encoded vectors
for the states of five dimensions, the three signals,
plus orientation, plus minus orientation,
the reactions go left, go straight, or go right,
and then this gamma parameter,
that's the only parameters of the DQN class
that we will be making.
And then once we create that object,
we select in the game class,
the action to play at each time.
And that depends on the last reward
and the last signal, which is the input.
And that's all.
That's the only purpose of making this AI.
That's in order to have a real artificial intelligence
playing the right actions at each time, the right move,
instead of having random actions like we observed
in the previous tutorial.
All right, so let's do this.
Let's implement our artificial intelligence.
And as we said, we are gonna start by importing
all the libraries that we'll be using to implement it.
So that way we will have all the tools we need.
All right, so let's start with the first one.
The first one is the inevitable, the NumPy library.
The NumPy library, I always recommend to import it.
It's the library which allows us to play
and work with the arrays.
And this mPy here is just a shortcut.
More convenience when we want to use NumPy.
All right, then second library is random.
So this is just because we will
be taking some random samples from the different batches
when implementing experience replay.
So we have to import this random library as well.
Then we will import OS,
that will be be just useful when we want to load the model
because you know, once the model is ready,
we will implement some code to save the model
and then another code to load the model.
That's when we want to, you know, save the brain
and load the brain whenever you want
to shut down your computer
and reuse the brain that was trained before
for some new experiment.
So that's important.
Then we are going to import the Torch library, essential.
That's because we will be implementing our neural network
with PyTorch, which I recommend much more
than the other ones for artificial intelligence
because it can handle diamond graphs.
So there we go with Torch.
Then from Torch, we are going to import Torch.nn,
The nn module is the most essential one.
That's the module that contains all the tools
to implement some neural networks.
And of course there will be a deep neural network
that will take as inputs the three signals
of the three sensors, plus orientation
and minus orientation,
and will return as output, the action to play.
Well actually, it'll return the Q values
of the different actions.
And using a soft max we will return the action to play,
only one the most relevant one to accomplish the car score.
So Torch.nn most essential one,
then we are gonna give a shortcut to the functional package.
From here we go, the functional package from the nn module.
So this functional package
contains the different functions that we use
when implementing a neural network.
So typically the last function,
we will be using the uber loss
because that improves convergence.
And the uber loss is contained
in this functional sub module from the nn module.
And since all this is pretty long,
we're gonna give it a shortcut
and we are gonna call it F, simply.
Then only three modules to import left.
So the next one is another essential one, which is Optim.
And we take it from still the Torch library
and then Optim, there we go.
And let's just call it Optim instead of Torch.optim.
So that's of course for the optimizer.
We will be importing some optimizers
to perform the cast degrade in descent.
So we will definitely need it.
And then we need to import autograd,
and that's only to take the variable class from autograd.
So the purpose of it is a little bit technical.
Basically, we need to import the variable class
to make some conversion from tensors,
which are like more advanced arrays
to a variable that contains a gradient.
So it's like we don't want to have only a tensor by itself.
We want to put the tensor into a variable
that will also contain a gradient.
And to do this,
we need to use the variable class to convert this tensor
into a variable containing the tensor and the gradient.
So that's a little bit technical
but that's what we have to do when working with PyTorch.
And we do this thanks to the variable class
but before getting the variable class
we need to import Torch.autograd
and let's give a shortcut as well, autograd.
And then from Torch.autograd, we import variable.
There we go.
And now we have all the libraries
that we'll be using to implement our AI.
So we won't bother importing any other library.
We have all the tools we need
and now we are ready to create the architecture
of the neural network.
So that's exactly what we'll do in the next tutorial.
And until then, enjoy AI.