-: Hello and welcome again to module two, Doom.
We will now get us ready
to start the implementation of our AI.
And as usual, the first thing that we need
to do is set the right folder
as working directory.
So, let's do this now so
that we can move onto what's more interesting.
So, as usual, I start on my desktop,
then I go to my artificial intelligence exit folder.
Then module two, now Doom.
And there we go, that's the folder we have to set
as working directory.
So, let's do this now.
We click on this tool button here,
then restart kernel, and then, yes.
And here we go, we now have
the right folder as working directory.
So, now as you can see, we have four files,
well actually, three files in one folder
in this working directory folder.
So, let's start with the first one.
The first one is ai.py.
That's of course, the file that will contain
our artificial intelligence,
and that's nothing else than this file here.
That is the ai.py file in which,
we will implement everything that is related
to building an AI, and especially building an AI
with the deep convolutional key learning model.
So, basically, that's where we'll have
the big adventure.
Then we have some other files.
So, we have the second file here,
which is experience replay.py.
And so, this time,
I put experience replay separately,
just because we already implemented it
and now, we want to focus on what's new.
And trust me, we have a lot of new things
to do with this new artificial intelligence
because not only we want to build an AI,
but we want to build an AI to beat Doom.
So, you can imagine that
this will require some quite advanced code.
So, no worries, we have a big code waiting
for us and you will learn a lot of new tricks.
That's why this experience replay trick
that you already know and that I remind,
improves a lot the training.
Well, let's put it separately
in this experience replay.py file
so that we can now focus on all
the new concepts, techniques, and tricks
that are waiting for us.
All right, and then, we have
the image pre-processing.py file.
So, that's another Python file,
which will take care of pre-processing our images
because you know this time our AI will have eyes.
And that's because the input states
are no longer encoded by a vector, but this time,
the input states are the images.
So, the first layer of the big neural network
that we will make will be the eyes,
and that will be the convolutional layers
of the convolutional neural network.
But to make sure these images can be accepted
as inputs of the convolutional neural network,
well, we need to pre-process them.
And so, this file will take care
of pre-processing these images so
that they can go into the neural network.
And so, I separated this file
because this is not directly related to AI,
and again, we want to keep the maximum
of our brain, and our memory, and our focus
on everything that is related to AI.
So, we are putting this separately
so that we can pre-process the images
in a flashlight, and save some energy
for the rest.
You can have a look at it if you want,
and if you also have the deep learning course,
where you can have a look
at the practical tutorials;
we talk about image pre-processing,
but here again, we really wanna focus on AI.
Trust me, we have a lot to do.
And eventually, the last folder,
well that's the videos folder.
So, right now, this folder is empty
as you can see, but when we execute the code,
some videos of the AI playing Doom
will be added to this folder.
So, that will be very exciting
because we will see on some videos
how well the AI is doing.
So, we will literally see the AI killing
the monsters and trying to run towards the goal.
So, you're gonna see this will be pretty exciting.
So, of course, the first videos will be very bad
because the AI will not be trained much yet,
and so it will get killed very fast.
But then, you will see that as the training
is progressing, while the AI will get better
and better, and eventually, it will manage
to kill some monsters, not getting killed.
And hopefully, we will be able to make
it reach the goal.
All right, so let's go back
to our AI file, which is this one.
And as you can see, I already took care
of importing all the essential libraries
and packages that we need to play Doom.
So, let's quickly have a look at them one by one.
We have of course NumPy
because we'll be working
with arrays, that's inevitable.
Then we have Torch, of course,
because we're implementing the AI with PyTorch.
Then we have the torch.nn module,
which contains all the tools to implement
a neural network.
So, for example, the nn module
will contain the convolutional layers
that will be part of our future neural network.
Then we have the nn.functional package,
which has the shortcuts F
and that contains all the functions
that are used in a neural network.
So, typically the activation functions.
We will be using
some rectify activation functions, but also,
some max pulling function
for the convolutional neural network.
And all these functions
are contained in functional.
Then we have Optim, which is of course,
for our optimizer.
I think, we'll be using an atom optimizer
and this optimizer is contained in Optim.
And then, the best of the best of PyTorch,
the variable class from the autograph module.
And that's all the power of PyTorch
because that's what contains the dynamic graphs,
allowing to perform very fast computations
of the gradient, even the gradient
of composition functions.
So, we will definitely be using it,
as for the self-driving car,
but trust me, for Doom we will be needing
it very bad.
Okay, so that's all for the essential libraries.
Then we need to import some packages related
to open AI Gym and Doom.
So, of course, we import Gym,
then we import some wrappers module
of the Gym library.
And one of these wrappers is kip wrapper.
So, that's basically to import all the tools
and environments of Gym.
And finally, we have this package
that we need to import, and which
is directly related to Doom.
And that's the action space and two discretes
of the Doom wrapper that basically contains
the environment of Doom, and more specifically,
the actions that can be played,
the number of actions
for the specific Doom game we're gonna play.
And I remind that there are six actions.
Move left, move right, turn left,
turn right, move forward, and shoot, attack.
All right, so that's basically
what you need to import for Doom.
And then finally, we of course, need
to import our two internal files,
experience replay.py for experience replay,
and image pre-processing to pre-process
the images that are nothing else
than the images of the screen
when playing the game.
And these images will be pre-processed,
converted into NumPy arrays, reshaped
to a certain format, and then they will go
to the neural network,
the convolutional neural network.
All right, so I guess, now that we are ready
to start the big implementation of this AI,
and now, it's very important for me
to tell you that, that leads me
to the very important point of this module.
It's that since you know I told you we have
a big implementation waiting for us,
well in order to not get lost in all this,
we need a good structure.
And so, I already highlighted this structure.
We will be implementing this in two parts.
The first part will be about building the AI.
So, that's where we will make the brain
of the AI.
And the brain, as you understood,
is nothing else than the neural network.
You know this big CNN composed
of some convolutional layers
and then, some fully connected layers
to predict the outputs
that are still the Q values.
And then, we'll make the body of the AI.
And that's a new representation
I'm bringing to you, and that's again,
not to get lost.
You will see that the further
we progress with the code,
the more you will see the structure
and everything will make sense in the end.
And to make sure that makes sense,
we need a representation of the AI,
and basically, this first part, building the AI
will be composed of three sections.
First section will be about making the brain
that is the neural network.
The second section will be about making
the body, and I'm calling it the body
because this is the part
that will tell the AI how to play the action.
So, you know first you have
the brain that detects the images
and predicts the Q values,
but then you need to specify how the AI
should play the action.
And that, it does it with its body,
like a human body would do.
So, the body will be the part
where we will specify the method
of playing the action.
So, for example, with our self-driving car,
the brain was the neural network we made
and the body was how the action was played.
That is with the submax method.
In here, that's the same.
We're gonna make a brain,
and we're gonna make a buddy,
which will play the action.
I will let you find out,
but the key point in all this
is that we will have a very structured code
so that not only you can take a step back,
and really understand what's going on.
But also, you will be able to use it
as a framework whenever you want to build
an AI for other purposes.
All right, and after building the AI in part one,
we will move onto part two,
which will be about implementing
the deep convolutional key learning model.
And there again, we'll have different sections,
and one of them will be, of course,
to train the AI.
So, I can't wait to dive into it now.
We're gonna start with part one, obviously,
and we're gonna start by making
the brain of our AI.
So, I can't wait, and until then, enjoy AI.