-: Hello my friends and welcome
to this new section on the visualization
of the results of the deep convolutional tutoring model.
First, I really wanna say huge congratulations
for having completed this whole implementation
of the deep convolutional tutoring model.
And now here we are at the front part where
we're gonna visualize the results of the AI we created.
And as I said at the end of the previous tutorial, you know
the last tutorial of the implementation,
I said that I had a surprise for you.
And well, the surprise is exactly this.
We're gonna visualize the results on Google CoLab.
And why is that a good surprise?
That's a good surprise because basically you
will have absolutely no trouble running the code, you know
running the code and visualizing the results.
Indeed, this is a new feature of the course.
You know, before this, we actually used to run the code
on Spider and Python, but it was extremely difficult to
you know, install all the packages, you know, Python torch
the gym dependencies and all the different dependencies.
A lot of students actually had tons
of issues installing everything and at the end
most of them couldn't run while the whole code
and mostly they couldn't see the final results.
So I absolutely wanted to fix this
and really the best solution that I've come
up with was to include the whole code.
So you will have exactly the same code
as what we've implemented in this part,
but I've included the whole code
on a Google CoLab notebook where the only thing
you will have to do to run the code and visualize
the results will be to click these play buttons here.
That's the only thing you have to do
because the beauty of Google CoLab is that most
of the libraries and packages are already installed.
For example, TensorFlow is already installed,
Pie Torch is already installed,
and all the torch modules functions
and classes are already installed
so you don't have to install them.
However, we still have some little dependencies to
install to make sure everything is connected well,
but we won't have to install them through a terminal
or you know, through the Anaconda prompt like we did
in the self-driving car.
We will just have to click this play button here
in this first cell.
Installing system dependencies for VizDoom.
Now I know what some of you might be thinking.
Why didn't we do the same for the self-driving car?
Well, unfortunately we couldn't do the same because
in the self-driving car module we had the user interactions
you know, when we plot some sense on the map to, you know
design the roads and put some obstacles.
Well, unfortunately Google CoLab doesn't have that feature
yet of allowing to have some user interactions
with some map.
So I couldn't do it for the self-driving car.
My apologies for that, but that's why I gave you
the whole chapter of my book on the AI crash course,
which contains all the instructions on the self-driving car.
I hope you could install everything properly.
Anyway, there was not much to install compared to Doom
so I hope everything went well for the self-driving car,
but for Doom here,
now you're covered, you won't have any trouble
running the code and visualizing the results.
So that's the surprise.
I hope you like it.
And now I'm gonna walk you
through this whole implementation to show you
that indeed we have exactly the same code
as what we implemented in this part two.
The only thing is that I merged the three codes.
You know, this is the code we had in the folder, you know
composed of the image, progressing that PY file
then the experience replay that PY file,
and then the AI that PY file with the two parts,
part one building the AI and part two, training the AI.
Well, what I simply did is I merged the three
of them by putting first the image progressing that PY file
I'll show you that in the workbook.
Then second experience, replay that PY file and third
the AI dot PY file.
There are just one below the other in that order.
Okay, so I'll show you this to you.
And then I just had to change two or three things
in this whole code and I'll show you exactly the changes
so that you can make sure to catch the difference.
Basically in that file everything is the same,
in that file everything is the same as well,
absolutely no change.
However, in that file, well here when we import the packages
for open AI and Doom, while we don't use the P package
in Doom rappers action space to import to this Discreet
we don't use that anymore.
We'll use something else you'll see,
and also I changed the dimensions
of the frames here, you know, 80 by 80.
I changed that to 256 by 256
so that you can have better visualization results indeed
because you will have bigger videos to watch.
Okay, that's just what I changed.
I'll show you now, let's start
with image processing because indeed, as I told you
I just merged the three files together.
So we start with indeed image processing.py, and
as you can see, this is exactly the same.
We start by importing the libraries
the exact same ones here.
Then we preces images with the pre-process image class
composed of the init method and the observation method.
And as you can see, this is exactly the same
pre-process image class init method and observation method.
All right?
Then we move on to experience replay where we have exactly
the same indeed we first import the two libraries here known
by named Toplin, right?
Then we define one step exactly the same
as what we have here, and then we make the AI progress
on several end steps with the end step progress class,
which is exactly the same as in the workbook, right?
I just did some copy past here.
We had exactly the same the init method, etu method,
and the reward steps method.
All right?
Then of course we implement experience replay
with the replay memory class, and as you can see
we have exactly the same in the Google CoLab.
Plus you will maybe better appreciate the structure
because I added some text sales so that you know
it can be even nicer to visualize on this workbook.
All right, so replay memory class
init method, sample, batch run steps,
basically exactly the same code. All right?
And then we move on to the third foul, AI for Doom
AI PY containing the two parts.
Part one, building AI and part two, training the AI.
So let me show you
that we have exactly the same on this workbook.
So AI for dom, AI.py, first
we import the libraries, the exact same ones, right?
As you can see,
then here that's the difference I already highlighted when
importing the packages for open AI and Doom.
Well, instead of importing gym wrappers
and then skip wrapper
and P packet to import to discrete, well
what we do is we only have to actually import gym.
That's the same.
Then import (indistinct) gym,
which are the new Doom environments basically
which were really well improved.
And finally, the rappers by the gym library.
All right? So just this difference here,
which will make this thing work on Google CoLab
and also on Python if you want to run it.
And then here, of course, I didn't have to include
that code within Google CoLab simply
because here we import the other files here, you know
because they're separated.
And since I merged the three files on Google CoLab, well
I don't need to do this import.
So here we move on directly to part one, building the AI.
And here, as I told you, we have the exact same code
except for the dimensions here, right?
If we look at part one here, part one, building AI
we start by making the brain with the CNN class.
And as you can see, we have the exact same code
except for the dimensions of the input frames here
which were when we implemented the code 80 by 80,
but I changed that in Google CoLab to 256 by 256
so that you can have better frames
and therefore better videos to watch, okay?
But all the rest is exactly the same.
So here we make the brain with the CNN class
then we make the body with the soft max body class
so exactly the same as what we have here.
And finally we put the whole thing together
with the AI class containing the brain and the body
and also some methods to perform the actions, right?
Exactly the same AI class composed of the brain
and the body, and then the same method to
perform the actions. All right?
So exactly the same, just the dimensions
of the input frames changed.
And now let's move on to part two, which will be also
exactly the same.
First we get the Doom environment, right?
Let me show you part two here as well.
We get first the Doom environment, and here, remember that
since we no longer use P packet to import the environment
but instead VizDoom, well, the only thing that changed
in that code is indeed this here and indeed we
get VizDoom corridor,
but this is exactly the same environment.
Alright? And then of course I change these new dimensions so
that you can have bigger videos here.
So we no longer have 80 by 80, but this time 256 by 256
and then we have executive same, we get the Doom of
and the number of actions, which we will see will be seven.
Then we build the AI by creating the brain object, you know,
CNN then the body object, soft mac body
and then the whole AI composed
of the brain and the body exactly the same
as what we implemented together.
All right? Then we set up experience replace.
So that's exactly the same here.
And then we implement eligibility trays.
Still executive same. And then what do we do?
Well, we make the moving average
on 100 steps exactly the same here.
So that's the exact same code.
Basically only the dimensions of frames
and the way to import the environment changed.
Okay? And finally, we train the AI
with exactly the same code.
Okay? So until here we have the whole code that you saw
in the folder when you downloaded the dataset
and code again.
Basically we also have the whole code
that we implemented together.
However, then to make the visualization work
on Google CoLab, well, I added this extra code, but
since it is really only specific to this CoLab notebook
and also only specific to visualize the AI
in action in the Doom environment, well please
you don't have to worry
about understanding all the codes that is here
because you know you will only see that once
in your life when you want to build
and train some AI to play in the Doom environment.
What I mean is that you won't have to use this code
for the other applications you'll be working
on when building and training in A.I..
So don't worry too much about this code
I just added that to make it work on Google CoLab.
But let's see what I do.
I first import libraries, of course
then I added some practical codes here
so that you can very quickly see the number
of actions you have in the environment, as well as a display
of the frame of the environment, just to see how it is like.
And the reason I added that is because I'm also
gonna show you how to play with other Doom environments
because you actually have many of them.
And so what you will simply have to do is
replace the VizDoom corridor
by the name of the other environment, which are here.
I'm gonna give you the link
of that page right after to this tutorial.
But here, as you can see, you have many Doom environments
that you can play with.
You know, VizDoom basics, VizDoom quarter, which is the one
we're gonna play with right now, VizDoom Defense Center
and others. You even have some indoor mazes.
And just follow the instructions here.
You will find them all in this folder.
You know, in this in it method, you can just
find the name here, VizDoom Basics, VizDoom Corridor
VizDoom Defense Center, and you have many, many
of them you see.
So if you wanna play with other Doom environments
feel free to do that.
You will just have to change the name of the environment.
Anytime you see VizDoom Corridor here on this workbook
you just replace that by the name.
And thanks to this code and this one, while you
will be quickly able to see what is the number of actions
the dimensions of the input shape, and also a display
of the frame to see how the environment looks like.
So this will be pretty practical.
Then I just made this helper function for the
visualization. Then I run the AI on
one episode, of course, after it is trained.
So we will first train it here, as you can see, by the way
I forgot to tell you
that I also changed the number of epochs from 100 to 20.
And that's simply because, you know
I increased the dimensions of the frame.
And since I increased dimensions of the frame
well the training will take much longer
because of course it's harder to train.
And therefore I reduced the number
of 100 epochs down to 20, okay?
And don't worry with that
we will already get well positive rewards.
But of course, if you want to improve the performance
and improve the results in the end
feel free to increase that number of epochs.
You will get even better results.
But you'll see that already with 20 epochs
the training will take quite some time.
Okay? All right, So where was I?
Yes, that helper function for the visualization.
Then I run the AI on one episode, of course
after it is trained.
And finally, this last cell of code will output the video
of the game play, you know, over this one episode.
And you will see how the AI will perform
over this one episode on a video which will be output
and which will be able to download in files here, you know
when we execute this code.
Okay? All right, so now time to the execution.
Time to really sit comfortably in our chair.
I hope you have a good coffee or a good tea.
Now we're just gonna press the different play buttons
in each of the cells to run the final results.
So let's do this.
But first
and that's very important to understand in Google Colab.
So this is the link of a Google CoLab workbook
which you all have access to, and therefore, of course
you can't modify anything in it, because if one
of you modify as well, the others will see the modification.
So this is in read only mode, and therefore
what you have to do
in order to make some modifications, you know
in case you want to change the name of the environment
or increase a number of epochs or even tune your A.I., well
you will have to create a copy of this workbook
which will be in your own Google Drive.
And that's why in order to do that
you will need a Gmail address
in order for you to have a Google Drive, okay?
So make sure to have one.
And once that's the case
well you just need to click file here
and then save a copy and drive, and this will create a copy
of this workbook in your own Google Drive
on which we'll be able to do some modifications.
All right, And now here we are
at the beginning of this execution.
So now we're just gonna play all the cells
and we're gonna get closer and closer to the final results.