-: Hello, and welcome to the fun tutorial
of this first module self-driving car.
It's gonna be epic.
We're gonna test our AI on the environment
and we're gonna test it on four different levels.
That is, we're gonna play a game.
The game will have four levels of difficulty,
and the AI will have to pass these four levels.
So what are gonna be these four levels?
First level one, the first level is going to be
to reach the airport and then do some round trips
between the airport and the downtown.
So as soon as we see the car do these round trips
well we pass level one.
Then level two.
Level two will be to still do these round trips
but on a specific road that we draw ourselves.
But it's gonna be an easy road because it's level two.
And of course the car will have to self drive
by staying on that road.
So it'll be a road that goes
from the airport to downtown and then the other way.
And so the car will have to do these round trips
by staying on that road.
If it does, we will pass level two.
Then level three, level three will be to draw some obstacles
on the map to see if the car manages to avoid the obstacles
and still reaching its goal.
So no worries, we'll draw some difficult obstacles
that the car will have to avoid, and we'll see
if it managed to reach the airport and the downtown.
And finally, level four,
the most challenging level for the car will be
to draw a very difficult road to reach to downtown.
So I don't know, you know, it'll be a road like some zigzag.
I'm not a brilliant architect
but I'll try to make a challenging road.
So let's hope we pass at least the first level.
That would be great.
Then let's hope we can also pass level two and three.
And if we pass level four, that would be wonderful.
So let's do this.
Let's take the challenge.
Well, actually the self-driving car
is gonna take the challenge
but we are the brain behind this
so let's still hope that works.
All right, so the first thing I'm gonna do is just
to give you a quick reminder about the map.
So that's the map.
And first we're gonna look at the map.
We're gonna look at the self-driving car without the AI.
So it'll just be a car having those random actions
that you saw at the beginning of this module.
So how can we look at that?
Well, we have to deactivate the AI.
And to deactivate the AI
we simply need to put a temperature equal to zero.
Remember that parameter here is the temperature
and right now it is equal to seven.
So that's a low temperature.
We will increase that afterwards.
But if we don't want the car to have a brain,
that is if we don't want to activate the AI, we simply
need to set the temperature to zero, to equal zero,
and same here, of course, that's the real temperature
in the code.
So there we go, and then we must not forget to save.
Because otherwise that won't include the change.
Okay, so now we don't have any AI.
The AI is deactivated.
So let's have a look at the map just to give us
a quick refresher, a quick reminder about what it looks
like, so I'm going to select everything and press enter.
All right, and there's our map and there's our car.
So as you can see, the car is having totally random actions
you know, to go left, to go straight or to go right,
and therefore it is not reaching the airport, which
is I remind at the upper left of the map and not reaching,
well, it just did, but that's totally random.
You see, right now it is at the airport
and it is not reaching the other goal
which is downtown at the bottom right of the map.
So we were just like here, but we can clearly see now
that the actions are totally random.
It is going nowhere
and there is definitely no artificial intelligence.
But no worries, we will activate it right now.,
I'm going to close the map,
and then I'm going to restart the kernel.
Restart the kernel.
You click on this tool button here, and then yes.
And now time for the show.
We're finally going to put this brain we made
in the car and activate the AI.
I'm super excited to see what's gonna happen.
We are gonna activate the AI right now,
and to do this we need to raise the temperature.
So to change the temperature, we just need to replace
that zero by, well, let's start with seven as we had before.
So let's specify seven here.
All right, let's not forget to save.
And now let's get back to our map.
And now we can just re-execute this again
because we restarted the kernel.
So let's execute.
And there we go, we have the car.
And what is it doing?
Well, it is trying to find its way, it's exploring,
it's understanding what it has to do
and it's about to reach the airport.
And there we go, first goal reached, wonderful.
And now the next goal is to reach downtown.
And there it did just reach downtown, and now it's trying
to find the airport back, going to the airport.
And there it did again.
Wonderful, so that works.
It didn't take time actually
to explore, learn from the mistake,
you know, the mistake here is to get further from the goal.
That's where we punish the car
by giving it a slightly negative reward, you know
it's minus 0.2.
So it learned from that mistake.
And by learning from that mistake, it managed
to get the positive rewards by getting closer to the goal.
And now it finally understood what it has to do.
It's definitely reaching the airport and
then reaching the downtown and then doing these round trips.
So that's perfect.
We have a self-driving car, but I can help
but notice it is looking like an insect.
The car doesn't really seem sure of itself.
You know, it doesn't have a very confident movement.
It's like doing left and right.
That's not looking like a car movement.
It looks more like a bug.
So we're gonna fix that.
And as you might have guessed, the way to fix that is
increase the temperature, because remember, the temperature
is the parameter and the soft mac function
that we can increase so
that the action is returned with more certainty.
So that makes sense that if we increase the temperature,
well we might end up getting a car more sure of itself
because the AI will be more sure
of which action it should play.
And that remember is because
the action will be played with a higher probability.
The only problem with this, increasing the temperature
is that remember the AI is less exploring the other actions
because by increasing the temperature,
the other actions will have low probabilities.
But right now, that doesn't seem to be a problem
because the car seems to have no problem reaching its goals
the airport and the downtown.
So we can totally increase the temperature if we want
this thing that so far looks like an insect look like a car.
So let's do this, I'm going to close this now.
There we go, restart the kernel again and press yes.
And now we're going to increase the temperature.
So let's do this, I'm going back to my AI file
then replace these t equals seven by 100.
There we go, then we save,
and now we have a self-driving car, sure of itself.
So we might get better results
and we might get something that looks more like a car.
So let's click on map and then let's re execute that again.
All right, what happened?
Okay, it did some kind of a burnout not sure why, but anyway
now we have something that looks more like a car.
You can see that it is going more straight.
It is not doing these quick left and right movements.
That's because now the car is more sure
of which direction to take at each time.
You know, it wants to take the best direction,
going to the airport and then to downtown.
So clearly we can now say that we passed level one.
The car is doing these round trips
between the airports and the downtown.
So we're gonna save that.
That's, I'm gonna show you how to save the brain.
We just need to click on this save button.
And if we look at what happens here,
well we have the curve of the reward.
At the beginning, we can observe some mistakes that it made.
So that's where the reward is negative.
But then it learned from its mistakes
and the reward increased little by little
until reaching a constant positive reward equals to 0.1.
But that's the maximum reward we set.
And that's because it ended up exploring.
That's the exploration phase
and then it just knew what it had to do
and that's where it was doing these round trips
between the airports and the downtown without any mistake.
So there we go, we passed level one, congratulations,
now let's get things more challenging.
Let's take things at the next level.
Let's try to pass level two
which I remind will be to do these round trips
on a specific road we're gonna draw ourselves.
So let's check that out in the next tutorial.
And until then, enjoy AI.