Instructor: Hello and welcome to the level three
of our self-driving car AI game.
So now the challenge is going to be to still do
the round trips between the airports and the downtown
but this time by avoiding obstacles.
And we will draw the obstacles ourselves.
You can draw any difficulty of obstacles.
I'll try not to be too tough with the car
because I want to make it to level four,
but let's keep this challenging.
All right, so as usual,
we're going to select all this code and execute.
And there we go with level three.
So let's load our brain.
Here we go.
Brain is loaded.
The car is doing the round trips properly.
And so now let's draw some obstacles.
All right, so what can we do now?
First, let's draw something like this.
Okay.
And then what we can do is draw another obstacles like this
and then maybe something like this,
something like this, and something like this.
Let's see what it does.
(exciting music)
All right, so now what it's gonna do.
Okay, avoiding this obstacle.
Now, avoiding this one.
No mistake again.
So still it has to learn.
It's still exploring, avoiding this obstacle.
Perfect. Very good.
Is it going to avoid this one?
Great. It did.
And going to the downtown and now going back to the airport.
Avoiding this obstacle.
Great.
Now, is it going to avoid this one this time?
And no, still mistake.
It's okay.
I'll get this thicker.
Maybe something like this will be better.
The car will understand more
because right now it is being quite stubborn.
Alright, still avoiding this obstacle.
It seems not to have any problem with this one
but kind of a problem with that one.
Better now.
You see I got it thicker,
which means that the punishment was harder
with a worse reward and now it managed to avoid it.
And this one avoided as well.
Perfect.
So now we have a functional self-driving car as it seems.
So avoiding this one again, great.
And now avoiding this one.
Great.
Still going onto some sand, but it's okay.
It will be punished for that anyway.
Avoiding this one.
So sorry, this one was totally unuseful
but you know we can do something like this if you want,
to get this even more challenging for the car.
All right.
And okay, we can add a big tip here so that it doesn't cheat
because it's still crossing the tip of the obstacle.
And we can do the same for the other tips
of the other obstacles.
There we go.
Okay, better now.
Let's see what it does now.
Okay, there we go again, trying to reach the airport now.
Okay, perfect.
So it's going around the obstacle
it's not trying to find the best path,
but anyway, the goal is to reach the two goals:
the downtown and the airports.
Then we can add some code to try to find the best path.
But we definitely already have a self-driving car.
All right, great.
Great job.
And there we go again.
It's still going on to some sand
so it's still being punished.
We can actually look at the score function, which is here.
As you can see, it gets the punishment
and the score function is actually decreasing with time.
So that's because I got the obstacles thicker.
And right now, as you can see,
it's trying to avoid the tips of the obstacles
more and more.
It's doing a better and better job as you can see now.
It's going around faster now.
Except for this one.
But if we save again and look at the score function,
as you can see we see some kind of improvement here.
But anyway, the car is gonna get better with time.
But I think that right now we can say
that level three is passed.
Because anyway, it manages to do the round trips
between the airports and the downtown
by going around the obstacles and that's what we wanted.
So maybe let's move on to level four.
Level four is gonna be very challenging.
I don't think we'll be able to pass level four
but that will be the challenge of this module.
You will have to change something in the code
to pass level four.
It will either be something
that it has to do with the reward
or something that has to do with the strategy of the game
or even something with the neural network
or something with the DQN algorithm.
Well, you will look for it, you will do some research,
and you will try to get an even better car than this one
that avoids any kind of obstacle
that will never go onto some sand
or that will try to find the best way,
the best path, the shortest path.
So good luck with that.
It's gonna be excellent practice for you
and I can't wait to see your solutions.
So now let's move on to level four.
And this time it's gonna be very, very challenging.
So I'll see you in the next tutorial.
And until then, enjoy AI.
(exciting music)