-: Hello and welcome again to our self-driving car module.
So in this tutorial, I'm going to explain
the environment on which we will implement
our artificial intelligence and that will contain,
of course, the car that we will train to drive itself,
and to avoid obstacles, and on which we will draw some roads
and some blocks for our car to navigate around them.
So we will later build this artificial intelligence to
train this car, to drive on the road, you know,
without crossing the limits,
and avoiding some obstacles that
we will put inside the road.
So this is a pretty exciting challenge
and actually there are two separate files
as you can see.
There is the AI python file that our artificial
intelligence that will do all the training to
train the car how to self drive,
and we have the map.py file
that is the code that makes all this environment.
So here is that code,
that's actually 200 lines of code, a little more.
So this code is not typically related to AI,
it is just a code to make the environment, to make the map.
So I'm gonna go through each
of the sections one by one to explain,
but we're not going to implement this code
line by line from scratch
because we want to focus on artificial intelligence.
But let's still go through the sections
one by one to understand what's happening.
So first, we import the essential libraries.
That's for any code.
We need some libraries to
perform some tasks more efficiently.
Then we import all the Kivy packages.
So, that's not very important
because this is all specific to Kivy.
We're using Kivy to make the map,
and so we're importing a lot of classes
and objects to be able to make this map
and add some tools in the map.
All right, then this line is important.
This line is AI related because basically,
this is where we import our brain,
the brain of the car
which will be an object of this DQN class.
And the DQN class is our artificial intelligence itself.
You will see we will implement the DQN class
in the following tutorials.
And as you might have guessed,
DQN stands for Deep Q Networks.
So we will implement a Deep Q Learning Network,
and then once it's ready
we will be importing it here with this line from the AI.
And the AI is of course our AI Python file.
All right, so can't wait to implement this.
This is gonna be quite a journey,
but you will see this is gonna be very exciting,
because thanks to the AI
the car will be able to drive itself.
All right, and now before I move on to the next sections
we have to explain how we will train this car.
I'm not going to explain the neural network right now,
but I'm going to explain the idea
of how we are gonna train the car to drive itself
and to avoid obstacles.
So you know, in real life,
if you want to train a real car to avoid some walls,
or some obstacles, well, what would you do?
You would definitely not take real walls
or real big obstacles and smash your car onto them.
That would cost you a lot of money.
Instead,
a more intelligent idea would be to punish your car,
not when it smashes a wall or an obstacle,
but when it goes onto some sand.
So it's like you have a field,
this field has some roads on which the car has to stay,
and the roads are limited by some sand.
And each time the car goes onto the sand,
it's like it's going onto an obstacle.
Because once the car goes into some sand,
it'll be slowed down
and we will make sure that the car is penalized.
It's punished for that,
and that is one essential point
of our artificial intelligence.
The bad reward comes whenever the car goes
onto some sand and is slowed down.
All right, and therefore here,
I'm introducing last x and last y,
which are the coordinates of the last point
in memory when we draw some sand on the map.
All right,
and then we get our artificial intelligence
which we call brain, and that contains our neural network,
and we will call it brain because this is actually the brain
of the car and that contains our renewal network.
All right, so in this line of code,
as you can see I'm creating an object of the DQN class.
I will remind what classes and objects are,
but brain is the object, DQN is the class,
and 5, 3 and 0.9 are the inputs of the class.
So that's very simple.
Five corresponds to the states that
are encoded vectors of five dimensions.
We will see what they are,
perfectly describing what's happening
in the environment on the map.
Then three is the number of actions.
There will be three possible actions.
Go left, go straight or go right.
And 0.9 is the Gamma parameter
in the Deep Q learning algorithm.
All right, and then we have the action to rotation.
So action to rotation is a vector of three elements,
0, 20 and -20.
And so we have to do this because the actions are encoded
by three numbers, 0, 1, and 2,
and that corresponds to the indexes
of this action to rotation vector.
So for example,
if the action that is selected at time T is 0,
well 0 corresponds to the index
of this action to rotation vector,
and the value of index 0 is 0,
and therefore we will go straight.
Then if the action selected is 1,
well 1 corresponds to the index
of this action to rotation vector.
And the value of this vector that has index 1, is 20.
So 20 corresponds to a rotation of 20 degrees,
and that means the car will go 20 degrees to the right.
And then if the action selected is 2,
well 2 corresponds to the index 2
of this action to rotation vector.
And therefore the car will do a rotation of -20 degrees,
and therefore, it'll go to the left.
All right, then we introduce the last reward variable,
because at each state we'll be getting the last reward.
So remember,
if the car doesn't go onto some sand
then the reward will be positive.
And if the car goes onto some sand,
well it will get a bad reward.
And at each time,
this variable will contain this reward
that it gets at each time T.
And then we initialize the scores,
which is a vector that will contain the rewards,
not all of them,
but the rewards onto a sliding window, so that you know,
we can make a curve of the mean score
of the rewards with respect to time.
All right, then in this code section, we initialize the map.
So we initialize, for example, the send variable.
So that's important.
The send variable is actually going to be an array
in which the cells will be the pixels of the map.
And in each cell,
we will have a 1 if there is some sand,
and a 0 if there is no sand.
At the beginning, we will not be drawing anything,
so there will be no sand at all.
And therefore all the cells
of the sand array will have a 0,
so there will be 0s everywhere.
And as soon as we draw some sand,
well the cells on which we draw the sand, will get a 1.
And we initialize the arrays with all the 0s right here.
Sand equals NP 0s.
Then we have this important thing, which is the goal.
So the goal is a point in the map
which we will train the car to reach,
so it's like destination.
So what is this goal going to be?
Well, this is going to be the upper left corner of the map.
So we'll train the car to go
to the upper left corner of the map,
and then once it reaches the upper left corner of the map,
then we will train it to go
to the bottom right corner of the map.
So we can imagine the following scenario.
The upper left corner of the map is the Airport of a city,
and the bottom right corner
of the map is the downtown of the city.
And we will train a taxi or Uber to do some round trips
between the Airport and the downtown.
And of course,
we'll make the task difficult to this taxi
by drawing some more and more difficult roads,
and adding more and more obstacles on the street to see
if the taxi can still manage
to go from the Airport to downtown.
So this is gonna be fun.
And so that's why here
I'm setting the coordinates of the first goal,
that is the Airport,
which is at the upper left of the screen.
So the map will be like a square, like this.
And the coordinates of the origin that is
the coordinate 0, 0 is right here,
and then larger is this distance here.
So the coordinates 20 and larger -20 will therefore
be right here,
the upper left corner of the map.
And why did I choose 20 and not 0?
Well,
that's because we want to train the car not to
rush into the walls, you know,
we want to train it to avoid the walls as well.
And therefore, the goal must not be 0 because we don't
want the car to touch the wall.
We want to touch the goal,
so we have to put it right here.
And then I'm just introducing the last
distance variable,
which just gives the current distance
from the car to the goal and that I'm initializing to 0.
All right, and now, time to make the car and the game.
So we're gonna make two classes,
one class for the car and one class for the game.
And inside these classes,
we will already make some connections with our AI.
So we'll do that in the next tutorial.
And until then, enjoy AI.