-: Hi everyone and welcome back.
In the last lecture
we ended up starting to build our environment.
We have essentially an 11x11 grid here
that we want our Postman to iterate through
to solve our challenge.
We want to now start to think about,
once we have the grid,
we also need to have actions
for our Agent.
And I'm gonna delete these really quickly;
we don't need these cells.
I just wanted to have it in for some spacing.
Just makes it easier to view.
Gimme one second.
Let me delete these and we can leave this one here.
So, I left the text in that we have
and we want to set our actions as an
up, right, down and left.
With Python, it's pretty straight forward.
We can build the list and set our actions.
Our actions, are going to be equal to,
as we have, up, right, down, left.
So let's set it up,
right,
down,
and left.
We need to give our act, our Agent, excuse me,
some ability to maneuver with these actions
through the maze.
In addition to that, we also have to start setting rewards.
Now, this is where it's going to
start becoming a little more tricky
because we have to set different states of our environment,
different states in the sense of
we want to be able to assign these -100 and -1,
these steps or these state values to each square
within the grid.
So we see, to help our AI Agent learn
each state or location in our city,
we want to have a reward value.
That's how our Agent's going to learn.
So the Agent may begin at any white square
but its goal is always the same.
It wants to maximize its total rewards within Q-learning.
Negative rewards,
we know that they're referred to as punishments.
These are used for all states except the goal.
That's how we're gonna establish that optimal policy.
Which encourages the AI to identify
the shortest path to the goal by minimizing its punishments.
All right, also to maximize the communal rewards,
the AI Agent will need to find the shortest path
between the item packaging area, our green square remember,
and the other locations of the city
where the Postmen can travel. The white squares.
Agent's going to learn to avoid crashing
into any of the city boundaries.
Those are the black squares as we see with a -100.
We wanna stay away from them.
They have more of a punishment.
So in order to do this, we have our grid,
the environment that we created above with our rows,
but we also want to assign these values to it.
So in order to do that, let's try to think
about how we can establish that.
We can use NumPy,
and we can also start setting it to a -100
for environment rows and environment columns.
So we have our environment rows and environment columns.
So let's call this rewards
equal to
numPy.full,
and let's pass in our environment rows, environment columns.
And to set our values, we can start with -100.
In addition to that,
we also want to set our rewards window
to
use the indexes of zero and five,
equal to 100.
And this is gonna make sense in a second.
So we're taking a look at our green square.
We have our zero and five,
we have our green square set as 100,
taking these or this location to set the value.
Now I'm gonna paste in the next snippet of code
so we can go through it,
so you don't have to watch me write each step out
because it's a little repetitive.
And now we have our white spaces.
Let's set our note for our reward points.
And in this snippet of code, we're using a dictionary
and setting each of our values within our dictionary
so we have our aisles.
We're thinking about that as each individual row.
And we can set with our slicing, with our index,
one through nine, and we want to use an iteration
with our for loop to set these values.
In doing so, you'll see if we look
at 1-10, 1, 7, and 9,
and by using this, we can actually set the row index
in our range of 1-10,
which we're working through in our environment,
we can set the column index in our aisles of our row index
with our dictionary, our rewards row index and column index,
we can set it to -1.
So what this is doing is essentially,
if we look at each specific-
If we take a range here, for example
aisle 9 for i in our range
we have -1 set throughout the entire environment
or each state within our environment.
For eight, we have three and seven.
So if we can scroll up,
we can see that in three and seven
we're setting a negative one,
since they would all be -100,
or set to a negative a hundred.
And with this iteration, we can set those rewards
or set each state that we're defining
within our aisles to negative one.
It makes it very easy.
Instead of having to write further logic
or maybe more detailed functions or statements
we can iterate through and set these values.
I highly recommend that you take a minute to
explore experiment if you wanna change the environment later
after we run this solution.
It's a great way to help learn and reinforce these policies.
But this is starting to take shape.
And a cool thing we can do is we can actually visualize it.
So let's do four-row in rewards.
Print row.
And let's print this.
And we can see I might have to actually re-run the cells,
my apologies, I wasn't connected
to the notebook working through here.
Give it one second.
It's gonna throw an error.
I need to go back through and re-run the cells.
So let me just run this really quickly.
I want to import numPy.
I can actually just, oh, my apologies.
Let me go through this here and I'm gonna come down.
We wanna run our environment, we wanna run our actions.
The other ones are just texts, so we don't need them
but I like to have that in
so you guys can have a reference to it.
We want reward points
and we finally want to visualize this.
We can see the visualization
the actual numerical representation
in numPy of our environment.
Really cool.
So we have our environment set up.
Amazing work.
Hope you guys are finding this useful.
Now we're gonna cap it off here because
in the next lecture we're gonna start training the model.
So this was to set our Agent's actions
to set our environment, to set our rewards, our punishment.
This is gonna help the Agent establish
the optimal policy within Q Learning.
Overall, we have this visual representation
or this image built
and we can see it here if we print out our row.
Amazing.
All right, I won't keep rambling on.
Let's cap it off here in the next lecture.
Let's start training the model.
I'll see you guys in the next lecture.