-: Hi everyone, and welcome back.
In the last lecture, we ended up finishing
or finalizing the actual definition of our environment,
the representation of our values, our punishment
and rewards, and our actions in the environment.
In this lecture,
what I want to do is to introduce the training, the idea
behind how to train the model, and to help get you started.
And then in the next lecture, provide the solution,
and more of a breakdown for these steps involved.
So, we need to train the model.
What would be required of training?
This is an approach that we can use for the following steps
for training the model.
We wanna choose a random non-internal state,
which would be the white square of our agent,
and that's how we want to begin
in our episode for the training.
We then of course
want to choose an action for the current state.
We need to have our agent being able to move
around the environment.
So, actions for us and our agent in this challenge
are gonna be chosen using Epsilon Greed.
This algorithm will usually choose the most promising action
for the AI agent, but occasionally choose
a less promising option in order to encourage
the agent to explore the environment.
We really wanna find that optimal policy.
Then, we wanna perform the chosen action
and transition to the next state.
Move to the next location.
And why I'm saying this is I want you guys
to think about how you can break this down
into functions to solve this problem.
Next, we need to receive a reward for going
to a new state, and then calculate the temporal difference,
we have to update the Q value
for the previous state in action pair.
And if the new current state is a terminal state,
we would then go to one.
Otherwise, we'd go to step number two.
So, the entire process, we're gonna aim to run
for a thousand episodes to train.
This is gonna give us enough opportunity, or our AI agent,
sufficient opportunity to calculate that shortest path
between the item packaging area and other locations
in our example, Citi.
Awesome.
So, please think about how you would approach this,
and I wanna help give you an idea to try and solve this.
So, we'd be looking at, in our solution, we're gonna be
using the following functions.
I can actually comment this out
since it's a code cell, my apologies.
We don't want a dollar sign.
We want to comment these out.
And I'll help get you started
on the first one.
Overall, we're gonna have these functions defining our steps
for training the model.
And to help get you started, let's take a look at how
we would approach is terminal state
in creating a function within Python for this.
But first thing's first, we would of course want
to define our function as is
terminal state.
And these are the names of the functions that you'll see
to give you an idea that might help you break it down.
What we wanna do is take the current row index,
and the current column index.
This is gonna help give us the position
of our agent, and we can add
here for our rewards, a true or false.
So, we need an if statement.
So, let's set our if, our rewards of our current
row index, and current
column index, equal to negative one.
If they're in that state,
we would return false.
Otherwise, or else we would return
true.
Pretty straightforward enough.
And this is how we're gonna get an idea
of our is terminal state.
After we have, if it's existing in the terminal state,
we then wanna get the starting location.
As a hint, you can take a look at using the current
row index and the current column index,
and setting NumPy with a random.
We wanna initialize that randomly
to the environment rows, environment columns.
But in the next lecture, you'll see a breakdown,
you'll see the rest of the functions
with some notes to help give you an idea.
I really hope that you guys take the chance to experiment
with this because it is just an awesome way to learn,
help give you an idea, and this is to get you started.
So don't worry, you'll get the solution in the next lecture.
And then, we're gonna wrap things up by actually
assigning an epsilon, discount factor, learning rate,
those type of things for training, running the training,
and viewing the results.
Awesome.
Let's stop here.
Again, try to solve this, but if not,
if you wanna just advance, go to the next lecture,
and you'll get the solution
for the training, for defining of these functions.
All right, I'll see you guys in the next lecture.