-: Hello, and welcome back
to the course on artificial intelligence.
So we've talked about the Bellman Equation,
and we've analyzed our little maze.
Let's have a look at the plan.
What is the plan?
Well, here is our maze analysis,
and we know that we can see, actually,
the values of each state.
We can see what the value of being in every single state is,
and therefore, the AI can,
or the agent can navigate this maze.
So what is the plan?
Well, the plan is simply like a treasure map
for the artificial intelligence.
Instead of looking at these values,
let's just replace them with arrows,
which indicate in which direction the agent should go
because it knows those values.
So, an ideal scenario after it's explored this environment,
it knows the values of being in each state
and therefore it can come up with this map.
So let's have a look again.
We know that here value's one.
So if you are here,
out of the two,
the better one is this one, so you go right.
From here, out of the two, this one's a better one,
this one's a better one, this one's a better one.
Or actually, from here you have two options, right?
So here's kind of like a tie.
So you just pick one at random, doesn't matter which one
because the value in either case is the same
and moreso even if we look through,
it'll take the same number of steps to get to the end.
From here, you've got three options,
but this one is the better value,
from here this one is a better value,
from here, obviously this one is a better value,
because here you just get a minus one reward right away.
And from here you have three actually,
but this one is the best one out of all of them,
this value of the state.
And so therefore, if we replace them with arrows,
it makes sense that this is how the agent would go
if it starts here,
or if for some reason it ends up in this square,
it knows how to get out here.
It starts in this square, it knows how to get out here,
and so on.
So that is what a plan is.
And don't confuse plan with policy,
because we are gonna be talking
about policies for their own.
Policies are very similar to plans,
but they have a little trick to them
because the environment's gonna be a bit different.
It's gonna be stochastic.
And that's what we're going to talk about
in the next tutorial.
So I can't wait to see you on the next one.
And until then, enjoy AI.