-: Hello and welcome to this Python tutorial.
All right, so now we're gonna make the push function
which will do two tasks.
First, it will append a new transition
or a new event in the memory.
And then second, it will make sure that the memory
has always 100 transitions.
I'm saying 100 because we gave the example
of 100 events in the previous tutorial,
but in fact this will be much more than 100.
This will be rather maybe 10,000 or 100,000.
We'll see. But anyway,
this value will be the capacity.
All right, so let's make this push function.
So as usual, we start with def, to define a new function
and then we give a name to this function.
So we call it, push.
And this function will have two arguments.
First, as usual, self, that refers to the object.
Add the next one.
What do you think that will be?
Where I'll remember, this push function will be used
to append a new event in the memory.
We already have the memory, so what we need now,
as a variable, is an event.
So that will be our argument, our input,
and we will append this input in the memory
which is a variable of the object.
All right, so event,
you can actually call it event or transition.
That's the same.
And you will see in the next code sections,
what exactly is this event, what form it has.
Actually I can tell you now, this event,
this transition that we're adding to the memory
is a total of four elements.
The first one is the last state, that is st.
The second one is the new state.
That is st plus one.
The third one is the last section, that is 80,
the action that was displayed.
And the fourth one is the last reward,
the last reward obtained. That is rt.
So that's exactly the form that this event will have.
All right, and that's all.
We just need the event because we just want to
append the event to the memory,
and then making sure that the memory
has capacity elements.
All right, so now let's go inside the function.
So the first thing we'll do is append
the new event to the memory.
And that's very simple because we're gonna use
the append function.
So that will be direct.
And when we use the append function, we must start
with the list to which we want to append something.
And this list is of course, memory.
So we start with memory.
And since memory is a variable of the object, we start here
with, self dot memory.
There we go.
So self dot memory, and then we add a dot
and then the append function, which is the first one.
So append, and inside the append function,
we input what we want to append to memory,
which is of course our events.
So event here.
And that will append the new event,
composed of the last date, new state,
last action, and last reward, to the memory.
All right, so that's the first thing done.
And then the second thing we need to do is
make sure that the memory always contains capacity elements.
So let's say capacity is now 100,000.
That's probably the capacity we'll choose, because
then 1 million elements might make the training slow.
So let's say 100,000.
Now we're gonna make sure
that our memory always contains 100,000 transitions,
100,000 events, and never more.
So of course at the beginning
it will have one, then two and three.
But then once it reaches 100,000 events,
well it will always have 100,000 events.
So to make sure that we simply need to make an if condition
with this upper bound that we don't want to go over.
So if, okay so the idea that we'll use here
is that if we go over the limit,
well we will delete the first transition,
the first event of the memory, and therefore
we're gonna take the length function
to take the length of the memory,
that is the number of elements in the memory.
So here in the length function, we input self dot memory.
That's the memory.
So if the number of elements in self dot memory is larger
than the capacity, well, in that case
we will remove the first element to make sure
that the memory always has the same number
of capacity elements.
And to delete the first element
there is nothing more simple.
We're gonna use another function
which is the Dell Python trick.
So Dell, and therefore
we want to remove the first transition
which is the oldest transition in the memory
because the last transitions are the ones that we append
and therefore that's the newest transitions.
So the first transitions are the oldest one.
And so here we want to delete self dot memory and bracket.
And we take the first element of the memory
which has index zero.
So self dot memory zero.
Now interesting, I have a little warning which says
that there is a non-defined name capacity.
That's because the capacity here is not the input.
That must be the capacity variable attached to the object.
And therefore, here we need to add a self,
self dot capacity.
And now the warning is gone.
So now you understand even more the use of self.
That's really to refer to the object
to take the capacity of the object that will be created,
that is an instance of the replay memory class.
All right, so we're done with this push function.
And so now we can move on to the next function
which is the simple function
which will take some random samples
from this memory of the last capacity elements.
And doing this will improve a lot
the deep learning process.
All right, so let's do this in the next tutorial.
And until then, enjoy AI.