All right, so memory Linkchain have gone through a lot of iterations, and right now I'm giving you
the overview of the latest, best practice for setting up memory in your chat bot.
All right.
And overall, you can think about memory as simply stuffing all the memory and all the messages into
the lmco.
In some cases, it's simply not going to be enough because we'll pass the token limit.
It's going to cost us a lot of money, and we don't really need to send everything every time to the
LM, because even if we are using an LM with a large context window like Gemini 1.5 Pro with a million
token size, then it would cost us more money, it would be slower, and we may get worse results,
because we simply send a lot of garbage that the LM don't really need to handle.
And as you remember the saying, garbage in, garbage out.
Currently in link Chain, there are three main strategies to handle this.
And the first strategy is to simply ignore this problem and not do anything and still stuff everything
in your LM code.
So this is also useful when you have short chats between the user and your bot.
It's the most easiest way to start.
The second way is to trim out old messages, so we'll simply get rid of those messages at the very beginning
that probably are not going to be relevant to our chatbot.
And this is a heuristic here.
Of course, this is not always the case.
And another strategy is to do some processing over the messages.
So for example to summarize all of the messages and to save only the summary of them and the last couple
of messages.
And up until now we discussed which messages to save, whether to save them all, maybe to filter the
old messages or to save a summary.
But we didn't discuss where are we going to save those messages and how we're going to persist them.
And the new way to do it with the link chain ecosystem is to use link graph, where we have their check
pointers, which are going to help us persist those messages.
And it's very easy to use.
I'm going to show you the examples.
We're not going to have live demos and dive deep into those examples.
However, I do go over them in my lane graph course.
So if you want to get a coupon to those courses, feel free to ping me or post in the groups.
I love sharing coupons with you, but this is out of the course's scope because it really introduces
a lot of new topics, which I think it's better to separate in a different course.
All right, so let me show you an example of how do we pass all the past messages into our LM.
So you can see right over here we have this chat prompt template that we use the from messages function.
So right over here you can see our system message of our system instructions.
And here you can see a message placeholder object with variable name equals messages.
And this is our way of telling link chain that instead of this variable we're going to dynamically inject
here all the past history of the user.
And we're going to inject here a bunch of other messages.
The format is going to be a dictionary.
and you can see it right at the bottom here.
Then we are invoking the chain.
But we send a dictionary of messages.
And then we have here a list of all the past messages of the user.
So we have here the human message.
Then the I responded and then we have the human message again.
So this is the history that we are pending right now.
So again this is what we are sending to the Elm.
We still haven't discussed.
How are we sending it.
So in arena application, we'll use a persistent DB to save all of those messages and retrieve them
and get them and then to send them.
Okay.
So this is the sending part of the messages.
And you can see it's very simple to use.
All right.
So let's discuss now of how link chain or specifically lane graph can help us persist those messages.
And line graph introduces a new terminology which is called a check pointer or checkpointing.
And it basically means that every iteration, every user message that we send, or every AI message
that we send, then we are simply going to, not we.
But Landgraf is simply going to take this information and it's going to persistent in a DB.
So here we have a memory saver checkpoint which saves it in memory so it doesn't persist it.
We have other checkpoints like PostgreSQL, MySQL, Redis and MongoDB saver.
So a lot of integrations are more to come with Landgraaf, which help us persist those messages in persistent
databases.
All that we do is create this checkpoint object and pass it into our Landgraaf graph.
And again, I know that you are not very familiar with landgraaf graphs.
I do cover it in my Landgraaf course.
So again, sorry I could not fit this in in this course, but it's simply too much and I just want to
show you the concept.
But the most important thing that you need to understand is that this checkpoint is going to do all
the persistent for us, and it's going to persist.
It in the DB.
All right.
So let's discuss about how to treat messages.
So how to ignore messages that are old?
Maybe that we don't want in order to save on tokens and on latency and on cost and link chain introduces
a concept of a trimmer.
So this is an object that is going to trim those messages.
And we can create a trimmer with the trim messages function.
We can give it a strategy which link chain offers a bunch of ways of how to handle the trimming and
what to remove.
You can see we put here max tokens and the token counter to be Len.
So you can and you can trim by token number.
You can trim by messages number.
And you have a lot of options here.
And the terminology is very reminding us of the text splitter.
And in order to invoke the trimmer we use the invoke method.
And we plug in all the messages that we want to process.
And then we're left with the trimmed messages that we can simply send to the LM, like we're seeing
right over here.
Now, you might be wondering, why do we need Link Chain to do it for us so we can do it ourselves?
But this is a very convenient function which is already implemented for us.
So why reinvent the wheel where Linkchain knows about all the use cases and gives us a solution for
them?
I mean, that's at least my opinion.
All right.
So we discussed about trimming.
Let's discuss about summarization.
So this is another technique for saving tokens and for sending a very precise context to the LLM.
And this is basically done with the prompt here.
And you can see here this summary prompt.
And this prompt is simply going to receive all the history that we have.
And it's going to summarize it into one summary.
And this is what we are going to save in our persistent storage.
Every time we summarize the messages we don't need the raw messages.
So we simply go and delete them later.
And this is how we save tokens.
So the check pointer stayed the same.
But we did do some manipulation to the data before checkpointing it.
All right.
So I just want to reiterate on what we saw in the video and what I wanted to show.
So first of all, you don't need to understand Landgraaf right now.
I just want you to understand the concepts of what are we saving to the memory so we can save either
all the raw messages, we can save some trimmed messages, or we can save some summarization of the
messages.
And this is what we saw here in this documentation of linkchain.
Now of course, you can add your own processing of the memory of what you want to save, and you can
extend this functionality to whatever logic that is better for your application.
So Linkchain does give you the freedom for it, and it's very easy to do so.
The actual saving of the history and messages.
So persisting it is actually done by the Landgraaf check pointer which is the new preferred way to doing
things.
And the check pointer is an object which is simply going to take this data and it's going to make DB
queries to send it in the target DB, nothing more.
And I do go and elaborate on this in the link of course.
So that's pretty much it for this video and see you in the next one.