Instructor: So in this video
I wanna focus on the LLM as a reasoning engine.
And specifically I wanna focus on the LLM call
that we make to determine which tool to use
or whether if we have the answer or not.
So we solve the prompt that we're sending,
which is documentation of the original question
with all our tools and showing
and describing to the LLM how it should format its output
with the action action input format.
But now, after we added the agent scratch pad,
I want to show you exactly what are the calls
that are being sent to the LLM,
and I want to show you why it is working as it is working.
So to do that I want to create a new file
and I'll call it callbacks.
And this file is going to have all the logic
that is going to log all our LLM calls.
We're going to create a new class
and we'll call it AgentCallbackHandler.
And it's going to inherit from a class,
which is called the BaseCallbackHandler.
Now let's import it from LangChain.
So what's the BaseCallbackHandler?
I'm gonna go to Google, I'm going to search
for callback handler LangChain
and I'm going to click on the first result.
So basically we have this base class
that we can inherit from
and override all of the functions that we have here
that will be triggered in every LangChain interesting event.
What's an interesting event in LangChain?
Maybe a call to the LLM, a response from the LLM,
maybe when we select a tool, or after we execute a tool,
or when we get an error,
or when we have a new token on the LLM.
So all of this we can override
and this helps us with tracing.
So let's go back and I want to override on LLM start.
So it's when we send something to the LLM in on_llm_end.
So this is when we get a response from the LLM,
let's now import all those type hinting.
And now let's override those methods.
So when we start and send something to the LLM,
I want to print nicely what is the prompt for the LLM.
And this is I'm going to get from the prompts argument,
and I'm going to take the first one
because I'm assuming there is going to be one prompt.
And once we get the answer from the LLM,
I want to log it as well.
So I'm simply going to write some nice prints
over here as well.
And I'm going to access the response objects
and get what was generated by the LLM.
So you can just copy that.
It's not very interesting.
You can inspect those objects yourself.
So I'm not going to go over and describe this entire object.
Of course, all of this code is available
in the courses resources.
So let's now go and import our AgentCallbackHandler.
And now we want to use it
and because we're logging in LLM responses
and LLM calls, so we're going to inject this object
into the LLM variable, which is an instance of chat OpenAI.
So this one here receives an argument of callbacks
and right over here we'll send it
to the AgentCallbackHandler.
So this will log all the responses and calls to the LLM.
Let's not forget to instantiate an instance from this class.
And let's run everything.
So we can see now we have a couple of prints
and we will soon see what are there.
So we can see that we start with this prompt
and this is the ReAct prompt you see right now.
And here we plugged in all the tools that we have
and we have also the tool names
and we have also what is the original question.
We see now what is the LLM response
and this is what was sent to the output parser.
And if we copy that,
I want to show you in action how it works.
So I copied that and I want to head up
to OpenAI playground.
And right over here we can put our prompt.
So I'm simply going to plug it in
and I'm going to execute it.
Now you can see right here
that the response we get from the LLM
is not the response that we logged.
And this is because this is the same response,
except that here we have also an observation part
and another thought and another final answer.
Now I remind you that this is the first iteration
of our reAct loop.
So we need this to select the correct tool.
Now if we would've sent this to the output parser,
then we saw already that it would generate
some kind of error for us
because it would have also a parable
and also a final answer.
Anyways, so this scenario,
we see that the LLM generated too much for us.
We only need the first four lines here.
So how do we fix it?
With a stop token.
So remember when we initialize the LLM,
we plugged in a stop token of back slash observation.
That's exactly why we need it
because once the LLM would encounter the token
of backslash end observation it,
it'll stop generating tokens
and it'll not include it,
yielding us with the first four lines
of this response, which we need.
So let's add in the stop sequences,
the backslash observation
and send the response
and we can see that we get indeed the answer that we need.
And this is the exact answer that our agent received.
So let's head up back to our logs of the LLM
and we can see that in the LLM response, that's what we got.
And now we went for the output parsing.
We got this output, we used the output parser
to parse the tool name and the tool inputs.
Then we ran the tool and we got an observation,
the result of the tool.
So after that we initialize another iteration
of the reAct prompt.
This time we sent in the agent scratchpad
all of the history of what has been done so far.
So we can see right here with the action,
action input, and the observation from before.
And we can see that from here
that the LLM responded us with the answer.
So it responded as I know the final answer,
final answer call, and then the final answer.
And then our output parser read that
and it created an object of agent finish
with the final answer.
And then we finished the iteration.
So we didn't actually create a while loop,
but let's go and do this
because it's going to be a very easy fix.
So I'm gonna go kind of to the top of the file
before we execute anything,
and I'll define that the agent step is going to be empty,
and now I'm going to create a while loop
while the agent_step is not an
instance of agent_finish.
So as long as we have something to do
and now I'm going to take everything and indent it.
However, we don't really need the second evocation
of the agent because we're going to do that
and execute it as long as the agent step
is not agent finish.
So let's remove that.
And that's pretty much it.
We get and execute line 96 if we finish this while loop.
So the agent_step is definitely agent_finish.
Anyways, let's go and format it a bit
and let's now run it and test it
and see that everything is working as expected.
And essentially what we are running right now
is the agent loop.
So this is what happens when we initialize an agent
with the LangChain built-in function.
We give it a zero short react description
and this entire while loop,
this is exactly what the agent executor is running.
And if you want to support me in the course,
I would appreciate very much
if you can leave me a Udemy rating.
This really motivates me to continue
and create new videos for this course and reach the content.
And it also helps feature students
to decide whether this course
is a right fit for them or not.
So if you don't mind, I'd appreciate if you can pause,
go to the rating section and leave me a Udemy review.
Thank you so much and see you in the next video.