Hey there, Eden here.
And in this video I want to introduce to you to the with structured output method, which is the preferred
way to get structured output out of large language models.
This is a much more reliable method to get structured output, and it is the modern method of getting
it now.
It's more reliable because it leverages function calling of large language models.
Now we're going to elaborate and go very, very deep on function calling.
So don't worry about that.
In this video I'm simply going to show you the interface and how to use it.
And at the end of the video we're going to be comparing the two methods.
And we're going to really see the differences between them.
All right.
So let's go to the code.
And I want now to run everything.
So I will have a fresh trace.
So I'm going to remind you that the output parsing actually was the last step we did where we leveraged
the parse method of the pedantic output parser.
This entire process relied on the fact that we injected to our original prompt the formatting instructions
to the LLM.
We hope for the best, so we hope that the LLM is going to enforce that.
And then a link chain is going to have an easy time parsing it and turning it into a pedantic object.
Right.
So you can see the um the call was successful.
And by the way, it was successful because we used a model which is pretty strong, which is GPT four,
even though it's not the latest GPT five, and it's more than capable of doing this task.
Now, in the early days when we had GPT 3.5 or 3.5 turbo, that wasn't the case, and we'd actually
get a lot of errors when doing it.
And it was actually very error prone.
And it's actually nice to see that as the models become better, the applications inherently become
more reliable, right?
So let me show you the trace in Langschmidt.
Just to remind you everything we did in the previous video.
And let me open up the trace.
And here you can see the last call we made.
And here you can see the format instructions that we instructed it to enforce the schema of our agent
response schema.
Okay.
So it's eventually some text we inject to the prompt being sent to the LLM here.
Now notice this text by the way.
It was sent over and over and over again to the LLM.
Even though we didn't need it, we only needed this part of the prompt in the last iteration when we
wanted the answer from the LLM after it executed the tool.
And in the previous steps, we didn't need it.
We didn't need it for the reasoning to choose the search tool.
So this is only for the output schema right.
So this is problem number one or optimization number one we can make here.
All right.
So after the LM responded with the answer, hoping to be according to what we requested, we ran the
runnable lambda to get the answer field.
And then we ran the runnable lambda, which used the dot parse method of the pedantic output parser,
which took this JSON schema and simply casted it into a Pydantic object.
All right.
So let's go now and let's switch to the structured output.
So the first thing I want to do is to get rid of this pedantic output parser import.
And let me remove also the output parser object.
Here.
We do not need it anymore.
And here's the magic.
I'm going to create a variable called structured LM.
And I'm going to use the lm.
We initialize GPT four and I'm going to use the with structured output method.
And I'm going to give it the input of the pedantic objects of our agent response.
So what's this with structured output method?
It's going to create us a new version of the model, a new instance of the model that is specially configured
to produce output in a specific structured format, which is going to be the agent response schema,
which is going to be the blueprint which defines exactly what kind of structured data we want the model
to return.
Now, the structured data now is the new model instance.
And when we're going to ask it something, it will always try to respond with the data that exactly
matches the schema of the agent response object.
So normally llms they just return as plain text.
So this can be unpredictable.
This was the case in older and less capable models.
And it can cause problems downstream when we'll try to parse it in the application layer.
Now conceptually, you can think about this line sort of like injecting to the prompt like we did previously
with the schema instructions.
However, this is leveraging the function calling capabilities of LMS, which is going to make everything
much more reliable.
And we're going to be elaborating on that in future sections.
So don't worry about it.
All right.
So now let me get rid of those format instructions.
We do not need it anymore.
And I'm simply putting here some empty strings here.
So remember this special prompt the special react prompt with the format instructions.
So right now it's going to be populated with empty strings.
We can also use the original react prompt.
It doesn't really matter.
So I'm just going to continue.
All right.
So it's important that when we're going to run our reasoning engine with the create react agent function,
notice we're passing in the regular LM without the structured output.
So we don't pass through the variable structure LM we want to use that special instance with the structured
output only in the last iteration after the LM has created the answer, and we just want to format it
nicely as the pydantic object of the agent response object.
All right.
So let me now remove the parse output line here.
We don't need it.
And the last execution in our chain is going to be piping it to the structured LM here.
And in this last step here in our chain the structured LM is going to convert the text output to structured
data.
And this is going to be based on the tool calling capabilities.
And again I do not want to dive deep here in tool calling.
We're going to have an entire dedicated section for that where we're going to see underneath the hood
how everything is working.
But what's important to see here is that the structured output here is replacing a, the injection of
the schema to the prompt, and In be the parsing itself, which previously we executed it explicitly.
So to recap, this is the interface of using the with structured output method.
All right.
So let me go now and execute the code.
And I'm going to fast forward everything.
And what's important to see here is the output.
And then we're going to explore the trace.
Right.
So here we can see the output.
And we can see we got the output which adheres to the pedantic object.
We can see we got the answer and we can see the sources.
So everything is a like we expect it to be like in the previous run.
So let me go now to Langschmidt.
And let's see now the second trace and notice something interesting here.
Notice here the prompt that we sent to the LLM.
Every iteration we don't have the format instructions.
So we didn't give any information about the schema and how the answer should look like.
We're leaving everything to the LM, and this is actually saving us also tokens.
So let me go to the runnable lambda which is going to extract the a result here.
And this is eventually what the LM is going to return in the output field I remind you.
And here we can see this is simply text here.
And in the final part of our chain we're going to now make another LM call with this as input.
So let me go and show you it.
And you can see we have something which is new here which is tools.
And we can see it's called we can see here agent response.
We can see here the pydantic schema of our agent response and what the LM returned us here.
The structured LM is an agent response object with the following fields.
So you can see it right over here we have the answer field.
It has the sources field with the URLs, and it's going to return it in a JSON, by the way.
And now it's displayed as a YAML.
So this is how Link Smith chooses to display the JSON response.
And the important and cool thing to show you here is that remember the pedantic output parser?
We didn't put it explicitly here, so link chain put it for us when we used the with structured output.
And it's simply going to take the JSON.
And I remind you this is not a YAML.
This is a JSON.
You can go and click in the see it as JSON.
So link chain is going to take this JSON response from the LLM which adheres to the schema which the
LLM use function calling to create it.
And it's going to use the pedantic output parser in order to convert it into a pedantic object.
It's going to be very similar to what we did before.
However, here link Chain is doing it for us.
Cool.
Let's go back to the code and before we commit everything.
Let's discuss the differences between pedantic parsing and with structured output.
So as far as ease of use, pedantic parsing is harder because we need to inject a prompt.
Then we need to parse it.
And when we use we structured output, we only need to plug in the pedantic object and that's it.
Now as far as reliability and this is the main difference, we structured output is much more reliable
because it leverages tool calling of Llms.
And I'll be showing you this in later sections of the course.
So this is the main reason why it's preferred to use with structured output.
Now when it comes to model support, the original pedantic parsing is supporting all models.
I mean all models that has decent reasoning because it doesn't rely on function calling while the structured
output method relies on function calling.
Now, it's important to note that all the state of the art models these days have function calling,
so this is really a non-issue now.
And lastly, regarding control, the prompt control.
While the pedantic output parser, the original way has a bit more control because we really control
what's going to the prompt to the LLM, we can really use all of these features when describing the
pedantic object.
So eventually when using with structured output, we can still get the same results here.
All right.
And just to reiterate, the preferred way to go in output parsing and getting structured output is to
use the with structured output method.
All right.
Let me open now cloud code and let me commit everything to the repo so you can find the code.
So I'm going to write in the prompt create a commit from my new changes in a main.py file.
All right.
So it's going to write the commit.
Let me now go and push it to the repo and let me show you the commit and the diff.
So let's go to the repo.
And I'm going to choose the branch of react search agent.
Here.
Let me go now to the commit list.
And here the final commit refactor output parsing.
Here you can see replace pedantic output parser with the structured output method and downgrade the
model to GPT four.
And yeah so this is the diff.
You can see here I will be linking it alongside with the traces in the videos.
Resources.
Now I just want to remind you that in this video, I only wanted to show you the interface of how to
use it.
And I didn't talk anything about function calling.
And we have an entire section dedicated to that.
So don't worry if you don't understand how this is working under the hood.
We're going to soon figure it out.