Earlier on in the course, we looked at how you could use the Structured Outputs API from OpenAI to
get structured outputs from both the response format and also using function calling or tools.
In this video, we're going to have a look at how you can do that inside of Lang chain using this concept
called an output parser.
Now, when you're working with large language models, you'll often find that you don't want to return
data such as text.
You'll want to return structured data.
We can do this inside of Lang chain by using an output parser, which generally in Python will use a
Pydantic model, but in TypeScript or JavaScript it will use a Zod package model as well.
So let's have a look at how we can do that.
Firstly, we install all of these packages and we would set up the OpenAI key here.
Then what we'll do is we'll then go down here and we'll then import a couple of different things.
So we'll import the chat model.
We will also import the Lang chain call prompts.
We're going to import a chat prompt template that we saw before.
And also a system message prompt template.
Now we're also going to import a Pydantic output parser.
So I'm going to do from lang chain core dot output passes, we're going to import the pedantic output
parser.
And then what we're going to do is we're then going to also add in pedantic import base model.
We're going to have the field.
We're going to set up our chat model with model is equal to chat OpenAI.
After we've set up our chat model, we're then going to also add in one more package which is from typing
import list.
And I'm going to place this at the top of my imports.
Then what we're going to do is we're going to set up two different types of pedantic models.
One for a joke, which is a pedantic class of joke with a base model.
And notice we're inheriting from the pedantic base model class.
We're then going to have a setup, which is a setup to the joke and a punchline, and we're also going
to create a different one, which is a jokes base model.
And the jokes base model will just take a key of jokes, and it will return a list of joke.
And the field description is a list of jokes.
We then have our output parser that you can see which is currently set to output.
Pass a joke.
Pedantic object.
Then what we need to do is set up a template so I can say template is equal to answer the user query.
And we're going to add in some format instructions that we'll see in a second.
So we'll put format instructions.
And then we're going to put a new line.
And then we'll put the query.
And we're going to set up a system message prompt template.
And we'll do that and call it system message prompt.
And this is slightly different to the one that you saw before where we're doing this kind of format
where we have a from template.
And I'm going to also get rid of this one.
And we'll just have a chat prompt.
Template.
Chat prompt is equal to chat prompt template dot from messages.
And we're going to put in the system prompt.
They're going to get rid of this.
And then what you'll see here is we've got our template set up.
And this system message prompt template will basically use a template string.
And then when we invoke that, we're going to fill in both the format instructions variable and also
the query variable.
Now just to show you how that would look, we're going to then say messages is equal to chat prompt
dot invoke.
And here we're going to fill in these two things.
So we're going to have both the query what is.
We'll say what is a really funny joke about Python programming.
And then we'll also have the format instructions.
Now the format instructions are coming from Lang Chain.
And we'll just have a look at what the messages is.
So you'll see now that we've invoked that and we've saved this to a messages which has the prompt value,
you can see the actual format instructions gives a real good indication as to how you specifically want
to extract structured data.
This is at the moment just using the prompting version with a really good description of what to extract.
And then we've got our query at the end.
Here you can see what is a really funny joke about Python programming.
So this is purely using prompting techniques, which is how you originally had to do it when there wasn't
anything like tools and structured output APIs.
So now that we've got the messages formatted, what we can then do is do model dot invoke messages,
and then we can have a look at the result.
And if we look at this result and we look at the content of this result, you can see that it has actually
produced a JSON string that we could then pass.
So if we give this to the output parser, what we can then do is we can go and try and do this with
a parser.parse and actually storing that.
Let's try and for example store that.
So we'll have joke object and we'll store this.
And now what you can see here is we're actually able to extract a structured joke object directly from
the result content.
So purely just from prompting and adding in this parser dot get format instructions, we can already
start to get structured data out directly from the LLM.
And that's this format.
Instructions is originally how you had to do it when you were just using prompting techniques.
If we have a look, we can then do, for example, the joke object and the joke object punchline.
So this is how you can get structured data out using an output parser.
Now if we're having a look at the documentation for Lang chain, they have a better way of doing this.
Now with the dot with structured output that will actually use a model's ability to use something called
JSON mode, and JSON mode will give much more reliable results.
Rather than simply relying on prompting techniques, we can use the Pydantic models that we've been
doing before.
The only change we have to make is that when we make our LLM, we also then make a structured LLM using
this dot with structured output passing in the model that we previously had.
And then we can invoke that directly.
Let's have a look at how we can do that.
Firstly we're going to create a new cell and we're going to call it structured LLM.
And then we're going to just basically have our model dot with structured output.
The first thing that you're going to put in is the schema.
This can either be an OpenAI function tool slash schema.
It can be a JSON schema a are typed dict class or a pedantic class.
Because we're using pedantic, we're just going to put the joke in here.
And then as well as that, if you want, you can also change the method.
So you can see you can either do function calling or JSON mode.
Let's leave the method out for now.
Now we also just have to do result is equal to structured lm dot invoke.
And then we can say what is a really funny joke about Python programming.
And then if we look at the result.
And we look at the type of that result, you'll see we have a main joke pedantic object.
And we have both a setup key.
So I can do result dot setup.
And I can also do results dot punchline.
So that's it.
That's how you do output parsing using lang chain.
Going forward I would recommend using the model with structured output.
The only reason why you would actually use the one before is if the large language model, or the model
that you're using doesn't support JSON mode or structured outputs via function calling.