Maximilian: Now, there's one last setting
which you can access as a power user,
which I want to show you,
and that's this structured output setting.
And the idea here is that you can kind of force the model
to give you this output in a structured format,
in JSON format, specifically.
Now, JSON, in case you don't know, is simply a format,
a data format that looks like this.
This here is JSON. That's an example for a JSON document.
It, in the end, is text that's formatted in a certain way.
And having a clearly defined format
and having text that follows a certain format,
having data that's formatted in a certain way,
therefore, is of course very useful if you, for example,
plan on using this output as an input
in another application or another step of an application.
That's why it's quite common that you want LLMs
to produce JSON output, formatted output,
so that you can use the output of an LLM,
not just as a human, but instead, for example,
also in another application.
For example, here, I might wanna extract
those key financials not as text as is the case here,
but instead, I might want to extract the key financials
in a structured format.
And to get the model to do that,
you can go to the model settings here,
turn on Structured Output, and then you have to define
a JSON schema here in this box.
Now you can visit this page
to learn how a JSON schema looks like.
If you scroll down, you will see an example,
and you see that it is itself a JSON document
that describes the structure
of the to-be-generated JSON document.
So for example, here we can copy this example
and paste it in here.
Now it's a bit hard to edit here.
Later, we will also see how we could do this in code,
but for now, this will have to do.
Now, the start here can all stay the way it is,
the ID, the schema.
Now for the title, I'll say, Financial Data.
The type should then be object.
You need to have a properties key,
and then you describe the shape of your data, so to say.
For example here, I want to have a year field,
which is of type string, which means text,
and I describe what will be in this field.
And this description is important,
because it tells the model
which kind of data to put into this field.
So here, I'll say the year of the report.
Then I want to have another field, which could be revenue.
And here, maybe the type should be number.
And the description here is the total revenue
of that year or of the year.
And then here, I want to have another field,
let's say operating_income,
and here the type is integer, sorry,
that should also be integer here for the revenue,
not number, that's a specific kind of number,
a number without decimal places.
So that's the revenue and also the operating income.
And it doesn't matter whether the type comes first
or the description.
So here, the description then is
operating income for the year.
And you could have other requirements
like a minimum value of zero, but I don't care about that,
so I'll remove it and remove that trailing comma.
And you could add more to extract more data,
but I will just specify these three fields,
year, revenue, operating income,
that's all, that's all I want.
And now, we could set this year
by having this check mark enabled
and then go back here to our prompt and say,
Return the key data as JSON.
And that prompt combined with this structured output setting
that's set for this chat behind the scenes,
that should get the model to hopefully return data
in that JSON format,
a JSON format that follows this structure.
Now, as you can see,
the results I got here don't look that great.
Not only is the year missing,
the numbers also look a bit off, to be honest.
And therefore, what I recommend
when working with structured output is that in your prompt,
you also repeat the values you want the model to extract
so that you increase the likelihood
of it picking the right values.
That's the case with all models
whenever you are working with structured outputs,
not just when working with open models.
So structured output is not a strong guarantee
that you will definitely get the data
in the format you requested,
but it makes it way, way more likely.
And combined with a strong prompt
that asks for JSON and maybe specifies the key data
you're interested in again,
which of course has to be in line
with your structured output schema here,
it is pretty much guaranteed
that you will get back a result that is text,
that is formatted according to your JSON schema here.