Maximilian: Now, regarding that programmatic usage
of these locally running AI models,
it's also worth noting
that LM Studio also offers another API
that has different endpoints than the OpenAI compatible one,
which I was using in these examples I showed you.
So, you could also communicate with that LM Studio AI server
through that API and you would talk to the same models.
The API just has a different shape
and accepts parameters in a different shape,
and you can learn more about that here in the docs.
And it's worth noting that you can, for example,
also request structured outputs by, again,
sending such a JSON schema to the API
because you can configure your requests.
For example, when using the OpenAI SDK here,
you can set a response_format property
and set this to some JSON schema
that describes the shape of the data in JSON format
that you would like to get.
You can also set other parameters
like the temperature here, for example.
And therefore in general, of course,
if you plan on interacting
with your locally running AI model
through LM Studio programmatically,
you should definitely also take a closer look
at their documentation
to learn about all the features that are available
and all the parameters you can send
and set through that API.
You also might want to explore Headless Mode,
which allows you to run that LM Studio API server
without running the LM Studio application itself,
so that the server keeps on running
even if you close this application,
which could be useful if you try to run that
on a remote server.
But that's all beyond the scope of this course, of course.
This is not an in-depth programming course.
Instead, it is about interacting
with locally running open large language models.
And as you saw,
you can also interact with them programmatically.
For example, here with LM Studio,
with help of the OpenAI SDK.