Presenter: Now, Ollama does not just have its own API,
which is really powerful because, as explained,
you can do everything you can do
through the command line, also with help of this API.
But they also have an OpenAI compatibility
just like LM Studio did.
So the great thing here is that you can use the OpenAI SDK
as explained in that LM Studio course section
because that OpenAI SDK whilst being built by OpenAI
and whilst, of course, being built in order
to use their API, this SDK can also be used
with other APIs by overriding this base URL setting
and by, for example here, setting it to this address
where the Ollama API is running.
Now, all of that wouldn't work if the Ollama API
wouldn't support it and the default API indeed
has a different shape,
so that API wouldn't work with the OpenAI SDK,
but Ollama simply also offers an additional API
in addition to their built-in default API.
And that additional API is unlocked
and available out of the box.
You don't need to do anything special for that.
So, by just changing that base URL to this URL,
you can communicate
with the Ollama OpenAI compatible API
through the OpenAI SDK, for example.
And, therefore, for example, you could have code
like this where I'm using that OpenAI SDK,
where I changed that base URL
and where I set that API key to not get a warning or error,
but the value doesn't matter.
It's not going to get used anyways
because the model is running locally on our system.
We don't need to authenticate.
But I did set up this client
and then I'm asking for a chat completion.
So for an AI-generated response,
based on that made-up chat history
which I am sending to that Gemma 3 model again.
And, again, here, you could enter any other model identifier
as well, including one of your own customized models.
But I'll go for Gemma 3 again.
And, therefore, now, if I execute this basic py file,
I'm using that OpenAI SDK to communicate
with my locally running, Ollama-powered Gemma 3 model.
And, therefore, here, I got the response by that model.
Of course, you can, therefore, also build your own chat
by again, using the OpenAI SDK
and the Ollama API simply by having an infinite loop
where you keep on asking for more input
and where you then keep on asking the model.
Or you could build an image parsing application
where you find all jpeg and png files
in this demo in a given folder,
in my case here in the images folder,
which indeed is a folder I have in this project,
which includes this image of a lake taken from a plane
and of me in my recording setup.
And you could load these images
and send them again with help of the OpenAI SDK to Ollama
if you're talking to a model that has computer vision,
which is the case for this Gemma 3 model.
The images are converted to a format called Base 64
which is essentially a text representation of the images,
which is the kind of format Ollama and the Gemma model wants
for taking a look at the images, so to say.
And then you can ask questions about these images,
like, for example, you can ask Ollama,
and the Gemma 3 model to describe this image in detail.
And I do this for both images.
So if I execute this image parser python file,
it starts with the max image here,
sends that along with the prompt
to Gemma 3 powered by Ollama.
And since I'm not streaming in the response,
this can take a while.
And then here, I got that response
where indeed it again correctly describes
that I'm making a thumbs-up gesture
that I'm in a recording streaming setup
and all the other details here also sound good to me.
And it's then, of course, also processing
that other image of that lake.
And here, with it being done,
it, again, correctly captures
that this was taken from an airplane.
Here, we also see that we got some layered mountains
and, of course, that lake.
So that all works and is another example
of using an Ollama-hosted,
locally running OpenAI model programmatically.