Instructor: The great thing about Ollama
is that it has a quite powerful API
that essentially allows you
to do pretty much everything you can do with Ollama
through the command line.
So for example, create models,
list available models, remove models.
You can do all these things through their API as well.
And attached, you'll find a link to the API documentation,
which again, is unfortunately
just a markdown file on GitHub.
But there you learn that you could also create a model
by sending a post request to the right endpoint.
You would have to specify a name for the model,
and then basically specify all the details
you would otherwise put into your model file,
like the base model, maybe a template,
parameters or messages, or a system message.
So you could build a model programmatically.
And of course, for your own use on your system,
you might not wanna do that using a model file,
and ollama create might be more convenient.
But of course if you're building
some kind of application which you want
to expose to other users from all over the world,
and you wanna automate the process
of building customized models,
then this could be a useful feature.
And it's not just that,
you can really do everything else as well,
including deleting or pulling models.
You can also generate vector embeddings,
which can be useful if you're building RAG systems,
but you can of course also generate a completion
or a chat completion that takes
entire chat history into account.
And I got an example for that here.
It's a very simple Python script.
And of course, you find all that code attached.
And in that script, I send a request to
that /api/generate endpoint, which is this endpoint here,
to generate a simple completion.
So a simple output for a received prompt.
Now this is sent to localhost:11434,
so that's the port that Ollama uses for exposing the API.
That's the default port it uses.
You could change that with environment variables,
but by default, that is the port it will use.
And then here, I'm configuring the request data
I'm sending along with the request,
specifically the model that should be used,
the prompt, and whether I want to stream the result
or just get it in one block.
And for the model name and the prompt text
here in this Python code,
I simply have two parameters
in this generate response function.
And the concrete values
for these parameters are set down there.
So the model is essentially my gemma3 model,
but I could also use my service-agent model here,
for example, that I have on my machine.
So this model which I created before,
and then here the prompt is hard coded into the file.
But of course, since this is Python,
I could also use the input function
to ask the user for input.
And now, whenever I execute that file,
I would have to enter an input manually in the command line,
and then that would be sent to the model.
Well, and then all that's happening
is that a post request is sent to that Ollama server,
because a post request is expected
and then my prompt is handled by that AI model,
and the response is then parsed, returned by this function,
and in the end then, here, this response is being output.
Or to be precise, here I output the response text.
And you can learn all about the possible settings
you can set when sending such a request to the Ollama API,
and more about the response format
in the official documentation, of course.
So therefore here, if I execute
this Ollama API file with Python,
I can ask what's LM Studio?
And you might recall that in my system prompt
for this model I'm using here,
I actually told it that it should not help
with questions related to LM Studio.
And that's why indeed here, it tells me
that it's a friendly and helpful assistant,
but that it's not able to provide information
about the specific software I asked for.
Of course, if I would switch back to gemma3:12b-it-qat,
which is that base model without
any special system instructions or anything like that,
if I would ask that same question here,
it would take a while to generate that response,
but then I would get a response that describes
in detail what LM Studio is,
that it's a popular desktop application
that allows you to run large language models
locally on your computer.
And we also get a lot of metadata related to that response.
So that's all that part here, including all those token IDs.
And I get that simply because I'm outputting that here.