Maximilian: So the model file concept
is an important concept.
It allows you to build customized versions
based on existing models.
You can also take it a step further though.
You can technically also build new models with a model file
that are not based on base models from the Ollama catalog.
Instead, you could have a GGUF file,
which is a file format that stores these parameters,
these weights, along with some metadata.
And you could get such a GGUF file
because you train the model yourself or from Hugging Face.
Now for example, here, I have the Qwen3 model,
or to be precise, a quantized version of that model.
It's a normal model shared on Hugginng Face.
And that is a model
that is indeed already supported by all Ollama.
So you do find Qwen3 in their model catalog as well.
But if it would not be included there,
and for some models on Hugging Face,
that might be the case
because whilst most models and definitely all important
and popular models are part of the Ollama model catalog,
some models might not be.
And therefore for such situations,
you could download such a GGUF file from Hugging Face.
You could download the model,
the parameters from Hugging Face,
and then use that downloaded GGUF file as a base
to set up your own Ollama-compatible model.
How do you do that?
Well, here for this example,
and you find a link to that attached.
There is a list of download links here
for different quantized versions of that model.
And I wanna use the Q4 KM version here.
And in general, as you also see here,
it's recommended to go for the Q4 versions,
the four bit quantized versions, and then KM or KS.
These typically give you the best trade off between quality
and performance and reducing hardware requirements.
And you can download this file here
directly from Hugging Face.
And again, this is just an example.
You could also have a model
that you trained yourself of course,
and then you could create a model file
next to that downloaded model.
So next to this downloaded GGUF file,
which is a file that contains
all these parameters and weights, and some metadata.
And now with the from instruction,
you would point at that downloaded file.
So you would use that file name including the file extension
and specify it like this here.
And now you would build a brand new model,
or you would define a brand new model
based on that downloaded model,
based on these parameters and weights.
Of course, you can still add a system message
and parameters,
but you also might want to specify a template.
Now the great thing about GGUF files is that,
as I explained,
they do not just contain the parameters,
but also some metadata.
And one important piece of metadata
they do include is the chat template,
which in case of Qwen3 is rather complex as you see.
You can also see that on the Qwen3 page of Ollama.
There, if you open the template,
you got a raw complex template there as well.
By the way, these template formats are different.
Ollama uses the Go templating language.
On Hugging Face here,
that is a different language.
So you would need to use
that Go templating language expected by Ollama
if you were to set up your own template.
Now I'll first show you what happens
if you don't specify a template though.
So I'll just set up a basic model file
that just has this from instruction and nothing else.
And of course we could add a system prompt parameters,
but I won't add anything.
And I'll then create my own Qwen model version
based on that model file with Ollama create.
And if I now run this and I ask it something,
I get a super weird response.
And I got this weird response by the model
because it had no structure at all, it had no template,
it did not understand my message I sent to it in the end.
The message I sent to it
was not structured in a way,
was not wrapped with the right identifiers,
it knew from the training process.
Now of course, I can use the chat template here
from Hugging Face and try to use that,
but as mentioned, this must be adjusted
to use the Golang templating language.
And whilst you could of course try to convert this manually
or with help of AI, here,
I'll take the easy route
and I'll just use the one that was built by Ollama.
And of course in reality, I would use the entire model
that was published by Ollama.
This is just a demo here,
but here I'll just use their template, copy all of that,
go back to my model file, and add such a template.
Add the triple quotes to insert a multi-line template,
and insert this here, like this.
Now you can ignore the fact that something's red here,
this should work.
So now with that template added,
I'll first of all remove my model, my broken model,
and then I'll repeat the command for creating it,
now based on this updated model file,
which does now include a proper template.
And if I now run this updated model
and I ask, "How are you,"
I get a more reasonable output.
Now this model, the Qwen model,
is actually a reasoning model which tries
to think first before it responds,
which is why we see this block.
But it generally understood my message
and it gave a reasonable response.
So that's working the way it should,
and that therefore does not just show you
how you can set up models from scratch
just based on such a GGUF file, for example.
But it also shows you the importance of adding templates.
Again, because it's important,
you don't need those when using the models
that are published in the Ollama model catalog.
And you also don't need to specify templates when building
your own customized versions based off those models.
But if you build a model from scratch, as I just showed you,
then these templates do matter.