Maximilian: Now when it comes to setting up model files,
there's one interesting instruction you can add,
and that's the template instruction.
The template instruction allows you
to specify a, well, template
that will be used by the model.
By the way, you can add multi-line instructions
also for the system prompt, for example,
by wrapping it with triple quotes,
just as you can do it when chatting with Ollama
through Ollama Run.
But that's just a side note.
But what is such a template then?
What is this weird thing here?
Well, it is also something you find on the model detail page
on the Ollama website.
There you have this template part.
And for example, there you can inspect the default template
set for the gemma3 models.
This also looks rather weird.
You'll also find templates on Hugging Face.
So for example, here for this quantized Qwen3 model,
there is a chat template available.
And if I click this, I get an even weirder template.
So what is all that about?
Well, you must not forget
that all these Large Language Models are token generators.
They generate tokens based on the input tokens.
That's all they do.
And therefore,
they also don't know about things like a system message,
or user messages,
or assistant messages that have been generated by the AI.
So in order to really understand, kind of, the chat history,
or things like a system message,
these models must be taught, so to say,
during the training process,
that certain identifiers in a chat history
mean certain things.
For example, the gemma3 model was trained to understand
that if it detects this start of turn identifier here,
which is written exactly like this,
with <start_of_turn>,
that means that a new message starts.
A user message or an assistant message,
which in gemma's world is called model.
And it was also taught to understand that
end of turn means that a message is over,
and that thereafter a new message might start.
So these are identifiers
that were used during the training process of that model
to teach it to differentiate
between user and assistant models,
and potentially also understand other things.
Though here in the gemma model case,
it's a really simple template.
So that's mostly it.
You also might recall that earlier,
when we used show info,
we saw this weird stop parameter,
which is set to this thing here.
That's just another identifier
that was used, during the training of the gemma model,
to signal the end of a message.
And why is this important?
Because gemma,
because it was trained with those identifiers,
will, for example, insert this identifier
whenever it generally is done with its output,
with the response it generated.
But because it's a token generator,
it might keep on going thereafter
because it doesn't have any logical understanding
of what it means to stop.
It just was trained to have a high likelihood
of inserting, or of outputting, this token here,
after outputting enough other tokens that make sense
for the prompt it received.
So the likelihood of this token being output by the model
simply increases
the more complete the response it gave got.
That all happens due to its training process.
The surrounding application, Ollama in this case,
then is able to capture this token to see it in the output,
and stop the model from producing more tokens.
That's how the model works under the hood.
And that's also the idea behind the template.
This template needs to pick up the structure
that was used during the training process of the model
to steer the model into the right direction,
to wrap the chat history message
into a structure the model knows, in quotes,
from the training process.
That's normally not something you,
as a user of these models, need to worry about.
You can ignore that.
For you it just works.
But under the hood,
we are dealing with dumb token generators,
and that's why something like a template,
or as such a stop token, is important
to steer the model into the right direction,
and to make sure
that it truly understands the chat history,
and that the chat history has the same format, so to say,
as the training data did during the training process.
Because without a properly set up template,
the model will essentially just produce garbage,
because it would receive input that's not in line
with the structure it saw in its training data,
and therefore the likelihood of producing tokens,
and words that make sense, would decrease,
and the likelihood of getting bad results would increase.
That's why setting a good template,
that's in line with the training process template,
is important.
Now, the great thing about LM Studio
is that you don't need to worry about that at all.
And for all Ollama, you also don't need to worry,
if you use models from the catalog,
or if you build your own customized versions
based on models from that catalog.
Because for all these models,
the template is already predefined and baked into the model.
But if you were to build your own model from scratch,
because you trained your own model,
and you got the raw parameters,
then you would need to set up such a template
based on the template you used in your training process
to make sure that interacting with that model
works as intended.
So, long story short, you typically don't need to set that,
but I want to explain what this is all about,
and why it might matter in some more advanced use cases.