Maximilian: So viewing information
about the model is, of course, helpful and interesting,
but changing some settings
with help of /set can be even more interesting.
Now, if you type just /set,
you get a list of things you can set, for example,
you can set parameters like the temperature.
You can also set a system message
for this chat session then, and a couple of other things,
for example, whether it automatically wraps words
if they reach the end of line,
whether you want to enable JSON mode
and force the model to always reply
with data structured as JSON,
which can be helpful if you plan
on using the model response
in some automated workflow or some other application.
And a couple of other things as well.
/set verbose can, for example,
be interesting to get a bit more
behind-the-scenes information
when interacting with the model.
So here I did set verbose mode.
You can always disable it by setting quiet thereafter again.
But with it being set, if you, for example,
ask, What can you do for me?
You'll get a response just as you're used to.
But then at the end of this response,
you get some summary statistics,
some information about how this response was generated.
You, for example, get the number of tokens
that were generated per second,
giving you some useful insights
into the performance at the speed of this model
on your system and some other pieces of information.
Of course, you also might want to set a system message.
And for that, you just type /set system,
and then any message of your choice,
like Always reply in rhymes,
no matter what the user asks you.
So now I did set that system message
and we can see it if I type /show system.
That proves that it has been set.
And then, of course,
I can send any other follow-up message and for example,
I ask it to summarize what you can do for me.
And due to that system message,
it goes ahead and does that in rhymes.
Now, if I quit this session,
which you can always do with /bye,
and I then start a new one by running ollama run
and then the model identifier again,
you will see that this system message
is now no longer set
because the system message,
when setting it with /set system,
is always set for one specific chat session,
not for the model itself.
You can also set a system message for the overall model,
but that's something we'll get back to later.
Now, one other important thing you can adjust
is the parameters of the model.
For that, you can type /set parameter,
and if you hit Enter,
you'll get a list of the parameters you can set.
For example, top_k, top_p, and the temperature.
And these are, of course,
the settings that control how creative
or predictable the model is.
And again, attached, you find a link
to a lecture where I explain these parameters
in greater detail.
But you can, for example, also set the context window here,
the context size.
As I explained before, depending on the model you're using,
there is a maximum context window size
supported by the model,
but that's not necessarily
the context window size that's active.
And why is it not always using the maximum?
Well, because all that space
for all these tokens must be reserved in memory
and therefore, you typically don't wanna
set a huge context window size if you don't need it,
because that will eat up a lot of memory
and potentially may even need more memory
than you got available on your system.
But you can, of course, /set parameter num_ctx
to change the context window size
that is assigned to this chat session
and set it to 10,000.
And now, this session here
would have a context window size of 10,000 tokens.
But, of course, as I just explained,
it therefore also needs more memory space, more VRAM space.
And that's essentially it for the most important settings.
As you see, you can, of course, set more things,
more parameters, and especially top_k,
top_p and temperature are parameters
you might wanna set according to those explanations
I provided in that previous course section
with that lecture you find attached.
But most other things you can set
are typically not things you'll need to change all the time.
For example, the history here,
which you can enable or disable, is not the chat history,
which is what you could think.
Instead, that is just the history of messages you sent,
which you can always recall
by pressing the Up and Down arrow keys.
That is the history that's being saved.
And if you know that certain commands
or messages you send to the model should not be included
in that history, you could set nohistory
before running a command.
So if I now type, Hello,
I'm sending this as a message to the model,
and now, if I quit that model
and I restarted and I pressed the Up arrow key,
I don't get Hello as a suggestion,
but instead, /set nohistory,
which was the last command I executed
before I disabled the history.
So that will disable cross-session command memory,
so to say.
If you don't want Ollama to memorize
certain commands or messages across sessions,
you can set nohistory here.
I did re-enable it though.