Maximilian: So let's see
the fact of these different settings,
and I will start with temperature.
As mentioned, a low temperature value
exaggerates differences,
hence making the more likely candidates
and tokens even more likely.
High temperature values,
on the other hand, flatten differences,
hence leaving to more diverse
and potentially creative output.
So if I open a new chat
and I set my Temperature to 0
to exaggerate all probabilities
to the point where I'm guaranteed
to always pick the most likely candidate
because that is what happens with 0.
You got no randomness going on with 0.
As you also see the tooltip,
0 will yield the same result every time.
So if I set this to 0,
and I then send a message like,
"The sky is blank space,
and I hit Enter,
I get blue most of the time and some text.
Now, I'll copy my prompt,
open a new chat, paste it in,
and I still have Temperature set to 0.
And of course, since that always gives us the same output,
as I mentioned, we get exactly the same output as before.
So if I compare these chats,
we see these options are the same
and all the words are exactly the same.
The only thing that differs is the speed, so to say,
but the text is exactly the same.
Now I'll open yet another chat,
and I'll ramp this up to 1
and send that same prompt.
Now it is a very simple prompt,
so the output is not expected to be totally different,
but it should also not be exactly the same.
And indeed, whilst it may look similar,
you can already see it has more tokens,
so it is longer.
And if you compare it side by side,
or well, one after another, I guess,
you see it is different text.
Like this text here, for example, is different.
It's way shorter here,
and the options are also different.
Here, it's Orange/Red/Pink/Purple.
Here, it's just Orange/Pink/Purple.
And some other text is also different.
So that's the Temperature value.
You can always go back to the default
by clicking this trash bin, the Reset button, which is .1,
so quite predictable with not a lot of randomness.
And you can change this
and also save it to your preset if you want to.
Now for the sampling here we got Top K, Top P, and Min P.
The Repeat Penalty, as the name suggests,
simply controls how much the model is discouraged
from repeating the same token,
and typically, the default setting here is fine.
It is a value I actually never change.
Now if I open a new chat here,
I will try a Top K Sampling of 1.
This extreme value should again ensure
that we always get pretty much the same output
because we always pick the most likely token.
So if I paste in, "The sky is,"
I get some text here,
and if I open a new chat also with Top K set to 1,
but the Temperature's still at .1,
and I sent the same prompt,
you see if I compare them again,
it's the same text.
Now if I use a more realistic
or more common value, like 40,
so that the top 40 tokens are considered,
you will see that we don't necessarily get the same result.
Now since the Temperature is quite low,
it's not unlikely
that we get the same result actually.
As you see here, it is actually the same,
but there could be some differences.
And if I were to ramp up the temperature
as I did before with Top K at two 40,
we will get different results.
Now, Top P Sampling set to .95 simply means
that we consider all candidates
that combined have more than a 95% likelihood,
up to 40 candidates.
If you go above 40, we cut that,
and we only consider the top most 40.
But in general, we include all candidates
that make up 59% in overall likelihood,
and we discard all candidates
that don't have at least 5% probability.
That's what's going on here with that all reset,
but the default was 40 anyways,
and that's, therefore,
how you can configure the output generation
and the likelihoods.
As mentioned, it's probably not something
you're going to be doing all the time,
but if you need to, the setting's there.