Maximilian: Now there is more you can control here.
Most importantly, structured output is interesting.
But before we get there,
I wanna dive into some other important settings
because of course we also got our main app settings here
in the bottom right corner.
Now here you can control the appearance of this app,
but you can also tweak the runtime and hardware settings.
Runtime simply means which underlying tool is being used
for loading and running these models.
And as I mentioned way earlier in the course,
LM Studio like Ollama typically uses llama.cpp.
Now if you are on a macOS device
on a modern Apple Silicon Mac device,
you could also use MLX as an alternative engine.
And you see that I got both engines installed here.
And which engine will be used depends
on which file is loaded, which model file.
In my case here, this is a GGUF file
and therefore it will actually use the llama.cpp runtime.
And I can change this here,
but if I had downloaded an MLX file
and some models exist in this format,
then this MLX runtime would be used.
So that's just nice to know.
You can theoretically also manage the versions of
that runtime if you knew that you wanted to switch
to a very specific version,
but in reality, it's unlikely that you would want
to change something there.
Still interesting to see
that multiple runtimes may exist depending
on your operating system and underlying hardware.
Speaking of hardware, that's also something
that can be inspected here in the settings.
Specifically, if you had multiple GPUs,
you could control which GPUs are used for the model
for running the model.
In my case, I only got my one Apple chip,
which acts as the GPU, so that's the one that is being used.
And if I had disabled that,
then my models would be loaded onto the CPU instead,
which is less efficient than using the GPU.
So you typically wanna enable your GPU
and make sure that the models get executed there.
Here you also see the available VRAM capacity.
As mentioned earlier,
those Apple M1 chips have something called unified memory,
which means almost the entire RAM is VRAM.
You also see that down here,
and of course you will see the values for your system there.
Now this guardrails part is also interesting.
The idea here is
that you can control whether LM Studio allows you
to load models into your memory
that shouldn't be loaded into memory or not.
The default is strict
and that is generally what you should keep enabled.
But if you knew that you wanted
to load a pretty large model into memory
and you knew that it would work
and that you would only use it in a way, that's no problem,
you could set this to off
or more relaxed so that you are able
to squeeze bigger models into your limited resources.
That is something you should do with care though,
because of course you can overload your system here
and potentially crash your system.
So I do recommend sticking to strict here,
but if you had the knowledge
and you had the advanced use case that you needed
to basically go beyond the limits,
you could disable or relax those guardrails.
You can also choose custom to control
how much memory may be used for the model to be loaded.
So that also allows you to go into the other direction.
You might know that of the available memory,
48 gigabytes of VRAM in my case,
you only want to use let's say
10 gigabytes at most for a model.
You could then set a custom guardrail to limit that,
and any model that would require more memory would be
blocked from being loaded.
And that can of course be useful
if you also have other processes running on your system
that need a significant amount of VRAM.
Now here, I'll go back to strict.