Instructor: This course is about
running large language models,
openly available large language models,
specifically locally on our machines, on your laptop,
on my laptop, or of course,
also on that remote server you maybe rented.
And therefore, of course, we need
to explore hardware requirements
and an important concept called quantization
because as you might guess,
running large models could in theory be tricky
if you are not having a supercomputer at home.
But as you'll see in this section,
it's thankfully not really a problem,
and you can run very capable,
large language models on your laptop, your server,
or whichever machine you have.
Now, in order to understand how that's possible
and which hardware requirements you do need to meet,
we'll take a look at model parameters
and their sizes and why that's important.
We'll then derive some hardware requirements
from that information, and we'll then explore
how quantization helps us lower these hardware requirements
without getting a bad performance
or without impacting the performance
of these open models in a negative way.