Instructor: So there are open Large Language Models
which you can download and run on your system,
and in this course, you will learn how exactly that works
and you will learn about hardware requirements.
And that indeed you can run
many of these models on consumer hardware,
even on low end hardware.
But the question is, why would you want to do that?
Why not just use proprietary large language models?
Why not stick to ChatGPT, Google Gemini, the X AI, Grok AI
or anything like that?
What's the advantage of using open Large Language Models
compared to proprietary ones?
Well, for example,
because they're free to use.
They're open, you can download them
and run them on your system,
and that's exactly what we'll do in this course.
Now you must respect their license,
but private use is pretty much always allowed.
Commercial use is also typically allowed,
and I'll get back to the license part later.
So you can use them for free.
Whereas for proprietary models,
you of course, have to pay
either based on your usage
or a subscription fee,
like it's the case with ChatGPT
and basically all these other AI chat bots.
But of course, you happily pay for these models
because the open models are way worse
than the proprietary ones, right?
The performance is worse, the results are worse.
Well, not really.
If you take a look at benchmarks,
and of course, you should take those with a grain of salt
because models can be optimized for benchmarks.
And of course, the providers of these models
only publish the results they like.
But still, if you take a look at benchmark comparisons
like this one here,
which was published by Google,
you can see that they're Gemma three models,
which are their open large language models
are not much worse
than their best in class proprietary models.
Now this is of course, a bit older
by the time you are watching this,
and actually even by the time I'm recording this,
there are newer models available,
but this is just an example.
The trend of course continues.
These open models are not much worse
than the proprietary ones.
If you ignore the benchmarks
and you instead take something like
the chatbot arena leaderboard,
which is a leaderboard where users like you and me
vote models up and down,
you will see that they open models like DeepSeek,
but all the Google's Gemma 3 model,
which is another open model just to make that really clear,
rank amongst the top ranks of that leaderboard.
So clearly they're not far behind the proprietary models
and they're free.
Because unlike those proprietary models,
these open models or their weights
specifically can be downloaded
and can be ran locally on your machine or on your server.
And therefore you don't just get the free usage
if you ignore hardware cost of course,
but you also get 100% privacy.
The data you send to that model, your prompts,
the output it generates,
any documents or images you might be using in the prompt,
all that stays on your machine, it never leaves it.
And that of course, is a huge advantage
compared to using proprietary large language models.
There. Your prompts or the generated output
might be used for further training.
Even if it's not, it might be locked.
And even if that's not the case,
you still have to send your data
to the servers of AI or Google.
So if you are using a Large Language Model
to analyze sensitive data,
to analyze a confidential document,
you might not wanna do that when using a proprietary model.
You might not even be allowed to do that
if you're working in a company.
So that's a huge advantage of running open models locally.
It stays on your machine.
You have guaranteed privacy.
In addition, you got no vendor lock-in,
you have full control
over that model that's running locally.
For the proprietary models,
if OpenAI or Google or X, doesn't matter,
if they decide that they wanna roll out
a new version of a model,
that they want to change some quotas or rate limits,
there's nothing you can do about that.
If the model suddenly performs worse
than it did yesterday, you're out of luck.
Of course, if you run it locally on your machine,
however, that can't happen,
you control which version of which model is running there.
So you have full control, you got no vendor lock-in.
Kind of related to that point,
these locally running open models,
of course, are offline first.
They are running on your machine after all,
so you got low or almost no latency
when interacting with them,
which is of course particularly interesting
if you maybe build your own internal tools that leverage AI.
You can also use an open model
in your own AI powered applications,
and that is one example use case,
one of many we'll take a look at throughout this course,
how you can use such locally running
open models programmatically.
Of course, with proprietary large language models,
for example, ChatGPT, an internet connection is required.
You can't use them from inside an airplane
or when servers are down.
And that's why open large language models
when being used locally or on your own servers
are great for many use cases.
You can use them as regular chat bots
because as I showed you,
they do perform really well for that.
So you can just use them as a replacement
for ChatGPT or any of these other chat bots.
And therefore you can, for example,
also use them to generate code
or do anything else like that.
But they especially shine when using them
in tools you might be building,
when using them for tasks like
text summarization, data analysis
or content generation with few-shot prompting.
And these are examples we will explore
throughout this course
and I'll show you how to use these open models for that.
And therefore, it's really just the cases
where you need cutting edge performance,
where you need the best in class performance,
where you have to reach
for a proprietary Large Language Model.
So this course is actually also not about
replacing Chat GPT with such a locally running open model,
though you could probably do that for many use cases,
maybe for all use cases
it's about getting the best of both worlds
and it's about leveraging and using
such a open, locally running AI model
for all those use cases where it really shines
and where it can offer a huge advantage.
And we'll explore all that step by step in this course.