Maximilian: So what exactly
are open large language models?
What exactly is open about them?
Now to understand that, let's take a step back
and let's look at what exactly defines
a large language model.
And in the end you could say it's two things.
It's the training algorithm, the code that was written
by the people that want to train such a model.
So by the engineers working at OpenAI or Google,
for example, and it's then that code in the end
that influences or controls the training process.
And as the result of that training process,
we get a large language model that's in the end
defined by its model weights or parameters,
which is another common term you might have heard.
In general, you might have heard about large language models
being referred to as 600-billion parameter models
or something like that.
And by the way, if you didn't, that's no problem.
I'll briefly explain it here, because for this course
and in general when working with large language models,
it's important to have at least a rough understanding
of how these models work.
So if you ask a model like the one that's being used
by ChatGPT, which is not open by the way,
but still a large language model, if you ask such a model,
how do LLMs work, or anything else of course,
that input, your text, gets broken down
into smaller pieces called tokens.
A token can be an entire word or a part of a word,
and those models use tokens instead of words
so that we don't have an infinite amount of words
for all those different languages we have over
the entire world, but instead, a finite amount of tokens
that are building blocks of different words.
But a token can also be an entire word.
These tokens are then represented numerically
by assigning a unique token ID to every token.
And it's then these token IDs
that are fed into the trained large language model.
That large language model is essentially a massive,
complex artificial neural network made up
of interconnected layers and billions of nodes
and connections that connect these different nodes.
And every connection has a weight associated to it
or a parameter, as we also call it.
And it's these different weights
that link all these different nodes that in the end
influence which output token or token ID
will be produced by the model for a given input.
Though it's worth noting that the model
will actually not just produce one token,
but a list of token candidates
where each candidate has a probability assigned to it.
That probability is derived from all those weights
and parameters based on the input token, the input text,
and not just the latest token by the way,
but the entire history of tokens, the entire history of text
that was fed into the large language model.
And then based on those generated probabilities,
one of those candidates is picked.
And if you control the application
that's using the AI model, you can actually configure
how that token should be picked,
so how many tokens should be considered and so on.
And we'll get back to that a little bit later in this course
once we actually start using open models
locally on our system.
So these models are really just statistical models
that generate output tokens and assign probabilities
to the different output tokens
based on the input you fed into them.
And they produce token after token
after token, so they're token generators,
but they generate those tokens based on all
the previously generated tokens and your input text,
and that's how they're able to generate text
that makes sense, where the different tokens and words
and sentences are connected to each other.
That's in a nutshell how large language models work.
And by the way, if you wanna learn way more
about how they work and what you can do with them
and about prompt engineering and all that,
my dedicated generative AI course
is of course the way to go.
Here in this course we won't dive deeply into that,
but it is important to have this general understanding
of what's going on here.
So we have these model weights
or parameters that describe the connections
between these different nodes in this neural network.
And it's these parameters that influence which output
will be generated for a given input.
That's the result of this training process,
this huge list of weights and parameters
that link the different nodes.
And when we talk about open large language models,
it's typically these weights or parameters
that are publicly released under a specific
license that controls how you are allowed
to use these weights or parameters.
And I'll get back to that license part a little bit later,
but that's important.
These open large language models typically
are not about their training code being open.
That's typically not the case.
Instead, it's the result of the training process,
these model parameters.
That's the part that's open.
And that's of course the important part when it comes
to running these models locally,
because you typically don't wanna train them locally.
You would need the training code for that,
but you wanna run them locally.
And for that, you just need these parameters
as you'll see throughout this course.
And that's also the difference compared to closed
or proprietary models like the ones
that are being used by ChatGPT.
There, you don't get access to the model parameters
or weights, so you neither have access
to the code that was used for training
nor to the result of that training process.
All you can use is the application, ChatGPT,
OpenAI built for you,
or you can use the model through OpenAI's API,
but you cannot download those weights and run them locally
or on your own server, for example.