In this lesson, we are going to talk about two very important topics context and context window.
The context in an LM is the information the user gives to the LM when asking for something.
So any question or request we make as users to ChatGPT, for example, is the context.
For example, we can ask ChatGPT give me a summary of the Bible in less than 100 words.
That is the context.
Or we can tell ChatGPT explain the theory of relativity to a six year old.
That is the context.
Or act as if you were a botany professor and explain photosynthesis to me.
That would be the context.
So all these words are the context that the user gives to the LM.
And thanks to that context, the LM knows how to fine tune its answer.
Okay.
So the context frames the question we ask to the LM.
And the context window is the maximum size of the context.
Okay, think of it as the maximum number of words that the context may have.
In reality, we are not going to measure context in words, but in other terms called tokens.
We will learn about that later.
But think about context window as the maximum size of the context.
We will learn a little bit more about this later.
So in the next lesson we will learn about tokens.