Okay, so just to make this real for you, let me talk through a diagram for the small idea behind Rag
giving LMS extra knowledge just by shoving it in the prompt.
So let's say we're in a chat.
This is a chat.
The user is asking us a question.
Maybe they're asking what a ticket price is to London.
So in comes a question and we have N810 or some other, some low code or real code platform that is
responsible for figuring out how to answer that question, presumably using an LLM.
Well, just before we answer that question, we first look up if we've got anything useful in our database
that here I'm calling our knowledge base.
You may remember that word because we used it briefly before with 11 labs, but basically just a database
full of useful knowledge.
And maybe what we could do is we could look for the word London.
We could just sort of scan for the word London in our database, collect everything to do with London,
and then we simply shove all of that in the prompt.
The prompt says the user is asking how much ticket price is to London.
Here's extra information that might be helpful.
And then everything that has the word London in it in in our knowledge base.
And that goes to the Lem.
The Lem predicts next tokens.
And it will be consistent with all of this context.
And as a result, the answer is likely to answer the question how much ticket prices to London?
But of course, you know the problem with this.
The problem is what if the user doesn't ask how much a ticket prices to London, but they say how much
does it cost to go to Heathrow Airport and Heathrow Airport being the airport, one of the larger airports
in London, and the the word Heathrow say isn't as it happens in our knowledge base, so we don't find
anything.
So the Lem has no clue.
It's brittle.
This this small idea is a good idea, but it's brittle and it depends on, like the text in the question.
And it depends on whatever code we write to do that retrieving It's, uh, it's very clunky.
And so the big idea is all about figuring out how to do a kind of fuzzy look in the knowledge base,
a fuzzy search.
It's sometimes called a semantic search, a search for things that are interesting in our knowledge
base.
That's what it's all about.
And that is where we have to take a tiny detour to talk about something called embedding models.
Okay, here we go.
An embedding model.
Now, if this is all sounding technical and too nerdy for you, you're thinking, is this really going
somewhere?
I just want to assure you, this is very commercially relevant and important.
And in any commercial agent system you're going to build, you're going to you're going to make use
of this.
It's going to allow you to do more.
It's going to allow you to accelerate because you've got this expertise.
So so hang on in there with me.
It's worth it.
Okay.
An embedding model is a type of LLM, but it's different to the ones we normally work with.
It's sometimes also called an encoder.
An encoding model.
An embedder.
An embedding model.
A vector embedding model goes by all those different names.
It's the same thing.
It's a special kind of LM.
Most LMS that we work with take text, and they predict the most likely things to come after that text.
That's what they're trained to do.
That's the job of an LM.
But there is this different kind called an embedding model.
And its job is to take an input of text but not output what should come next, but rather output a bunch
of numbers, just come up with some numbers that represent the meaning of that text.
A bunch of numbers to reflect the meaning of it.
It turns language into numbers where those numbers represent the meaning of the text.
It's not not not the same as tokens.
If you the text always gets turned into tokens before it goes into any of these models.
But the thing about these numbers is that, as I will explain in a second, they they are they don't
just represent the words, they represent the meaning of the whole bunch of texts together.
Okay.
So let's let's say more about this.
So the embedding model, its input is some text.
The output is a is a list of numbers.
And when we see a list of numbers we sometimes call it a vector.
And it's a sort of nod to the fact if there were three numbers, if it if it outputted three numbers,
you could think of that as if they are a coordinate in space, like an X and a y and a z coordinate,
a z coordinate of some point in space, like right here.
Uh, and if it were three numbers, it could be that.
Now it's typically not three numbers, it's more like a thousand numbers.
And you could call that and some and, and we often do a point in a thousand dimensional space.
Now that sounds very sci fi and very nerdy.
And it's, it's, it's it's actually not as complicated as it sounds.
It just means there's a thousand numbers that describes this thing, and if there were three, it would
be a point in space.
It's a thousand numbers.
So it's like a point in a thousand dimensional space is what is represented by a thousand numbers.
But you can get fancy and call it a point in a thousand dimensional space.
Or sometimes people say a point in multidimensional space, or you can just say, hey, it's a bunch
of numbers that represents the meaning of this text.
So why is any of this important?
Well, to explain that just for a moment, let's, let's, let's pretend that we're always working in
three dimensional space.
There's always just three numbers.
These this model produces three numbers associated with a different input.
Text gets different inputs.
Three numbers come out.
Here's the thing.
If you give it two paragraphs of text which have a similar meaning, then they will end up generating
points, XYZ numbers that are close to each other.
And what I mean by that is, if you were to look at them in space here.
And these are all the different paragraphs of text.
Then the points that are closest together would represent text which has a similar meaning.
That's what it is.
So points close to each other have a similar meaning in this three dimensional world.
And of course, the experts here will know that it's not exactly being close to each other.
It's a calculation called cosine similarity.
But it's that's a detail that we don't need to worry about right now.
We can just say points that are close to each other represent texts with similar meaning, and that
applies in 3D space, but it also applies in a thousand dimensional space.
You have a thousand numbers and you do the same calculation to say, how similar are these two vectors
of numbers, these two lists of numbers.
And that tells you how close is the meaning of the inputs.
And when I say that the meaning is similar, I'm not saying that it has the same words in it.
I'm literally saying that they represent things with a similar meaning.
So, uh, how much does it cost to go to London would have a similar meaning to what's the ticket price
to Heathrow, even though the words are completely different.
But the model has been trained so that it recognizes the underlying meaning, and those two things end
up in a similar place in 3D space or in a thousand dimensional space.
They end up being close to each other.
That is the point.
That is what an embedding model is able to do, and it's really good at it.
And if you're wondering, how is it so good at it?
It's good at it because it's been trained with lots and lots of data.
It's been trained with tons of data, tons of things, and using clever tricks to give it information
that's similar.
And it's learned over time with lots of data how to associate similar things with similar points in
space.
Okay, so so at this point I want you just to take my word for it.
Let's assume that that you believe me.
There are these things, this type of LM called an encoder or an embedding model.
It's able to take text and give you a set of numbers that you think of as a vector such that if you
find out how close it is to something else, that will tell you how similar it is.
So what?
Well, what can we do with this?
Okay.
Well, what we can do with this is something that people like to call a semantic search.
What we can do with this is it allows us to do a sort of fuzzy search over a big set of data to find
stuff that's similar to the meaning of some text.
And why is that helpful?
Well, that's helpful if someone's got a question like how much a ticket price is to London?
This gives us a trick to look across tons of data and pluck out the data that might be relevant to that
question, whether it's ticket prices to London or the cost to travel to Heathrow or whatever it happens
to be.
It's going to have a similar meaning.
So that data is going to be given a vector which will be close to the vector of the question, and that
is the big idea.