And so of course, this is the moment I bring it together.
I give you the new diagram and everything I hope will now click into place for you.
We've got a user who's chatting with us.
They ask a question, maybe they ask how much does it cost to travel to Heathrow?
Something like that.
That isn't going to directly match with our database that comes to our code, whether it's our software
or if it's n810.
Okay.
What are we going to do?
The first thing we're going to do is take that question, and we're going to to call an embedding model
and say, can you turn this question into a set of numbers into a vector?
Sometimes people call this vectorize, which again sounds very sci fi.
So we're going to call an embedding model with the question and have it turn it into a set of numbers,
a vector.
Okay.
And then what I think you know what.
We then look in our database and we find information that might be relevant.
And how do we do that.
We've taken all of our database and we've turned everything into vectors.
We've got vector versions of everything.
And we say, here's the question how much does it cost to go to Heathrow?
What?
What in our database has vectors that is close to this because that's likely to be relevant.
It might not be.
It might be about something different to do with Heathrow.
It might be might be not to do with ticket prices at all.
But hopefully some data will be relevant.
So let's gather like like things that are within a certain distance or maybe the ten closest things
take them and that is going to be relevant.
And when you find them, you look up the original content, the text associated with them, not the
vectors, but the text that got the vectors that ended up here.
And you take all of that text together and what do you do with it?
You shove it in the prompt that you say the user is asking, how much does it cost to go to Heathrow?
Here's some content that might be relevant, and then lots of text that you've plucked out because it
was close in vector space.
And the LLM gives back an answer and it's the right answer and you're happy.
And that is the big idea behind rag.
And hopefully at this point that I feel like I hope I've demystified it for you.
It's connected.
You don't need to know all of the gory details, but it's good to get the general idea and to say more
about it.
And I'm probably telling you things you already know.
The the database at the bottom there.
If it's a database which allows you to look up information based on its vector and it's able to to quickly
query for things close to a vector, then it's often known as a vector database.
And back in the day, they used to be just a small number of highly specialized databases that were
good at this.
But nowadays it's so popular that all of the mainstream databases support having vectors in them for
looking up data.
So so databases like Postgres and like MongoDB, they all allow you to do this to have to have vectors
and to query based on vector similarity as it's called.
And so that's that's one thing.
Another thing to mention is some people get confused about this, this, this whole vector thing, and
they think that the information that ends up getting sent to the LM on the right, that prompt contains
some vectors in there.
So it's like, what are ticket prices to Races to London, by the way.
Here are here's here's the vector.
And here are vectors of similar information.
And then lots of numbers.
But that LM on the right doesn't know anything about these vectors.
That LM on the right just wants a prompt and it wants to predict the next tokens.
It doesn't want you to send it a bunch of numbers.
It's just expecting that you've used that.
You've selected relevant contexts.
And it doesn't know that you used an embedding LM and all this stuff in the middle.
It's completely unaware of that.
It just knows you've got a prompt with good relevant context.
And so the embedding model has nothing to do with that.
LM on the right.
They are potentially completely unrelated.
They could be from the same family from they could both be made by Google.
But they don't need to be.
They could be completely different.
The only role of that embedding model is to help with this whole fuzzy lookup, this semantic search,
search for data based on its meaning by turning everything into vectors and finding similar vectors
to my other knowledge.
But once we've done that, then vectors are forgotten.
What goes to the LM is text.
Is actual information Relevant content.
You probably knew that already, but it doesn't hurt to emphasize it.
Okay.
And so that's all I'm doing on rag.
And we're going to actually build some rag.
So so you get to use it.
But hopefully you see that the the amazing thing about this is that you can use this to give the sort
of illusion that you've got a model that knows about so much information that has knowledge of your
entire company because it's able just to get the relevant stuff for any question that's asked.
And so the big business use case for this is whenever you want, like an expert knowledge worker that
has expertise about all of the products, your company, about all of the employees at your company,
anything like that.
Then Rag is the go to technique.
So for people that are building like, like HR systems that need to know all of your policies and all
employee information, it could be Rag.
If you're looking to build a knowledge worker that has full expertise of everything about what your
company works on, sounds like rag.
Anything that's to do with that kind of quick expertise and information.
Rag is one of the go to techniques.
So as I said before, there is a whole cottage industry of of extra techniques.
There's there's graph rag, there's hierarchical rag, there's reranking so much.
And if if you feel hungry for this and you're going to be hammering me with questions about it all,
then I should tell you that in my AI engineer core track week five, we spend the whole week on Rag,
including the single most important thing about Rag, which is measurement, measuring it, evaluating
it.
Rag is incredibly hacky and experimental, and the only way to tame rag and know whether you're you
need to make changes or do one technique over another is to measure it, measure the quality of your
retrieval, measure the quality of your responses, and I cover all of that.
Week five of the core track.
Uh, if you're if you're if you're interested, if I've, if you've caught the bug.
But for now, it's time to explore a gentic rag.
The such a hot new topic.
And I wanted to to do it by explaining what's the difference between the kind of traditional rag rag
of the original variety and the new agentic rag?