Okay everybody, today is a purple day and that means it's a core learning day.
And there's a lot of core for us to do.
Now the the, the observant amongst you may have noticed that yesterday was a yellow day, but accidentally
I made the slides with a purple strip in the in the front and the back page when it should have been
a yellow strip.
And when I realized I'd done that at the end, I.
I almost went back and rerecorded.
But something told me that I'm probably the only one that notices or cares about the color of this strip.
But if you if you notice and you cared about the fact that I got the wrong color strip yesterday, then
many apologies.
I won't make that mistake again.
And I certainly haven't made that mistake today.
It's a purple day.
We've got core to go through.
Let's get to it.
So today I'm going to demystify and fully explain one of the hottest topics in generative AI, which
is rag retrieval, augmented generation, and then also the the new juicy twist on it in the form of
a rag.
I will explain all of this if you already know about rag and rag, then you can put me on two X and
zoom through this because it might be old hat, but for some of you this is going to be really interesting.
It's a technical topic, but I'm not going to go deep on the technical side of it.
I'm going to give you enough so you've got the intuition to know how to put this into practice to get
better commercial outcomes.
But first, before all of that, just a quick recap about APIs.
Okay, so last time we got in the weeds about the terminology around APIs, and I warned you that I
might review it a few times.
And this is one of those times.
So just to quickly recap some of this terminology stuff.
So I know you understand this now, but but this expression calling an API when you're talking about
like a web API, what you mean is you're making a web request using HTTP.
And the URL that you're hitting is what you call an endpoint, and the sort of way you describe it as
look, I'm making an API call by making an HTTP request to an endpoint to a particular URL.
And then there's this concept of a webhook, which is if you're on the receiving end of an API and you've
got a URL which you want someone to call as a kind of real time trigger to tell you about something,
to tell you that an event has happened, they're going to call your webhook.
Some people call them reverse APIs because it's like it's an API coming into you.
You might think, okay, I can call a few APIs to to do four different things, call four different
APIs.
And I'll have I'll publish some webhooks so that people can call me and tell me something's happened.
Reverse API that that is a webhook.
And so you might say, hey, I've set up a webhook, make an HTTP request here to tell me about this
thing happening.
Or you might say to someone, tell me your webhook that I can call to notify you about something that
is a webhook.
Okay.
And then there's this idea that there are different types of HTTP requests.
It's known as the method.
The most common are get and post, but there are some others and usually use get when you're fetching
stuff which will come back to you in the form of JSON and use post when you're sending information and
I've just added in that little extra tidbit that we discovered yesterday, which is that the post can
contain a body which is some JSON.
And now a post could just be something that some code has been written in some code that calls another
component.
It could just be part of a workflow.
It might have nothing to do with LMS, but the way we used it yesterday is that we built it into some
LMS.
Because we gave an LLM in 11 labs, we provided it with a tool that allowed it to make an HTTP request
to a webhook in order to pass something in, in the body.
What did it pass in?
It passed in a question and in.
And it woke up.
It got alerted.
It got this webhook.
It looked at that question, it passed it on to an LLM, and it gave back the answer after also using
a tool.
And that was our the flow.
And so, so these this this this gives you the gist of what it means to run these kinds of, of endpoints
and this webhook.
And hopefully that gives you a good sense of the big ideas behind APIs.
And now on the subject of big ideas.
Total change in topic.
New blank sheet of paper.
I am going to give you the story of Rag and as I say, apologies.
If you already know about Rag then zoom through this or correct me if you disagree with me or whatever.
Okay, so Rag is something that became massively popular probably a couple of years ago now.
It's, uh, it's, uh, this, this idea that is unbelievably simple and so simple that you've probably
thought of it yourself if you didn't already know it.
But people were working feverishly to give llms more expertise in a particular area.
If you work for a particular working on a particular commercial problem, you want to be able to have
an LLM that is an expert in that commercial area.
And back in the day, we used to do that by training llms more with more data for that area, which
is something that's very expensive and time consuming and needs a lot of data.
And this other idea came up, which is a sort of complementary idea.
You can do it in parallel with, with, with working on training or just doing instead of training.
And it's much easier and it's much cheaper, although perhaps a little bit less powerful.
It's just really a quick and easy and it's called rag.
And at the end of the day, all it's really about is making LMS appear like they have more knowledge,
appear like they they have greater expertise by just shoving extra stuff in the prompt.
That's all it's really about.
When I put it that way, you're probably thinking, what's all the fuss about?
But it's a it's a sort of little, little bunch of tricks and techniques to do that really well, so
that when you're asking a question, the LM has the best possible context to give an answer, as if
it's a total expert in that space.
So, uh, as I say, it's about making an LM more knowledgeable, having more more information, more,
more expertise about an area.
And it's Rag stands for retrieval augmented generation.
Lots of long words for not very much.
And it's basically a clever trick that happens to work really well.
And I often like to tell people that at the end of the day, rag is is a hack.
It's a big old hack.
It's not super ingenious.
There's there's a whole cottage industry of little techniques that are built up around rag.
And so people love to kind of rattle off different advanced techniques with, with long names and make
it sound like it's ingenious and complicated, but it's actually pretty simple and it's very hacky,
and it's very much a kind of trial and error, and it works really well.
That's the sort of that's the thinking behind rag.
And there's I like to say that there is a small idea behind rag and a big idea, and I like to like,
go through them both.
And if you've heard me say this before, I'm sorry.
Uh, but the small idea is just basically saying, hey, can't we just just make an LLM be more knowledgeable
just by adding in stuff to the prompt?
So if we are a travel airline and we're trying to answer questions about traveling somewhere, we could
just shove all sorts of information about our travel tips and our ticket prices to different places,
and we could just shove it all in the prompt.
So the prompt could say, like the user is asking this question, how much does it cost to travel to
Paris?
And here's relevant background information.
What might be relevant in answering the question.
And then shove lots of stuff in there.
And llms are very good when they get this big prompt, when they're predicting the tokens that will
come next, they will predict ones that are consistent with all of this context.
And so if the ticket prices to London are in this context, it is bound to include that in what it thinks
should come next.
And so this is just a very simple idea that leads to better outcomes.
You can try it yourself with ChatGPT you can say how much a ticket price is to London, by the way,
here's some extra stuff and then give it lots of information, including ticket prices to London and
it will do a great job in answering it.
No surprise.
So obviously, the problem with what I've just said there is that it's it's it's fundamentally not scalable.
If if we were a travel agent and ticket prices to London was was the thing being discussed, but we
have ticket prices to every single city in the world, then that's a lot of ticket prices.
And we couldn't take all that information and put all of it in the prompt.
Or maybe we could, but it would be so crammed full of irrelevant information that we'd be really setting
up the Lem to fail, because it's got to do this enormous job of reading through this, this, this,
this essay of information where all we were really after is one small sentence somewhere in the middle
of it with the ticket prices to London.
So the big idea is all about saying, look, is there a way?
Are there some tricks where rather than sending all the data that we've got, we can select a subset
of data, a relevant subset that is most likely going to answer the question.
It might not and we might give too much information.
Maybe only a third of the information is useful, but as long as some useful information is in there,
it's going to be better than the no information at all.
So that's the idea behind rag a trick to get a relevant subset of data.
And it's just like a hacky trick where you can you can tweak lots of lots of different ways of doing
it, lots of settings.
That might mean you send in more information or less information, and you're always trying to find
a nice balance where you're not overwhelming the LM with lots of irrelevant context, but you've got
the right information in there.
And that kind of trade off is is all part of doing Rag well, which many people are experts on after
after practicing for a long time.
And the the big idea behind this is that when you're trying to figure out what subset of the data is
relevant to the question, well, you can actually use an LM to help with that.
Not the same LM necessarily that you're going to ask the question to later, but you can use a different
LM to be trying to figure out, okay, of all of this information, what's most likely to be interesting
for answering the question, how much a ticket prices to London, and how can I find that information
from all of this?
That is the big idea behind Rag.