And this may be more more theory than you thought you were signing up for.
But this is going to be really useful.
This is going to give you some valuable intuition when we actually get to building agent systems.
Okay.
The third one is a simple one chaining LMS.
Look, you you probably know this one already, but you can write a complicated prompt to an LM, like
come up with a puzzle and then solve it.
And sometimes it's a good thing to do because it gives the LM a lot of flexibility.
We like to use the word autonomy.
It gives the LM the ability to choose to go in different directions, to come up with two puzzles and
solve them both if it wants to.
You know, you're giving it quite a broad remit, but sometimes you want it to always have one puzzle,
one solution.
You want to be very careful to make sure it's a hard puzzle, and you want to frame that well, in which
case you could divide this into two separate LM calls.
The first LM call is just come up with a puzzle, and you'd maybe refine that prompt.
You'd add in some some details, some specifics.
So it gets really good at coming up with a puzzle.
And then when it responds with the puzzle, you can you can build a second prompt to an LM and just
include that puzzle in your second prompt.
You can say, hey, I'd like you to solve a puzzle.
Here is the puzzle.
And then you put the puzzle in there.
Now when you do that, we would draw it graphically like this.
We would show it as if it's like one LM call, calling another LM call.
But of course, what's really going on is we're just making an alarm call, taking the output and using
that in our next prompt.
And so and so that that's how we would show it is these two different workflow steps.
And that's what we would call chaining LMS.
It's kind of obvious, but you use it a lot to put more control over your separate calls and to test
them individually.
Okay.
And the fourth one everyone's favorite is tools.
Tools is something that that feels super magical.
And it is in fact really mundane.
Uh, and again, this may be one of those things that, you know, but there's no harm in emphasizing
this.
Let's suppose we have an LM set up where the user asks a question like, what's the stock price of Google?
And we want to call an LLM with that.
So we've got some code or some software or a product like like an A10 and it's sitting there.
It takes that user question and it's going to make a call to an AI, to a large language model to get
that.
It's going to, to, to make a call to an LLM.
And it's an LLM that has been equipped with tools, as we would say.
It's got it's got these extra powers.
It's been told you have the ability to look up stock prices using this product, this product that we
saw last time called Market Stack.
And it's able to make an API request.
It's able to connect to the internet, make an API request and find out Google's stock price.
And then in its response back to us, it includes it says, hey, the price of Google stock is whatever.
And, uh, it seems magical.
And we know that Llms are responsible for taking an input and generating output tokens based on pattern
matching.
So at what point in all of that did this LLM suddenly have the ability to to say, oh, I'm not going
to generate tokens.
Instead of that, I'm going to connect to an API.
I'm going to connect to the internet.
Go and do something.
Get back the results.
And that's what I'm going to put in my output tokens.
How did that happen?
How did we equip an LLM with these tools?
And of course you already know the answer to this.
I expect it is more mundane.
It's not that GPT has this power to stop generating tokens and connect to the internet instead.
No, no, what really happens is this very much more simple the.
Net code when it prompts the LLM, when it's building this message that's going to GPT that it's going
to try and complete.
It doesn't just say what's the stock price of Google.
What it says is, look, you can do two things.
You can either respond with an answer, tell me the answer to my question, or if you wish, you can
respond and say, I need you to run the following tool on my behalf, and then I will or will call you
a second time with the result of doing that.
The tool you can call is looking up stock prices.
The question from the user is what's the stock price of Google?
And the LM will get that whole input sequence.
We've carefully packaged this input sequence.
And the output it will give will say I want to to find the stock price of Google, please use the tool.
And then you call it a second time with all of that and with the tool and it responds with the price.
So it's just a clever trick.
It's a conjuring trick by by calling an LM and by massaging the input to explain the tools that it can
use, and getting that output and interpreting it, and then doing stuff and calling it a second time,
we're able to give this kind of illusion that an LM is able to run tools, but it all comes down to
clever prompting.
Like so much with Llms, it all comes down to clever prompting.
The use of tools is a is a trick.
And just in case you don't believe me, I always love showing this screenshot here, which I did in
ChatGPT and you can do it to read this prompt.
I just wrote this in ChatGPT.
You are a support agent for an airline.
You answer user questions, just respond.
Only use tool to fetch ticket price for London to retrieve the ticket price for London or for any city
that you choose to name.
Here's the user's question.
I'd like to go to Paris.
How much is a flight and look at what ChatGPT responded to me.
It responded use tool to fetch ticket price for Paris.
It just responds with that and hopefully that crystallizes for you this this trick that is tool calling
right here in ChatGPT.
And you can go and do it to yourself to get that first hand experience.
And then the fifth trick is this idea of an agent loop.
This is this concept that you could have an LLM that is repeatedly called and is able to run tools continuously
until it's finished, and it's achieved its goal and it's done.
And in doing so you get this, this behavior where it appears to go off and do some work to achieve
a goal.
So here's an example.
Supposing we had this prompt to an LLM.
Your task is to find the current value of my portfolio.
You are able to number one retrieve my portfolio.
And number two you can look up share price just by responding.
Use my tool to retrieve portfolio.
Use my tool to look up a share price.
So this is given to it.
And how does it complete?
What are the tokens that it will put as the most likely next tokens?
Well, if it's given this kind of prompt, the most likely thing it would want to do, given that its
task is to find the current value, it's going to say, okay, I'm going to use tool to retrieve portfolio.
Fine.
Makes total sense.
And now what we do is we take that completion and we put it into the prompt.
We put the prompt, the same prompt as before.
We add in use tool to retrieve portfolio.
And then we add in the results of actually doing that.
We go off and do it.
We retrieve the portfolio.
And let's say that the answer is three shares of Google stock.
So we just put that in there in the prompt.
And this is the second prompt.
We send the LM in a loop.
We just keep going.
We send this and it then says okay.
The most likely way to complete this input is use tool to look up share price Google.
And we say fine, we take that.
We go and look up the Google share price and we come up with a new prompt which has use tool to retrieve
portfolio.
Three shares of Google use tool to look up share price of Google share price is $100 or whatever.
And we shove all of that in the prompt.
And this entire thing on that's written here goes in one prompt to the LM.
Every prompt is stateless.
It has the full conversation so far.
And now we continue our loop.
We call it again.
This time it responds.
The value of your portfolio is $300.
That's its final response.
It doesn't call any tools.
It's done.
And there you have it.
That is calling an LM in a loop with tools to achieve a goal that is a genetic AI.