-: Welcome, in this video we're gonna have a look at
tool calling and agents.
We'll understand what tool or function calling is,
how it works, and also look at building agents.
Tool calling, also known as function calling
enables large language models to interact
with external code or services.
The large language model can decide when to call a function
you define and providing structured arguments.
Basically this bridges the gap
between the language understanding
and actionable code generation.
As you can see on the right here,
the LLM has access to some tools
and it decides when to call it.
There are seven steps in the tool calling flow.
Firstly, you have to define
what tools the large language model has access to.
You often do this in a structured form called JSON schema.
After that, you then send a user message
or an assistant message to the chat model
and with an input and also the tool definitions.
The model will then determine if tools are needed
to be used to answer the query.
And if they are, the model will return the function name
and the structured arguments
that it wants you as the developer to run.
You will then have to be responsible for executing
and running the code and running the functions.
Then you will return the output to the chat model.
The chat model will then produce a final response
and it will incorporate the function results
into a coherent answer.
If you're looking on the right,
you can see the user input comes in here.
You define your tools, you query the model.
The model then analyzes it,
and if there is no tools needed,
it generates a direct response.
Otherwise it will do a function call, execute the function,
return the results to the model,
and then it will generate a final response.
So this is what it looks like when you generate your tools.
You'll have the type of a function, you'll have the name of
get weather, the description, the parameters,
such as a latitude and a longitude,
and also you'll say what parameters are required.
Okay, so we've looked at tool calling.
We are aware that we can equip a language model
with tools to enhance its capabilities.
What is an agent though?
An agent is an autonomous system
that uses an LLM to achieve goals through a cycle
of reasoning about the current state,
planning appropriate actions,
executing those actions via tools,
observing those results and updating and understanding.
Common components you'll find in an agent include tools,
memory, planning.
There's also this concept called the agentic loop,
which is basically a big wild true loop
with a series of things that happen inside of that.
So at the start of that, it will do some reasoning.
It might decide to use some tools,
it will execute those tools,
integrate those into the chat history.
Then it will decide to continue in a loop if it needs to.
Otherwise it will finish with a final answer.
Let's have a look at how that would look in Python.
We start with a Python list of messages,
check the weather in Paris and Berlin.
We enter our agentic loop, we get a model response.
Then we check for all of the responses to outputs
for a tool call.
And if there is a tool call, we execute that function
and else we just respond with an output text and we break.
So when there isn't a function call to happen,
we basically break out of the agentic loop.
There's other advanced agent patterns.
So for example, defining a custom successful criteria,
continuing the loop until the objective is achieved.
An example might be search the whether
in five different cities.
Also, there's key implementation patterns,
so parallel tools.
So tools can be executed in parallel.
You also have the option to choose
does the agent or LLM automatically choose the tools?
Is there a tool required
or a forced selection of certain tools?
Strict mode, ensure function arguments
match the exact schema correctly.
And also you have streaming.
So providing real time progress updates
during function calls.
Some best practices include writing
clear function descriptions and parameter documentation.
Keep the function scheme as simple and intuitive.
Apply software engineering principles,
so least surprise and no invalid states
and prefer fewer, more powerful functions
over many specialized ones.
So the next thing that we're gonna have a look at
is let's get some hands on and practice learning
how to do tool calling.
And then we'll be following that up
by building agentic loops.
I want you to go back into your GitHub repository
and go to the openai features and functionality folder.
Load in and start this notebook called tool calling.
And we're gonna go through
and write a couple of examples in here.
You can either run the code
or feel free to follow along
as we code this up from scratch.
Okay, so the first thing we're gonna do
is we're gonna install OpenAI and Pydantic.
Those are just some packages.
After that, what you're gonna do is you're gonna import
OpenAI from OpenAI,
import JSON and requests,
and we're gonna set the model to GPT-4.1 mini.
You're also going to need to update your API key,
so feel free to do that and then continue.
Now we have a get weather function,
which makes a get request to this open mateo forecast
and it takes two arguments, latitude and longitude.
We are then gonna build this up step by step
to figure out exactly how to do tool calling.
So firstly we have our tools Python list,
and we need to add on all of the tools in here.
We are gonna have one tool
and we're gonna have to type the JSON schema.
So the first thing that we need to do
is we need to create a dictionary.
And in that we're gonna have type is function.
Then we're gonna set the name of our tool
is equal to get_weather.
Then once we've done that,
we have a description of what that tool is.
So this tool will allow you
to easily get the weather.
Then we need to set the parameters of our function.
So we'll say parameters
and we'll do a dictionary type object
and we'll add a comma there, type object.
And then we have properties
and we have an object again,
latitude, which is a type of number
and has a description of latitude of the location.
And we have a second argument or property,
which is the longitude,
which is type of number.
Description is gonna be longitude
of the location.
And then that's technically enough at this point in time.
You can also add on things
like if you require certain parameters.
So you can say both of these,
the latitude and longitude are required parameters.
We then set up our chat messages.
So we've got our input messages
and we're gonna go and fill that out.
So we need to have a role of user.
And the content that we're gonna have here
is gonna be what is the weather like in London?
Okay, great.
So we have our input messages.
The next thing we're gonna need to do is call the API.
So we're gonna do response is equal to
client.responses.create,
and then we're gonna pass in
our model is equal to model
and that was defined earlier.
We're also going to put the input is equal to input messages
and we'll put the tools is equal to tools.
And basically at this point when you have this tool
is equal to tools
that's providing this large language model,
the ability to know which tools it can use.
The next thing we're gonna need to do
is we're gonna need to print the output.
So we will do print get weather response output,
and I'll do response to output.
Now if we just run this, now, if we just run this,
you'll see that basically the response is being generated
and you'll see that it's actually got the output
and it has this response function tool call
and it has the arguments.
So it's actually told us exactly
what function it wants to execute.
So notice it's got a call id,
it's got the name, the type function call,
and it has an ID as well here.
So this ID for the function call ID
and a status of completed.
So the first thing we need to do
is we need to tell the language model,
let's go and get the tool call out from this.
So tool call is equal to response.output [0]
and we will say the args of this is equal to json.loads
tool_call.arguments.
And then we get the results,
we'll say where the result is equal to get_weather
and we'll do args latitude
and args longitude.
And then what we need to do is we need to add
the function messages back into the tool.
So we do input_messages.append tool_call.
So that's the original message that it told us to invoke.
And we also then will put in the fact that we've
invoked that function.
So the first one is to tell it yes,
we're adding on the tool call.
The LLM wants to make this tool call.
The second message append is to tell the LLM,
hey, we actually executed this tool call
and here are the results, right?
That is really important.
So then we're doing a type of function call output.
And then after that we're doing a call id.
And you'll see here we're doing the tool_call.call_id.
And then we do the output
and we're just gonna string json.dumps
the weather_result.
Okay, so now that we've added that extra message history on,
the only thing we also need to do now
is we just need to make a new response to that.
So we'll do client.responses.create,
model is equal to model,
input is equal to input messages.
And we'll also do tools is equal to tools.
And then we can get the response to output from this again.
So we'll do response.output_text.
And let's see what this brings back.
So you've got our weather output
and it said the current temperature in London
is approximately 5.1 degrees Celsius.
So to recap, we created a get weather function,
we created our tool
and we made that using JSON schema,
we set up a input message which had a query
which would naturally invoke one of our tools.
We then added that into the client.responses.create.
Noticing we're adding this tools parameter,
which tells the model that it can use our get weather tool.
We then looked at the output of that
and we saw that it had this response function call tool,
which showed the arguments, the call ID
and the name of the tool.
We then get the output of that,
we load the arguments in from a string,
and then we invoke our tool.
After invoking our tool, we then tell the model,
hey, we know you had that tool call that you wanted to make
and here's the result of that tool call.
Then we then call the model again,
providing it with a model, the input messages and the tools.
And then the model then uses the tool result to tell us,
okay, great, yeah, here's the temperature.
Alright, let's step it up a notch.
So OpenAI also provides prebuilt tools.
So we're gonna use this tools type web search preview,
and we have an input message here,
"What was a positive news story from today?"
And we can just again,
provide the tools to the language model
and provide the input, which is a role of user.
And the content is this input message.
We're also going to, you know,
optionally you could show more specific information
if you want to go and have a look at, you know,
a country or a city or region.
And you can have a look at here.
So when we run these,
it will do a live web search tool call
and it will return the results directly to us.
So it's actually searching the web in real time
using one of OpenAI's prebuilt tools.
And you can see these are the different types of results
it searched and basically this allows you to build
easy web search functionality
directly into your large language model applications.
It's also possible to do streaming with tool calls.
So just like our get weather example you can add on
the stream is equal to true parameter
and you'll get back a lots of different results.
So the response created event,
the response in progress event,
and an output item added event.
And then you'll also, the most important one
is this response completed event
that shows you the entire response.
But you can build this up in real time
with the Delta events.
So just bear in mind that you can also use streaming
if we're doing tool calling so that you don't have to wait
for the entire tool result
to update your users in real time.
Cool, so we looked at how to build tools,
how to define those functions, and also handling responses.
And we had a brief look at the web search tool,
which is a built-in tool by OpenAI.
And also we looked at the fact
that you can naturally use streaming
when you are invoking tools so that you can respond to users
in real time.
In the next video we'll have a look at
how you can build agents from scratch.
Cool, see you in the next one.