So what is tool calling? Tool calling is a powerful technique where you make the
LLM context aware of real-time data such as databases or APIs. Typically you use
tool calling via a chat interface. So you would have your client application in
one hand and then the LLM on the other side. For your client application you
would send a set of messages together with a tool definition to the LLM. So you
would have your messages here together with your list of tools. The LLM will
look at both your message and the list of tools and it's going to recommend a
tool you should call. From your client application you should call this tool
and then supply the answer back to the LLM. So this tool response will be
interpreted by the LLM and this will either tell you the next tool to call or
it will give you the final answer. In your application you're responsible for
creating the tool definition. So this tool definition includes a couple of
things such as the name of every tool. It also includes a description for the tool.
So this is where you can give additional information about how to use
the tool or when to use it. And it also includes the input parameters needed to
make a tool call. And the tools can be anything. So the tools could be APIs or
databases but it could also be code that you interpret via Code Interpreter. So
let's look at an example. Assume you want to find the weather in Miami. You might
ask the LLM about the temperature in Miami. You also provide a list of tools
and one of these tools is the weather API. The LLM will look at both your
question which is what is the temperature in Miami. It would also look
at the weather API and then based on the tool definition for the weather API it's
going to tell you how to call the weather tool. So in here it's going to
create a tool that you can use right here on this side where you call the API
to collect the weather information. You would then supply the weather
information back to the LLM. So let's say it would be 71 degrees. The LLM will look
at the tool response and then give the final answer which might be something in
the trend of the weather in Miami is pretty nice it's 71 degrees. This has
some downsides. So when you do traditional tool calling where you have
an LLM and a client application you could see the LLM hallucinate. Sometimes
the LLM can also make up incorrect tool calls. That's why I also want to look at
embedded tool calling. We just looked at traditional tool calling but traditional
tool calling has its flaws. As I mentioned the LLM could hallucinate or
create incorrect tool calls. That's why you also want to take embedded tool
calling into account. With embedded tool calling you use a library or framework
to interact with the LLM and your tool definitions. The library would be
somewhere between your application and the large language model. In the library
you would do the tool definition but you would also execute the tool calls. Let's
draw a line between these sections here. So the library will contain your tool
definition. It would also contain the tool execution. So when you send a
message from your application to the large language model it will go through
the library. So your message could still be what is the temperature in Miami. The
library will then append the tool definition and send your message
together with the tools to the LLM. So this will be your message plus your list
of tools. Instead of sending the tool to call to the application or the user it
will be sent to the library which will then do the tool execution. In this way
the library will provide you with the final answer which could be it's 71
degrees in Miami. When you use embedded tool calling the LLM will no longer
hallucinate as the library to help you with the tool calling or the embedded
tool calling is going to take care of the tool execution and will retry the
tool calls in case it's needed. So in this video we looked at both traditional
tool calling and also embedded tool calling where especially embedded tool
calling will help you to prevent hallucination or help you with the
execution of tools which could be APIs databases or code.