Hey there.
Eden here.
And in this video we're going to introduce the concept of function calling aka tool calling.
And this is strictly a theoretical video.
We're going to be doing some hands on very very soon.
So I just want to introduce the concepts.
So function calling or tool calling refers to the model's ability to produce a structured function call
to an external function with its arguments.
So instead of just generating plain text it's going to generate a well structured answer which is very
easy to parse.
And it's going to appear in a very special place in the response from the LM instead of the generation
content.
Now, it's important to note that function calling is a capability of certain llms.
Not all llms support function calling.
However, these days this is quite a standard of the state of the art models, so you can pretty much
assume that all the big vendors like OpenAI, like anthropic, like Google, when they release a state
of the art model, it's going to support function calling and function calling or tool calling was introduced
back in 2023 by OpenAI, and developers needed to simply provide with the model a list of function definitions,
which includes the names, the parameters, and the descriptions of the functions, and the large language
model can choose to respond by outputting a JSON object, which is going to specify which function to
call and with what arguments if it needs to.
So behind the scenes, this is a model which is fine tuned to detect when a function should be invoked
based on the user's request, and then formats its response as a valid JSON adhering to the function's
schema.
So, for example, if a user is going to ask the LM, what's the weather in Paris to an LM where we
binded the function of get weather?
Then the LM is going to respond with a JSON which has the following information.
The name is going to be get current weather.
The arguments are going to be with the location of Paris and the unit Fahrenheit or Celsius.
And our application could then take this JSON, parse it, and actually go and execute this get current
weather function which should be existing in our application.
And we can then take this response, plug it in back to the LM and continue and so on.
And what was the actual motivation for the LM vendors to implement into the models function?
Calling capabilities is actually the react prompt, which we already saw in the course.
So you might notice that this is not so reliable.
This react prompt.
It sometimes outputs us some bad output, which is hard to parse.
And then our program fails and we can have a lot of problems using this prompt, even though it's very,
very cool, it's not that reliable.
However, function calling is more reliable and more deterministic.
All the heavy lifting is done by the LM vendor.
So all the reasoning and what we get back is a very parsable JSON object, which is very, very easy
to work with.
By the way, in future videos, I do dive deeper into the difference between function calling and the
react prompt, so I really recommend you checking this out.
Anyways, the two main capabilities that function calling gives us is one to connect our LM to external
tools, but the second one is to get structured output from the LM.
So it's going to leverage the LMS reasoning capabilities to extract information in certain fields and
return it in a very organized JSON, just like we wanted.
So we can then convert it into a Pydantic object and we can downstream to our application.
And this is very reliable.
And I actually covered this as well in the course.
All right.
Let's talk about the advantages of using function calling.
So first and foremost we get structured and reliable integration.
So because the model's output is a machine readable JSON file with a specific function name and arguments,
it's very easy to parse it and it's less prone to misinterpretation.
So opposed to the react prompt.
Now this model underneath the hood has been fine tuned to adhere to the function schema strictly, which
is going to reduce random formatting errors like we're used to with the react prompt.
So this structured approach is clean, it's very efficient, and it's going to enable us with a very
powerful and reliable tool usage.
Now another advantage is that it's very easy on the tokens.
So because function calling doesn't output us all the chain of thought that we saw in previous sections,
and we don't really get this high reasoning intensive prompting.
So the model can skip these verbose explanations and directly and only return the function call.
And the only drawback I can see for function calling, which is totally worth it.
By the way, we have opaque reasoning process.
And when the model decides to call a function, it typically does so without exposing its chain of thought.
So the reasoning remains internal to the LLM.
And we as the developers, we see only the final function, the name, the arguments, but we don't
really see the justification and we don't really understand why it did so.
So this function calling becomes more like a black box decision where we don't have any intermediate
rationale which is exposed.
So this can make debugging and auditing the model's decision harder, because we don't really see why
it chose that particular function with those specific arguments.
However, having said that, de facto function calling now is the standard and nobody really uses the
actual react prompt.
They're using the function calling features of Llms.
And today, we're at a point where LLM vendors like OpenAI, like Google, like anthropic, have really
perfected function calling.
And we get really reliable answers with function calling.
I mean, much more reliable than the react prompt, which is going to enable us to build more robust
AI agents and AI applications.