-: Hey there, Eden here.
So today I want to talk about
LangChain's tool calling feature.
And this feature in my opinion is not getting enough hype
and is super-important
because it gives us a lot of flexibility
switching between models that support function calling.
So up until now if we wanted to use LangChain
for function calling, we pretty much had to use OpenAI,
because all the implementation,
for example on the function_calling agent
was specifically tailor-made to the OpenAI's API.
And when other vendors came out
with function calling like Vertex Gemini or Anthropic Sonnet
and they had different API schemas for function calling,
and because the implementation was tailor-made to OpenAI,
then supporting it with other models was quite troublesome.
But now what LangChain did is to level up the field
and to offer one interface for function calling
or tool calling as LangChain calls it.
And basically it supports all the famous models
with function calling, of course, OpenAI, Vertex Gemini,
Mistral, Fireworks, and an Anthropic Sonnet.
And this was something that people have requested
for a while now because they were locked
for using only OpenAI function.
So the interface consists of a couple of things,
a bind function method,
which takes the function that we wrote
and tells the LLM that it may use it,
the tool_calls which is returned from the LLM.
And now when the LLM is returning an answer,
this tool_calls is going to be populated
if there is a function calling invocation.
And the favorite part is the tool_calling_agent.
So up until now we only had
the OpenAI function_calling agent.
Maybe I can show you here in the documentation.
So we had here the OpenAI function_calling agent
and if we wanted to use other models, for example,
Vertex Gemini or Anthropic Sonnet, we couldn't.
So now we can, with this new function
which creates a function_calling agent
regardless of which vendor we are using
and support all vendors who support function calling.
So we can see the example over here,
which is pretty straightforward.
We define some tools, maybe we can define even a function
or simply describe directly the OpenAI format
for function calling.
And when we create LLM, we can bind it with those functions.
Then every time we make an LLM call,
the LLM can decide
if we should invoke those functions or not
and return it in the output.
And let's go to the agent part of this new release.
And right here you can see
that we're creating a function_calling agent,
providing it with the Anthropic Sonnet model.
And this is something that up until now
was super-hard to do.
Let's go to the demo I've prepared.
So the demo is pretty simple.
I defined some tools, the multiply tool,
and I'm using also the Tavily search tool,
which is a way to search online for real time data.
And I'm initializing the tool_calling_agent
with one-time GPT-4 of OpenAI.
And the other time I'm going to initialize it
with Anthropic Sonnet, which now supports function calling.
And I'm going to ask a simple question,
what is the weather right now in Dubai
and to compare it with San Francisco
and I want the output in Celsius.
I'm also tracing everything with LangSmith
so we can examine the results
and see how everything is working.
So let's run it in debug for the first time with OpenAI,
and we got a result, let's examine it.
And we can see that we got the temperature in Dubai
right now is 28 Celsius and in San Francisco, 8.9 Celsius.
Now let's run it with Anthropic Sonnet.
And the goal here is to show you how easy it is
to switch now between the models.
So we got the result, let's go and examine it.
And we got here that the temperature in Dubai
is around 28 Celsius and San Francisco is 8.9 Celsius.
So this is pretty much what we expected.
Now let's go to LangSmith and let's now compare the traces.
So this is the first trace of using OpenAI.
So we can see the first time where the LLM was called
is with a prompt,
and the result is to invoke the tools twice.
One time to send the Tavily search to Dubai
and the other one to search the weather for San Francisco.
So one API call to the function calling
gave us two invocations of tools
and we can see right now the invocation.
So we simply ran the function with Tavily,
and we got there the answers.
You can see it right over here.
And this second time is for San Francisco.
And the final call to the LLM, the second call,
was simply to wrap everything together
and to summarize those results.
We can see this is the output that we saw when we debug.
And now let's go examine what happened
with Anthropic Sonnet.
So we can see here we have three API calls
to Sonnet instead of two.
And the first time the LLM told us
that we need to invoke the Tavily search tool
with the weather in Dubai.
And then we invoked it, we got a result
of what's the weather in Dubai.
And then the agent decided that we need to run it again.
And then we went to the agent
to see if we can return a result
and the agent told us that we needed to invoke another tool,
and this time the Tavily search tool with San Francisco.
So we went and we invoked it.
And then finally,
after we had both results
of the two function calling invocations,
then we can see that we have here the output summary,
and this is what we saw.
And that's it for this demo.
And my goal here was to show you how easy it is to switch
between models now with function calling.
And up until now that was not possible.
And thanks for LangChain for implementing it.
I think it's a huge step in commoditizing machine learning,
and it was requested by a lot of folks
who wanted the flexibility to try out
and work with different models.