Eden: Hey there, Eden here,
and in this video we'll explain the motivation
of creating the LangGraph framework.
Now it's going to be theoretical/philosophical
and it's really going to explain why LangGraph was created.
So first I want to start with a big shout out
to the LangChain team
that provided me some slides and some illustrations
so I can use in this video.
Alright, so let's talk about building AI systems
and if we'll take a look on AI systems
from the perspective of levels of autonomy
that those applications have,
then we have a spectrum.
And in that spectrum, on the one end,
we have deterministic code.
And those are systems, where we as developers,
we write the code.
We do not integrate any LLN,
so we only have deterministic code
like the functions and flow that you see right over here.
So we know exactly what is the output,
which is going to be at every step,
what's going to be the input,
which steps we're going to take.
And we basically have control, on the entire system.
And while those systems are very resilient and reliable
because we have control on their entire performance,
however, they're not flexible at all.
And this is because everything is hardcoded.
And if we take a look at the spectrum,
on the other hand, we have this idea of autonomous agents
and those autonomous agents can do everything.
They can make up their task of what to do,
they can write code, execute that code
then to reorder their tasks and to write another code.
And they're very dynamic
and they can really get a task
from the beginning to the end
and they're super flexible.
They can receive prompts like,
make be the number one YouTuber and they supposedly
can actually do this.
However, in reality, those kinds of systems,
they don't really exist.
So projects like RGPT GPT Engineer, BabyAGI,
they were trying to implement something like this.
Now, while those projects are very important to the industry
and I really think they pound on innovation
and pushing the boundaries,
they're not very production oriented
and we don't see any usage
of those kinds of systems in production.
Because there's simply too flexible
and we don't have control
because we rely too much on the LLM.
And when we do that, then the LLM tends to scatter around
and to not output us what we want.
And the reason for this is that LLMs
at the very basic level are simply statistical creatures
guessing one token after the other.
So autonomous agents, while flexible,
they're not very reliable.
All right, so those are the two ends of the spectrum
and let's now talk about what's in between the spectrum.
So one level after writing the terministic code
is to integrate an LLM into that code.
So we as developers, we still write the code
and we control the control flow
and we know exactly what's going to be executed.
But inside of this flow, then the LLM can be used
for example, maybe to summarize something
or to extract the information, to extract some entities.
But overall we have a lot of control over here.
The LLM simply helps us in this process
and controls only one output in this entire flow.
And by only doing this, by integrating an LLM,
we gain a lot of flexibility
because the generation of LLMs, can be very creative
and we still have most of the control
on the development side.
Now if we take this a step further,
we can introduce the concept of chaining.
And this is basically to take one output of an LLM
and giving it as an input to another LLM.
So to compose one call over the other.
And here you can see an example
of a RAG flow, Retrieval-Augmentation Generation
where we give the first LLM our original question.
We then integrate embeddings,
so we take the question, we embed it
and we retrieve with similar relevant documents
which are probably going to help answer that question.
And then we take the original prompt, we augment it
and we send everything to the LLM
which finally generates the answer.
And this is just one example
of chaining multiple LLM calls/using embeddings.
So we have a lot of other use cases
and a lot of other flow chains,
but this is basically one step further
which really gives us the possibility
to build really cool things.
So to summarize, in a chain we have LLMs
that determine the output in a bunch of steps, not only one.
And the more we progress to this autonomous agent
end of the spectrum,
we leverage LLMs to build complex systems.
Alright, so let's now continue
and let's now discuss the concept of an LLM router.
And an LLM router is a kind of chain
which leverages the LLM to decide where do we want to go.
So we can use an LLM to decide
whether we are going to execute code in Branch 1
or code in Branch 2.
So it can be for example to go and search in a database
or to go and search over the web
and the LLM can decide where to go.
So we are using the reasoning power of an LLM here.
Here for the first time, the LLM decides
which steps do we need to take
and this gives us even more flexibility
of building those kinds of systems.
However, you notice here that it's written
that there are no cycles in an LLM router
and you see this dotted line over here.
So what is below this dotted line
is what we consider an agent
or an agentic application.
Cool. So everything you see above this line
is actually very well implemented within LangChain
and I am a very big fan of LangChain
and I think we can build very advanced systems
with only those building blocks
with the LangChain framework.
Alright, back into this theoretical part,
you see this gap here,
right between this autonomous agent and the router.
Then I'll give you a spoiler,
LangGraph is positioned right over here.
There have been a lot of discussions of what is an agent
and what's the formal definition of an agent
and what's an agentic application.
And if you go and ask somebody what's an agent
or what's an agentic application,
you're going to hear from three people
three different answers.
And if you're going to ask them a day later,
you'll get three more different answers.
Now today, in my opinion,
the definition of an agentic system or an agent,
is very soft and there isn't a definitive answer for that.
I do like what Andrew Ng from DeepLearningAI
and Harrison Chase from LangChain wrote on the topic.
But I think they all share a similar consensus
of the core components of an agent.
And in my opinion, if you really want to dummy down
and to look at it at its core level,
then an agent is essentially
a control flow that an LLM decides where to go.
So you can see in this picture over here,
we have an LLM which decides whether to go to step one
or whether to go to step two.
So this is a very basic example of an agent.
We're using the reasoning power of an LLM
to decide where to go in this flow.
And you might ask yourself,
"Hey, what's the difference between an agent and a chain
"and specifically even the router chain."
Because I showed you earlier
and I told you it wasn't an agent.
And the main difference is that a chain
is one directional.
We move from the left to the right, while in an agent,
we actually have those kinds of cycles.
And those cycles we're going to soon see are very important
and they are what's going to give our application
agentic properties.
And specifically agents, at least agents nowadays,
they use function calling, to decide which steps to take.
And function calling is a very cool feature
of certain LLMs, where besides
our query that we send the LLM, the question,
we can send a description of tools.
So those are functions that we can execute in our backend
and we can orchestrate them
and we send the LLM, also the description
of those functions, their arguments, their title,
what they're supposed to do
and what their return value is
and we can do that very easily with the tool decorator.
And then the LLM, if it finds appropriate,
it can tell us that we need to invoke these functions
with those arguments.
So, hopefully we'll go, we'll invoke those function
with those arguments
and we'll get back the answer that we want.
Cool. So now let's talk about the most basic design
for an agent.
And it was first introduced in the ReAct paper
and it really, I think changed our industry.
So basically the algorithm for this type of agent,
is very simple.
So we start, we then use the LLM
to decide whether we need to use a tool.
So for example, to make a call to an API
or a call to a database and query there
and the LLM decides whether we need
to call this tool or not.
Then we go and call this tool with the arguments
that the LLM chose us to execute this tool.
We get an answer and then we feed back everything to the LLM
which can decide whether to use another tool
or whether to return the answer to the user.
So I don't want to dive too deep
into the ReAct algorithm and how does it work?
I do actually do this in my LangChain course
and my LangGraph course,
where we do implement this kind of algorithm
from zero and we really understand where it's coming from
and what is this kind of magic
because it really looks magical
from the first time you see it.
All right. So the ReAct paper, was super innovative
and right away the LangChain team implemented very nicely
within the LangChain framework
and we started seeing those agents pop up.
So those agents were actually very flexible.
So the LLM can decide whether to invoke tool one
or then to invoke tool two, or to invoke tool one
and then tool two, et cetera.
And it has all the permutation that it can wrap.
However, it was too flexible.
And since every permutation is allowed,
then also this permutation is allowed.
And if you've been developing agents for a while,
then you're probably familiar with this error
where the agent is in this sort of end the slope
where it's invoking the same tool,
over and over and over
and simply is stuck.
And there are many reasons for why this can happen.
Maybe we didn't define our tools correctly.
The LLM is non-deterministic or not strong enough.
It chooses the correct tool to use
or it gives it the incorrect arguments
or it even hallucinates a tool that does not exist.
So there are a lot of explanations
of why those things may happen.
And basically what I'm trying to show you here,
is that we have here a problem,
where we have an agent maybe that is flexible
like in RGPT
or even the ReAct algorithm, but it's not very reliable.
And what we want is something which still is flexible
but much more reliable.
So we can actually use it in production systems
and let users interact with it and get good results
which work outside the scope of a demo.
This is exactly, why LangGraph was created.
And you see this gap here
between the autonomous agent and the router.
So here, LangGraph is positioned
and the idea of LangGraph
is to not give the entire freedom to the LLM
rather to scope it and to take this freedom
and take it down by one dimension
and to still allow the LLM to have freedom
but not to give it all the freedom.
And with LangGraph, we represent our software,
our agentic software as a graph with nodes and edges
and we represent it as a state machine,
which can have cycles,
which will give us agentic properties
and it will look like the agent can reason
and it can think about what it needs to do.
And it give us very advanced capabilities,
but we control the flow as developers.
The LLM can play a crucial role in this flow
and decide where do we need to go in this flow.
But we as developers, we decide this flow
and by actually reducing one dimension
of freedom from the LLM,
we can actually gain a lot of reliability
and we can architect our system
such that it would be much more resilient and reliable
and all thanks because we have the entire control
of the flow.
And with LangGraph, we architect our software,
our agentic software as a state machine,
where we have nodes and edges
and this is displayed as a graph,
like you can see right here.
We have nodes and we have edges
and LangGraph gives us a lot of support
for building those kinds of graphs
which are very opinionated
for building agentic applications.
And you might ask yourself,
"Hey, why do we need LangGraph for that?
"I can build it with airflow
"or with network X
"or another graph framework."
And the reason is that LangGraph is very opinionated
towards agentic applications
and it's built to solve that problem.
And it offers a lot of building blocks,
like controllability, running nodes in parallel
and having conditional branching with the LLMs
and having persistence which is built in.
So we can store our current state of the graph
and what's being executed and what has been executed,
which helps us to implement very easily,
human in the loop flows
where we integrate human feedback
which calibrates our execution of our agent.
Time traveling, which is to replay some step
that didn't work correctly
and even debugging and tooling for tracing
because it comes integrated with LangSmith out of the box.
And by the way, you can write inside LangGraph
any code you want.
So it doesn't have to be LangChain code,
but you really can choose anything you want.
By the way, I think one of the motivations
for architecting the software as graphs
is because, most of the papers on agentic applications
and agentic behavior were illustrated as graphs.
So it also feels very natural
to describe those solutions as graphs
and it is very readable and very easy to maintain
and to test and to monitor.
So basically in LangGraph, we as developers,
we control the flow and we write what is the flow
and we can integrate an LLM to decide where to go
and what to execute in this flow and with cycles,
this is important.
And we represent this flow
with nodes and with edges.
So the LLM can use conditional branching
to decide where to go
and maybe to execute Node 1 or Node 2.
This is our state machine
and because we have a state machine, we need to have a state
and the state is something which is shared across the nodes
and across the edges.
And it's going to save all of our intermediate results
and it's going to give us useful information
and to the LLM, useful information
to decide where to go in that flow.