Hey there.
Ethan here.
Hope you're enjoying this section.
Learning about rag retrieval.
Augmentation generation.
So in this video I want to discuss and explore a bit the link chain documentation.
Now I want to start by saying that I love link chain.
I love the ecosystem and link graph and everything around it.
However, I do have some criticism to link chain and how they manifested their documentation.
So I've been with link chain since the very, very beginning.
I've seen all the iterations.
I've seen it how it evolved from from really the very, very beginning.
And they've made tons and a lot of changes to the documentation.
And specifically, after version 1.0, they removed tons of documentation and things that I think are
super, super important and are still in the source code and are not planned to be deprecated.
Not only that, I think their approach they took, for example, for direct documentation, which I'm
going to show you here, is wrong.
And I don't really believe that's the best practice.
And this is why what I teach in this section is a bit different than what you see in the documentation,
and I want to show you it.
So if I go to the documentation here, if you go under the tutorials, you have a bunch of documentation,
you have a semantic search.
So here you can see you have a tutorial on building a semantic search engine with link chain.
This is pretty much what we did in the previous video where we discussed about the ingestion and the
retrieval.
Right.
So you can see here you have the concepts of documents, text, feature embeddings, vector stores,
retrievers.
We've discussed all of them.
Right.
So in the examples here, here is that example for example of how to load a document.
So here you can see how you can load a PDF document.
Here we are printing the page content.
Those are the things we did.
They discuss about splitting here about embedding.
About what is the embedding output.
So things that we've already discussed and how we can go and add the embeddings into the vector store
with the add documents function.
But we don't really see an application.
So we get here only snippets.
And here in this example you can see that the taking a retrieval function which is going to use the
vector store similarity search.
And they wrap it as a chain.
So I've never seen this syntax before which is actually quite surprising to see it like this.
Here you can see some examples of them using the retriever and then invoking it here it's a batch invoke.
So this is the first thing.
So let's go to build a Rag application.
And then it leads us to build a Rag agent link chain.
And here they actually use a react agent with a searching tool a similarity search tool.
Right.
And first in this tutorial they show you how to build a Rag agent, which is a react agent with a search
tool to perform similarity search.
Then they show a two step rag gene, which is very similar to what we did in the course.
But there's a caveat here.
So let me show you.
So we discussed about indexing about retrieval and generation.
So we really know all about it.
And basically what they did here.
So here they are discussing indexing which we discussed already splitting the documents.
All right here let's talk about the retrieval and generation.
So this is the interesting part here.
So in their example they write a function which they call it retrieve context which is going to receive
a query and it's going to retrieve.
And it's going to return the retrieve docs.
And they take these functions and they wrap it as a tool with some structured output.
So in this implementation they create a retrieve context tool.
So here's the tool list.
And they create a react agent with one tool.
And they tell it in the prompt.
You have access to a tool that retrieves context from a blog post.
Use this tool to help answer user queries.
So I don't like this approach at all.
We leave the decision whether to call this tool or not to the agent.
I worked with hundreds of customers and I never seen anything like this in production because we do
not want to leave this to the LLM.
The create agent here is a react agent, and this is something which is pretty autonomous.
So it has all the freedom to do what it wants.
And I discussed this in the course.
And if we have an application for example a customer support, then we do not want the agent to go and
answer anything else besides our business logic.
So here we have a lot of room to fail.
And this agent can be very easily be manipulated to do nonsense.
Let's go and continue.
Here you can see, by the way, an example of this run where you ask a question with tool calling,
it decides what to call.
It calls the retrieval tool.
And this is actually very redundant and expensive and costly when it comes to tokens.
And it adds the latency because, for example, if we have a customer support agent on our application
which knows our business logic, we don't really need tool calling here.
This is just adding overhead, because in this case, we always want to go and query our knowledge base,
which is the vector store in this case.
So doing it with tool calls is actually quite redundant here.
And now what I like about this doc is the fact that they actually discuss this trade off here.
So here they're mentioning the benefits and drawbacks when using this.
So in the above Agentic rag.
So we used our react agent with a search tool formulation.
We allow the LLM to use its discretion.
So this is what I told you which was problematic in generating a tool call to help to answer the user
queries.
This is good general purpose solution but comes with trade offs.
Okay, let's see the trade offs.
Okay let's start with the reduce control.
The LM may skip searches when they are actually needed or issue extra searches when unnecessary.
And this is a really big problem because we are using the react agent here.
And here we have two inference calls.
When a search is performed, it requires one call to generate the query and another to produce the final
response here.
So this is something that adds latency.
Why they say it's good search only when needed.
The LM can handle greetings, follow ups and simple queries without triggering unnecessary searches.
So this is true, but it can also answer things that are non-relevant and can actually cause problems
to the company.
If somebody wants to try to jailbreak the agent, you don't really want to allow it.
Second contextual search.
And here they say contextual search queries by treating a search as a tool with a query input the alien
craft its own queries that incorporates conversational context.
This is semi-true.
So this is really an advantage because it's using function calling under the hood.
So this query that is going to be embedded and going to be performing the similarity search on.
So it's going to be changing constantly according to the user input according to the user input and
the conversation history.
So this is a smart move.
But we can actually implement this with an expression language and make this deterministic as well.
And here we have multiple searches allowed.
The LLM can execute several searches in support for a single user query.
So later in the course I talk about real Agentic Rag, which is based on Landgraf, which is going to
be based on research papers, which is actually going to do something much better than this, and it's
going to be much more deterministic.
All right.
Then they discuss another common approach is a two step chain in which we always run a search, potentially
using raw user query.
This is what we did in the course and incorporate the results as context for a single query.
This is exactly what we did.
This results in a single inference called per query.
Buying reduced latency at the expense of flexibility.
Because it's fixed, it's always doing.
It's always retrieving the documents then making an LM call.
In this approach, we no longer call the model in a loop, but instead make just a single pass.
We can implement this chain by removing the tools from the agent, and instead incorporating the retrieval.
Step into a custom prompt and hear.
What they did is to create an agent without any tools, and then injected this functionality to do the
similarity search to get the relevant documents inside the middleware so it can work.
But it's going like this and they create agent is in fact running in a loop.
So I do not like this solution because when you're going to use the create agent, in fact you do not
know what's going under the hood.
You can know if you'll dive into the source code and you'll analyze it.
But those things are changing.
So if you update the package, this can suddenly change and break your application.
So this is not explicit here.
And this is way, way too abstracted.
And I think in order to implement something which is robust and it's production ready, you really need
to have control on everything.
So this was ragged long chain.
Now they have here a section custom rack agent under Landgraf.
And this tutorial is excellent.
And I actually go and show this architecture in this course.
This architecture is based on papers.
This one is proved to work well.
It has a lot of things like checking hallucinations, checking answers which are relevant to the question.
And we are going to cover this really, really in depth in the course in the graph section.
So we're going to cover it later.
So I like this one.
So this is it for this video.
We'd love to get your feedback on it.