Hey, guys.
Ethan here.
And today we're going to introduce vector databases, embeddings, text splitters and a lot of cool
stuff.
This part is an introduction to vector databases.
So we're going to introduce some new topics.
In this course we're going to introduce embeddings vector stores.
We're going to be using pinecone.
We're going to introduce the retrieval QA chain and some long chain classes like document loader and
text splitter.
So you can notice that we are introducing a lot of topics.
And those are very important topics when it comes for developing an LLM powered application.
And in the rest of this course we'll see why those are so important.
Now let's begin by talking about long chain document loaders.
Now remember in the introduction video we discussed that long chain is super powerful because it helps
us interact with a lot of third parties.
You can connect your Google Drive and read documents.
From there.
You can connect your Or notion notebook and read your notebooks from there.
Or even your file system.
So there are a lot of documents you could plug in and read by link chain that it's able to digest.
So what link chain actually implemented is a lot of wrappers around those third parties.
That makes it super easy to connect to them and retrieve data from them that we can process with our
LLM.
So this data comes in the form of documents.
That's the terminology in long chain.
And a document is simply something that holds text.
So if we have a PowerPoint presentation it's basically represented with text.
Or if we have a text file of course it's text, it's an image, it's a PDF, whatever.
Everything is text.
And the document loader class is the abstraction that is going to help us use this text data.
We're going to load the data into documents, and we're going to work with documents and send them to
the LM.
So that's how it's going to be.
So it doesn't matter which document we have, if it represents something from the notion notebook or
something from our Google Drive, it doesn't really matter because we are working with an abstract thing
that is called a document.
Now, I know this might seem very abstract, but we're going to see the very implementation of this
and understand exactly what it is.
In this session.
So don't worry about it.
I promise you.
Okay.
Remember that annoying error that limited the number of tokens that we can use in the LM?
So this is quite common actually.
And there are a couple of ways of dealing with this.
We'll show one strategy of this and how to implement it in this session and other strategies.
I elaborate in the theoretical section of this course.
So that's introducing us text splitters.
Basically, when we want to deal with long pieces of text, it's necessary to split it up to chunks.
So it may sound simple to chunk it up.
But there is a lot of complexity when it comes to this, because there are a lot of different files
and there are a lot of different approaches, and we want to keep everything semantically related.
So basically the text splitter will help us split the text into chunks.
And if we want to comprise it later, it will help us to reassemble it together.
We'll see an example of this in this course and even in this session.
So this is very, very useful when we're trying to resolve that token limit because we can split our
text to smaller parts.
And then we have some kind of strategies to get around this token limit, but still process that large
amount of data that we want.
So now I want to show you a cool way of solving this in a very, very elegant solution.
The formal name for this method is called retrieval augmentation generation or in short, Rag.
And the gist of it is that we're taking our original prompt, augmenting it with some relevant context.
And that way the LLM is able to answer the original query when it has the correct context, and we'll
even implement this ourselves end to end in this session.
So stay tuned.
It's going to be very, very cool.
Let's say we have a huge file like a couple of gigs that we want our LLM to process, and we want it
to give us answers about.
For example, let's say it's a book and we want to ask questions about this book because this file is
too big.
Of course, we'll hit the token limit because there are way more than 4K tokens in a book.
And let's say that the question that we want to answer, the answer for it resides in a specific part
in the book or over a couple of parts.
But it's very narrow down and it's in very specific places.
So even if we split up the book into lots of chunks and we'll take one chunk or 2 or 3 chunks together
and send them as a context to the LLM to answer the question, then we'll make a lot of redundant API
calls, because the information for our answer is only in specific kind of chunks.
And if we have five chunks, it's okay.
But what about if we have 1 million chunks?
So if we have 1 million chunks, we'll make a lot of redundant API calls, which will cost us a lot
of money.
So what if there was a way to get with some kind of magic?
The relevant chunks that we need that contain the answer, or have a high probability of containing
the answer and only sending those chunks to the LLM.
So that way we only make a couple of API calls or even one, and we save a lot of money and we get the
response a lot faster and we're not doing any redundant work.
So if that was possible to get those relevant chunks then that would be amazing.
It would save a lot of time, a lot of effort and a lot of resources and a lot of money.
Luckily for us, there is a way to do it.
It's elegant and called retrieval.
Augmentation.
Now it's time to introduce embeddings.
Basically, text embedding is a classic technique, quite old, but super super useful in the natural
language processing world.
The idea is to create a vector space from the text, such that the distance between the vectors in the
space have a certain meaning.
But wait, what's a vector?
A vector is simply a sequence of numbers.
But what's cool about a vector is that it can represent a more complex object like words, sentences,
images, audio files in a continuous high dimensional space called an embedding.
Now, an embedding model you can think of as a black box which receives objects and those are represented
by text does some things to them that outputs them.
This array of numbers, which represents those objects in the vector space, as far as it concerns us,
we don't care about what happens in the embedding model.
All we care about is that text comes in, vectors come out now in good embedding models, takes text
which has similar semantic meaning, and the representing vectors in the vector space are very, very
close together.
You can calculate the distance between each vector to another vector.
Okay, so like in this example, if I have sentence one, which is I want to order an extra large coffee,
I have sentence two.
I'll have a tall coffee and I have sentence three quiero pedir cafe extra grande.
Then in a good embedding, the vectors representing those sentences will be close together in the vector
space or embedding space.
Now it doesn't even matter that those sentences are even not in the same language, because the semantic
meaning here is pretty much the same.
So in their vectors representing them, they'll have a small distance between each other.
Now how do we calculate the distance between the vectors.
It's possible and it's basic math, but it's really, really boring and we don't care about it.
All we need to know is that there are very smart people that did it for us with a lot of optimizations,
and this thing is happening very, very fast.
Now let's take another example that is going to bring us closer to the solution to the problem that
we introduced in the beginning of the video.
Okay.
So we have the text how tall is the Burj Khalifa?
Now the text below is simply a copy paste of the text in Wikipedia describing the Burj Khalifa.
This is the first paragraph.
Now in a good embedding model.
Then if we embed both of those texts, then there will be very similar together in the vector space,
their vectors representing them.
So let's recap a bit about the relevant chunk thingy.
So let's assume for this example that the LLM doesn't really know what is the Burj Khalifa.
It doesn't.
It was not trained on it.
So this information is not available for it and it can't really go online and look for it.
However, if by somehow we are able to get these relevant chunks of data and we'll do that by simply
searching for the vector similar to the query we asked, and it's very easy to do.
In fact, we'll show it very, very soon.
So if we have that, we can simply tell it to the LM.
Hey, I want to know, how tall is the Burj Khalifa?
Here is some information that is going to help you.
Answer me.
Please use this information and answer my question.
Then the LM will have no problem of answering how tall is the Burj Khalifa?
But who will be crazy enough to embed the entire values of Wikipedia?
I mean, that's insane.
That's tons of data.
Well, it turns out there are people who like to do it.
So you can see right here that we have some people that took the values of Wikipedia and embedded them
and represented the paragraphs as vectors.
So now if we want to use it in order to answer questions more accurate, we can do.
So let's say that in the vector space the query that we asked the LLM can be represented by the orange
vector that you see in the square shape.
Cool.
So if there is a way to find its closest neighbors, its closest vectors that have the least amount
of distance to it, then they would provide some very good context.
And if those vectors represent something that it's information and something that is a data source,
then we can take it, send it as the context to the query that we sent to the LLM.
So our prompt will contain our query plus this context information and tell the LLM to use that context
in order to answer the question.
Okay.
So that's the way we really give it those relevant chunks we talked about.
And the vector database is a database that is saving those embeddings, those vectors, and is able
to provide us with no time, the closest vectors to a vector we want.
So it takes the embeddings and it simply persists them and makes it easy for us to use them later.
So now let's recap everything we learned, and let's talk about the example we're trying to solve.
We have the huge file representing the book.
Now this file weighs a lot of gigabytes and it's very large.
We can split it up into chunks and large chain is helping us to do it very, very easily.
So we took the huge file of a couple of gigs, and we split it for thousands or millions of chunks of
text.
Now we can take all of these chunks and embed them using an embedding model and turn them into a vector
that each vector represents the chunk.
So each vector is going to be a list of numbers, which is going to represent that given chunk that
was embedded.
Now we can take those embeddings Settings and save them into a vector database like pinecone, for example.
Now, if you want to ask a question about the book, we can take that question as the query, embed
it into a vector, place it into the vector space where all the embeddings of the book of the chunks
of the book exist.
And now we can calculate what are the closest vectors to the query vector that we embedded.
So those vectors are semantically close to our query vector.
And that's what is representing those relevant chunks we talked about.
So now we can simply send this context of the relevant chunks with our query in our prompt.
Okay.
So we can plug it all in together.
We'll send it to the LLM.
And the final prompt will be what did John do to Alice in the book.
And then we'll send it in the context, the specific chunks that have this answer that have this information.
And then the LLM will be easily able to answer us this question.
And now let's take a deep breath and digest everything we just talked about.
I know it's a lot.
Now don't worry if you didn't understand everything.
It's okay.
It's a lot of information.
I do suggest to watch this video one more time, but don't worry, because we will be implementing everything
we've talked about and see how it's implemented in long chain.
So let's have a recap of what we discussed in this session.
So it was a very theoretical discussion on the following topics.
But don't worry because now we're going to implement everything.
Now I know it sounds scary, but the fact is it's actually quite simple.
And once you see the code, which you can glance at it right now, you'll see that it is very simple
in that long chain is performing a lot of heavy lifting for us.
This is why chain is so, so cool.
So in the next videos, we're going to be implementing all of what we've discussed.
And we'll be putting it to practice into the implementation.
So stay tuned.