-: All right, let's talk vector databases.
So, this is how to do information retrievals,
so you can make sure that your prompt
has all the information that you need to complete the query.
So, let's just install a couple of things.
There's two here.
One is OpenAI, which you need in order to do the query,
and also to get the embeddings.
I'll explain what that is in a second.
And then there's also FAISS, F-A-I-S-S.
That is, very specifically, a open source vector database
that Facebook does for you.
So, this is by Facebook, the actual AI lab at Facebook.
All right, how does this work?
I'm just going to bring this up.
All right, oh, I'm sorry.
We need to get OpenAI.
And we need to initialize the client.
And to pass in
api_key.
So, in order to get that,
you just go to the API Keys part of your dashboard.
When you run this, you can paste it in here and Submit.
And now we have the secret_key
and then we can create the OpenAI client.
All right, so, now we have that,
we wanna set the model that we're using.
Just gonna use quite a simple model.
The nice thing about vector queries
is that they tend to make it
so that you don't need a really good model,
in order to answer the question.
Because most of the time the answer
is actually in the text that you're retrieving.
The way that we find the text that we want
is we get a vector embedding.
And a vector embedding, I'll just show you what that means,
is just this long string of numbers.
And you can get this through the OpenAI API,
but there are all other sources as well.
So, just gonna put the input is the text.
And
model,
this is the vector embedding model, not
the LLM that we're using, all right?
So, it's a different type of model.
And then we're gonna get the embeddings
and that's just from response.data.embedding.
And then we're just gonna return
that, okay, so now we have a function
that we can put in some text.
And your text
goes here and we should get a big long string
of numbers outta the back.
Here we go. This is quite a few different numbers, right?
Hundreds of numbers and each one responds
to a specific dimension.
And essentially what you can think
of this as is like an address in space
and the space of all possible concepts.
Then things that are similar
to each other will be close together.
They'll have similar numbers, right?
So they're, you think of like living next door to concepts
that are similar to each other.
So, the concept of king will be quite close
to the concept of queen.
So, what we can do, we don't need to worry too much
about how these things work, but we can use them in order
to find things that are similar in our database
to what the user has asked.
All right, what I'm gonna do is I'm gonna paste
in this function here, which helps us chunk up our text,
helps us split our text up into different sections.
And this is just gonna split it into sentences specifically.
I copied this from somewhere else,
but I'm just gonna split the whole text up
into individual sentences.
And so, I'm just gonna paste the text in,
stuff about Batman.
So, here's a big long article about Batman,
all the different Batmans,
how they've changed over the years.
So, if we wanted to make question and answer thing
about Batman, you can see here, we split,
we could put a lot more information here,
but we split that page up into 10 different chunks.
And that's typical, by the way, for vector databases
because you wanna be able to pull back,
like this is what's gonna go into the prompt.
You want to be able to pull back,
either it's like an individual chunk,
like just a small sentence
or it could be like a whole page.
You need to decide how much
information the prompt's gonna get.
You don't wanna stuff the whole thing in the prompt
because it just costs you money for no reason.
It might confuse the (indistinct),
but you also don't want like too small of a sentence.
Like if you just had this bit from Adam West
to Robert Pattinson, it doesn't have any information here.
So, that's how you think about chunking
and it really depends on your use case, right?
We're gonna import numpy as np
and then we're gonna import faiss.
And what we need to do
is create our
vector database.
So, I'm just gonna get the vector embeddings for the chunks.
So, this is getting vector embeddings
for each chunk, for all the chunk
and chunks and then it's gonna turn that into an array,
a NumPy array.
And you can see what shape that is here.
That this is basically like a list now
of the different things in our array.
Okay, so it's 1,500 numbers for each one
and there's 10 of them, if that makes sense?
All right, so let's create our database
and this just creates it locally quite useful to work with
and we just need to create an IndexFlatL2.
Don't worry about what that means,
but that's just gonna get the specific shape there
and then say,
so that tells it basically we want 1,536 dimensions
and then we're just gonna add the vectors to the index
and hopefully that works.
Now, once we have everything added to the index
and we could add more stuff if we want,
we can now query that index.
So, we can say we're gonna make a function for vector search
and we just wanna pass in the query text
and how many results we want back.
We just call it K.
And then we're gonna say, query_vector.
We're gonna get the vector embeddings for the user's prompt.
So, what did they ask?
Like what's the address in space for the thing they asked?
And then we're gonna find the distances, the indices
for index search.
And then we just gonna do,
we're gonna, again, get this NumPy array
and it just needs to pass in an array here,
query_vector
and K is what we're bringing back
and that's just what FAISS needs.
So, lemme just scroll down here
and then what we're gonna get back is,
I'm just gonna copy this across from previous
piece of code because it's a little bit complicated
but I'll explain it.
So, we're just gonna get the chunks from the vector database
and so it's just gonna get like a user ID
that we get from the vector database to find out
what chunk is the right chunk
and then it's gonna zip the distances
together and the indices.
It's gonna make sense once you see it.
So, now we have a way to search the vector databases
for things that are similar.
Gonna do search_results = vector_search
we're just gonna say ("robert_pattinson")
who was the latest Batman.
So, lemme see.
Print the search results.
We just say for results in search_results
print(result) 'cause if it's gonna come back
with three results, if you remember?
K equals three. So, let's see if this works. Here we go.
So, we've got one chunk here.
Just make this a little bit easier to read.
Yeah, so we've got one chunk here
and then you can see that it does talk about Pattinson.
This one here, I think.
Yeah, so Nirvana's "Something in the Way,"
that's from the latest film.
Yeah, here we go.
So, you can see it's all over this chunk
and therefore it was very similar
to the word "Robert Pattinson."
Similar here, this is the second chunk to came back with
and Robert Pattinsons' all over it.
And then same here, he's all over it as well.
So, it's brought back the three most relevant
things to Robert Pattinson.
You can also see that the similarity is quite high.
1.27 in this case.
And this one's 1.3.
So, these are the ones that were the closest in terms,
so like more is worse, right?
Those are the three closest in concept.
So, now that we have that function,
we can do something called search and chat
or RAG, retrieval-augmented generation.
And that just means doing one of these vector searches
to query the database before we
answer the user, right?
So, this is a way to bring your documents
into the actual kind of chatbot that you have.
And this is what all the kind of chatbot-type companies do.
It's a vector search finds similar documents
or maybe past memories, whatever it is
to actually supplement prompt.
So, we're gonna get the search results,
we're gonna do a vector search first.
So, we've got our vector search function,
just gonna put the chat prompt in there.
And then K, so we'll get our search results
and then maybe let's just print those
so you can see what we go back.
And then we wanna prompt,
I'm just gonna copy this prompt across.
So, this is taking the prompt
and just passing in the context.
So, passing the search results
and then say, answer the question.
Okay, so that's useful.
And then we're gonna send a check completion there.
We'll just say, "Please answer the questions
prompted by the user, using the context provided to you."
If you don't know the answer, say, "I don't know."
So, that just stops it from hallucinating
and not answering based on its own context.
And that's quite useful when you're doing a RAG application.
All right, so we're gonna print out the reply
and let's ask a question.
So, we'll do the tryout of a search and chat.
I'm just gonna ask a question like,
"What song was playing
in Robert
Batman."
And let's see what we get back.
So, it did the query quite quickly
and it's got all this information now
and it talks about the songs in here, I think.
Yeah, Nirvana.
Here we go, Nirvana's, "Something in the Way."
So, it's managed to find that in the prompt
and then now it can answer correctly.
So, you can say, "The famous song playing
in Robert Pattinson's, Batman,
is Nirvana's 'Something in the Way.'"
That is how retrieval-augmented generation works
and it's very useful,
once you see this, you can see it everywhere.
So, a couple of different ways you can do this,
I just use this with quite a small data set,
but you can imagine, take this,
and put in all of your documents,
like maybe you wrote a book and you could pull in
all the different sections of that book into the query
so people can ask questions about your book.
You could also make a chatbot with memory.
So, if you save every memory in this vector database,
then you say, "My name is Michael,"
in the first conversation.
Then a couple of months later
you could say, "What was my name again?"
And it would do a vector search on that database
and it could say, "Okay, he asked for my name.
There's another time where he asked for the name,
that came back in the vector search and what did he say?
He said his name was Michael." Super helpful.
You can do RAG on a lot of applications,
but this is hopefully just a simple demo for you.
I wouldn't recommend, by the way, use FAISS in production
because it's quite heavy, resource-wise.
You'd have to host it on your own database.
So, typically I'd tell you
to use a hosted service like Pinecone
or Supabase, a pgvector.