-: Okay, let's learn about vector databases
using Supabase and pgvector.
So Supabase is a database host
that hosts Postgres for you.
It's one of the most popular ones. A really good solution.
I use it for most of my products,
and it has a integration with pgvector.
Pgvector means you can turn your normal Postgres database,
your SQL database into a vector database.
So something you can query
for retrieval-augmented generation or RAG.
So I'm gonna show you what all that means.
Don't worry about it too much right now,
but you'll be able to see it just a way of being able
to pull information from your database
but based on similarity rather than just on keywords.
We're just loading the environment.
I have a environment file, like a .env file
with the SUPABASE_URL, SUPABASE_KEY in there.
You'll be able to get that from your Supabase project.
So you need to create a Supabase project.
You can usually create one for free
and then you can get started that way.
But now let's go into some
of the code that we need to run.
First, we need to be able to get vectors.
It's a vector database, so we need to be able to get vectors
and those are coming from the OpenAI library.
So gonna create a client
and that's gonna be getting the OpenAI api_key from.
So that's also my .env file.
And then just gonna use the gpt-4.1-mini model,
as well as the text-embedding-small model for embeddings.
Let's call this LLM_MODEL, VECTOR_MODEL.
Gonna be this one.
All right, so how do we get vector embeddings?
Just gonna show you.
So I'm gonna say get_vector_embedding.
Gonna create a function
where we just pass in some text
and we get a response,
which is using the client.embeddings.create.
And then we just need to pass an input, which is the text
and the model, which is the VECTOR_MODEL.
And then we get the embeddings from response.data.
And it's the first thing in the list.
Then embedding.
We're gonna return that.
Okay, so just to show you how that works,
let's just run one.
Query_embedding
and then you can just put in whatever text string you want.
So whatever.
I'm gonna show you what we get back.
We'll show the length and then the actual embedding as well.
Okay, so it's a 1,536 number list
and each number is quite long, right?
Like, it's to a lot of decimal places.
But this represents basically the location
in space for this concept.
So if you put oranges, it's gonna be a different number.
There's still the same number of dimensions,
but concepts that are similar
or close together will be literally close together
in the numbers as well.
So the benefit of using vectors
is when you compare the numbers,
you can actually figure out how close you are to a concept.
That means you can search by similarities.
You can say, you know, oranges are similar to other fruits,
they're gonna be grouped together.
Whereas if I search for cars,
it's not gonna bring back an article about oranges.
So that's the basics of vectors.
But we have now the ability to create vectors,
but we need to go into Supabase
and then create our vector table.
The code we need is,
I'm just gonna put this in markdown, just this.
We need to create a table with items
and then we're gonna have a primary key
and then we're gonna have the vector embeddings as well.
And that we've said that this is 1,536
'cause that's how many numbers we have.
And then we're just gonna store the text.
So when we do a search, we're gonna be able
to bring back the text.
All right, so let me show you where you go to run that.
Okay, so this is Supabase
and you would put that code in here.
So ignore that for a minute.
You would paste it in here and then you would hit run.
Obviously not with this extra stuff.
This is the other code we're going to need as well
to enable the vector database.
And then this is to create a function to be able
to search the vector database as well.
So I'll just go through this quickly.
This creates the table
and we'll be able to see that table in a second.
And then this adds the extension for vector databases.
This creates or replaces.
This is basically like a function
that you'll either create the function
or replace it if it's already found
and it's called match_items.
And this match_items function is gonna query with the vector
and then it's gonna have a match threshold,
like similarity score that you can pass it
and then it's gonna tell you how many results you come back.
So when this creates a function for us to be able to call
with our code.
And what it returns is like a table
with the similarity in the text,
and then we have this is essentially how the search happens.
So it matches the threshold, and that's the SQL.
I've got that in the notebook.
You'll be able to copy and paste that.
You don't need to worry too much about it.
But now we have a table.
And now I'm just gonna actually, let me show you that table.
So that's in here.
You can see items
and I've already added these items in
so you can already see them.
It's from my employee handbook, "Unicorn Enterprises,"
and I have each kind of chapter
of the handbook stored alongside the vectors.
And if you look, you can see this is like
that 156th number list that we'll be able
to search by similarity.
Cool. All right, so we have that.
Now what do we need to do next?
We need to
actually load in the employee handbook.
So I already had it in mind,
but I'm just gonna paste this in.
I just did this to make it simple, right?
We have a few different chapters here and it's split.
So we're gonna split that by two new lines.
So this is going to break it into chunks.
And each of these chunks is then gonna go into the database
once we create it, right?
So if you look, just run that,
we can see that we've got paragraph zero,
paragraph one, paragraph two, et cetera.
So each one of these things could be searched.
So if someone asks about work-life balance,
then it'll return paragraph three,
whereas someone talks about coffee,
then maybe it will bring back paragraph four
because that mentions coffee.
Okay, so now I need a function to create the embeddings
and then upload them into Supabase.
The way that works is first, let's make a list
and we'll say for chunk in chunks
and we're gonna get the embedding
and that's our get_vector_embeddings.
We're just gonna pass in that chunk.
So that gets it from OpenAI.
And then we're gonna add the embeddings_append(embedding).
Right? So that passes that in.
And now we're gonna go through
and we're gonna say for chunk,
embedding in zip(chunks, embeddings).
What that's gonna do is gonna join those two lists together
and then give you both of them in a row.
So we're gonna get the data,
I'm gonna say supabase.table.
And we call our table items.
We just wanna insert the object,
which is gonna be a text,
chunks, embedding.
And just make sure that you use the same names here,
I use the same table name,
use the same column names as well.
Otherwise you'll run into some issue.
So this executes like this.
So that's gonna add everything to the table.
If I do that, then what we'll see,
if I refresh my table,
you can see it's added that same thing again.
Okay, that's where you need to go
and if add this.
You also need to make sure you ran this function as well.
So let me just paste this in here for you.
So like I said, you create the extension,
you actually need to run that before
or just add the extension.
Oh yeah, I added the extension manually in Supabase.
But you can just run this instead.
You need to create your function.
And now we're gonna actually use the function. All right?
So we wanna be able to take query
and we're just gonna ask,
do we get free unicorn rides?
'Cause that's something the employee
might want to know about their job.
So we have the query embedding
and we have get_vector_embeddings,
and we're just gonna embed the text, right?
So that's gonna give us the embedding,
and then we can just do this.
We can get the results from Supabase.
Use rpc, and this is gonna run our match_items function
that we made.
And we need to pass a few different things there.
Sorry, this should be query_text.
And soryy, this should be equals.
All right, so what do we need to pass?
We need to pass the query_embedding, that's one thing.
So that's what we got there.
We need to pass the match_threshold.
So 0.4, let's just say,
or we can change that, see how it looks in your results.
And that's basically how similar the documents have to be
in order to return.
And then this is how many documents you want back.
We can execute that.
I recommend putting it in the try block
so you can handle any errors.
Yeah, so then we're gonna, if it works,
then we're gonna say for results in results.
It's gonna come back in a data attribute.
And then let's just print.
Gonna print out a bunch of stuff here.
So we're gonna print the similarity score
that's gonna come back
and we're gonna print out the text as well.
And then if there's any issues,
then you can print out the error as well.
So let's see if this works.
There we go. It worked.
So we've got two documents and we asked about unicorn rides.
We can see it says unicorn here, it says unicorn there.
It doesn't have to be the word unicorn, right?
Like, it's not a keyword search.
So you could say, do we get free rides
of any mythical beasts?
And because unicorns are quite often mentioned
with mythical beasts, then yeah, here we go.
Unicorn rides.
Could also say are there any animals at work?
And that didn't get any similarity there
because nothing that was similar enough to animals.
So if we drop that similarity down, then you can see, yeah,
we did get petting zoo, unicorn.
We didn't actually say the word animal there,
but it was close enough, similarity score of 0.3,
that it did come back.
And then here we have mythical creature.
We didn't say the word animals,
but animals is similar to mythical creature.
Cool. Let's just say let's go back to our query.
Do we get free unicorn rides?
Cool, so now we have a way to get the documents
that are relevant and we want our LLM to be able
to search based on those documents.
We want it to answer based on those documents, right?
But let's just say we have...
Let's make a retrieve function.
We're gonna pass in the query_text and match_count.
This all the same stuff.
So we're just gonna paste this in.
It's just gonna go through
and just give you all the results.
So we have our retrieval function,
we have a query,
and then we're gonna create another function
called search_and_chat.
Search_and_chat.
And this is gonna use our retrieval, right?
So we're gonna put in our chat_prompt.
A=2, we're getting two documents back.
We can change that if we wanted more documents.
So let's print out the user query,
and then here's where the good stuff happens, right?
We're gonna retrieve the documents first,
and then pump that into the LLM.
So I'm gonna say let's add some print stuff first actually
just so can have a nice look at what it looks like.
Makes sense to you.
And then we're gonna prompt_with_context.
So let's create our prompt,
and we're gonna put the results
of the vector search into the prompt
so that we know we have context from our documents, right?
So this the whole point of RAG, right?
Be able to pull in information into your documents
so that the LLM can make a better decision.
So you can say, answer the question, put the chat_prompt.
So this just separates it by new lines, right?
And then we just pass in our prompts.
So this just messages equals.
I'm just gonna paste this in.
This is like a standard prompt that I use.
So we're saying please answer the questions provided
by the user using the context available to you.
If you don't answer, say no, I don't know.
And then we're gonna pass in this prompt_with_context.
And then in order to call the LLM, the final part,
the most important part,
response = client.chat.completions.create.
We're gonna pass in the model, let's call this gpt-4.
Oh yeah, we actually have model.
I'm gonna we call it LLM_MODEL.
And then let's pass in the messages.
And that should be everything we need.
So let's run through.
So we're gonna search_and_chat.
I'm gonna say, I'm gonna put that query in there.
Do we get free unicorn rides?
And it's going to do a vector search,
find the documents from our employee handbook
that's most relevant to unicorn rides.
Then it's going to put that into the prompt
and then send that prompt to the LLM,
and then get the response and print it out for us.
So let's see if this works.
User query: do we get free unicorn rides?
It retrieved these documents.
And then, okay, so we've got three documents here.
They all have quite high similarity.
And it says yes, you get free unlimited unicorn rides.
You can see that here you get unlimited unicorn rides.
That was in the third document here.
We actually got the right answer here.
And by providing that context to the LLM,
it knows something that it didn't know before.
So that's the really powerful thing.
Now, with RAG applications,
obviously there's a lot of work to do.
Like you wanna really think
about what documents you put in here,
check whether the search is doing a good job or not.
Are you pulling in the relevant context?
And if so, then is it using the context correctly?
So that those are all different things you can test.
But that is basically how RAG works
with pgvector in Supabase.
Now, you can also do this locally in Postgres as well.
I just use Supabase 'cause I prefer it.
But it does have a local version you can run
without paying anything. It does have a free version.
But if you want to,
you can actually just use this with Postgres locally as well
or post it on the server.
So it's a very flexible,
I would say probably the most flexible and scalable result.
There are really good vector databases
that are specifically optimized for vectors,
Qdrant and Chroma, Pinecone and a bunch of others.
But this is just what I use
'cause I'm already in Postgres anyway
and so it just makes sense for me to just add another table
and give out another column,
which is like the vector column.
So I find that works quite well for me.