-: Okay, let's talk about
how to do vector databases with Pinecone.
So we're gonna install Pinecone OpenAI
and then also this datasets library
because we're gonna load datasets from there.
And this is, basically just following
the typical Pinecone tutorial.
So I'm just gonna really explain
how the code works as we go.
So that's installing that.
We also want to get an OpenAI API key
and gonna need a Pinecone one as well.
Some point.
So run that when that's ready.
Gonna run this as well.
Okay.
All right, so, once you run that,
then you paste the API key up here
and we've got this, okay.
All right, then we're going to just make sure
that OpenAI is working.
Here's a query I created earlier.
So who's the 12th person on the moon and when did they land?
So we're just gonna get a response.
Chat, completions, create.
That's how you get a response.
I'm just gonna say the model and the messages.
And then, the messages would be,
say the user role of the query in here.
Right, so that should be everything.
And then, when we get the response,
you need to get the first thing on the list
and it's message content.
All right.
Okay, so the 12th person on the,
walk on the moon was astronaut Gene Cernan.
Or Harrison Schmitt.
Oh, Harrison Schmitt again.
Okay, yeah, so-
Yeah, you can see that it's not consistent
and that's because it didn't have context.
And so it's loosening a little bit
and now it's saying the 13th person
was Harrison Schmitt.
And then I actually said who was the 15th.
So, do you see what I mean?
LOMs tend to hallucinate.
So, if you don't have context, that can cause a big problem.
And what you want to be able to do is search for context.
So let's take, let's take like,
a more realistic query that you might want to actually ask.
I'm just gonna paste this in here.
I've just created a function to able to call OpenAI,
just complete function,
and here's like a question we might have.
What training methods shall you use
for sentence transformers when I only have
a pair of related sentences?
So this might be a user query
if they're asking your documentation and therefore,
if you can provide a good response to that,
then yeah, then that could be a good thing.
So this is the example that you can use.
Now, we need to get an embedding model
in order to get context to put in the prompt.
So just gonna, this is a different type of model
from the normal OpenAI that you might be calling.
I'm gonna show you what this looks like.
So we get embeddings.
We just need to put in.
Okay, so we're gonna send in some texts
so that we're sending in two different texts there.
And then we're gonna give embed model and get the data.
So what that gives us, is this embedding.
Actually two of them, right?
One for each sentence that we put in.
And this is just a big list of numbers, 1,536 numbers.
And what that allows you to do,
is to search based on similarities.
So if the numbers are close together, then you know that,
that topic is similar to what you're searching for.
We've created two vectors here, so you can see that.
Yeah, two, list of two.
And just to kinda show you
that the length of embedding is 1,536 numbers.
So there are different types of embedding,
different providers.
We, I can also kinda show you what the embedding looks like.
So if you look at that there,
then you can see this big list of numbers.
All right, so now we have our embedding,
we want to load the data set.
And this is what the data looks like.
So, it is just little transcripts of these YouTube videos.
And you can see this is like
"Training and Testing an Italian BERT-
Transformers from Scratch."
So what, this is, again, from the Python tutorial,
what we're doing, is we're gonna load
all of those transcripts,
we're gonna join them together, right?
So in this case, you can see that this only
has like a little bit of text
and it starts at zero seconds, ends at nine seconds.
We're gonna join them together, based on two things.
One is the number of sentences to combine, right?
And then the number of sentences
to stride over to create overlap.
What that means is, we're gonna
combine 20 sentences together,
but there's gonna be an overlap of four sentences
between that and the next sentence.
So sentences one through twenty would be in the first chunk
and then sentences 16 through onwards
for the next 20, as to 36.
That's gonna have some overlap between the two.
The reason why you do that
is so that you don't cut off in the middle of a sentence
and we're just gonna append the data there.
And then you'll see, you can see like the,
the text is a lot longer now, right?
So this is like a much bigger chunk of text
and that means it's much more likely
that the answer will be in one of these chunks of texts.
You need to also set up your Pinecone API key.
So you can get that from going into Pinecone, API Keys,
and then this is the code to create an index.
When you run that, then it's gonna, I'm not gonna run it
because I've already created this index,
but it will create an index like this in Pinecone.
So you see, you've got employee handbook
and then you've got, it just kind of sets this up for you.
And you can see I've got the data in here already.
These are the different texts, right?
You can see there and it's got a title, URL,
when it was published, et cetera.
But the way that you load that stuff in,
you run this PC index and describe it, right?
So that, that's all the kind of setup.
Then this takes a long time to run.
So we're not gonna run it for the tutorial,
but this is the batch size,
this is how many embeddings we create and insert at once.
So we're doing a hundred inserts at once,
and it's gonna run through, based on the batch size,
and it's gonna find the text that we need.
It's gonna basically, join them together to get the text.
It's gonna create the embeddings
and it's gonna sleep for five seconds
and then it's gonna put those embeddings into the vectors
and then insert them or upsert them into Pinecone.
You know, tons and tons of these sentences, right?
That's the specific thing you're doing
with this vector database.
Then once you have that, you can search,
so this is how you do a search in Pinecone.
You create the embeddings
for the new thing that you're searching.
So I've just done that through OpenAI again,
and you can see I've just got the query.
And then you just query based on the namespace.
So this is, you know, this is something you get from here,
you can see that here, namespace,
and then you pass in the vector
and then you pass how many results you need.
So in this case, I've got two,
but you know, you can get more.
And you can see that this is actually
giving you a match score,
it tells you how close it is to match,
but also the text as well.
And then with that, as with all RAG,
you wanna be able to stuff that into a prompt
and then let the LLM answer.
So for example, we had information about a moon landings
and then we'd be able to avoid
the issue that we saw earlier.
In this case, we're trying to avoid the issue
of not knowing how to do
a specific type of machine learning, right?
So we're creating the embedding,
then we're passing that into Pinecone, querying,
we're getting the context,
and then we put them into the prompt,
so we say, here's the context, and then we put that in here.
So let's kind of join all those things together,
put the end of the prompt,
and then, if it exceeds the limit,
then reduce the context one by one.
So this is just like a smart thing to do.
If you end up getting too many documents,
you don't want to get past a certain size
of your prompt, then you can do that.
This is less important to do nowadays,
where the prompts have, you know,
quite a lot of context window,
but it used to be really important to be able to do that.
And then when you query that, you're gonna retrieve it
and then you're gonna be able to print that
and what you get, is a much better answer, so you can see,
"If you only have pairs of related sentences,
you could go ahead and try training
or fine tuning your sentence transformer
using Natural Language Inference
with multiple negative ranking loss."
So this answer is from the actual context that we got.
So you can see, "Pairs of related sentences
you can go ahead and try training or fine tuning
using LLI with multiple ranking loss."
So we actually pulled that from the context and therefore,
the LLM was able to answer and we didn't have to
match the search query exactly, right?
Like, search query is, if we look back up here,
search query we're trying to answer is,
"Which training methods should I use
for sentence transformers when I only have
a pair of related sentences?"
So, it's not actually a direct keyword match,
and that's what's really powerful about vector search,
is that you just have to get it partially right
and it's gonna tell you what things in your database
are close enough, and then that's enough
to give you the response.
Cool, so that's how Vector search works with Pinecone.
There are lots of different options.
I recommend using something in production
that's not just local, like Face, for example,
is an open source library that you can use locally,
but it's only really good for prototyping
and if you wanna go to production,
I would use something like Pinecone
or you can use Supabase, which we also have a lecture on.
Yeah, hopefully that introduces you to the topic.
All of the Vector database is the same.
So, which in terms of what they do,
they just have different optimization algorithms.
Some are faster, some are cheaper.
So it is really important for you to check the
up to date details, but like,
the actual concept itself
hasn't really changed since they came out.
So, hopefully, that's useful for you.