Okay, so let's study the llama index framework in more detail in this second lesson.
And this second lesson is going to be a very interesting for you to learn how artificial intelligence
engineers work when we are, uh, meeting a new tool, framework, language, whatever.
Right.
So we are going to do in front of you exactly what we do with whatever framework or new tool or new
language we, we face.
Right?
So we first thing you do is you go to the website of the framework, the tool, uh, the language,
etc..
Right?
So in this website you are going to find two kinds of information.
First what you can call the marketing information, which is the main message that the owners of the
framework wants to send to you.
And this is very interesting.
And we will analyze what is the main message and the different messages that they are sending us.
And the second part you are going to focus your attention in is the documentation.
We will also take a look at that in detail.
So first let's take a look.
So we are going to work with this second notebook.
We we are prepared.
We have prepared for you.
And as you can see in the in the initial part we are loading the dot HTML file.
And then we, we just make this note here saying that the previous name of llama index was GPT index.
We explain the reason why in the in the previous lesson.
Right.
So let's start, uh, analyzing studying this home page.
So every uh, home page, every website home page has what we call a pitch.
The pitch is the main message is the main idea they want you to keep.
In the case of llama index, the pitch is unleashed.
The power of LMS over your data.
Do you remember the previous name of llama index?
GPT index?
This is exactly what that name was about.
Unleash the power of LMS over your data.
So use LMS to search into your data and to use your data in many different forms to integrate with other
tools, etc. etc..
Right?
So down you will see the secondary message is like okay, we use llama index to.
Ingest data.
Two for data indexing and for and as a query interface.
Okay, so boom boom boom.
Main message.
Secondary message okay.
So this gives you the idea.
Do you remember what we said about Lambda index.
They are much more focused than long chain.
From the beginning.
The mind tells you we are about rag technique.
Okay, this is what we understand here.
Then they tell you about their main use cases.
And as I was telling you, even when they provide you these four use cases, than chain is about this
and this.
This is Rag technique okay.
These other things are okay.
Interesting.
But no no no right now Lamar index is about this okay.
So you can see a screenshot of applications using this.
This by the way is second sites.
We will see.
Or the origins of of sacred sites.
You will see this document in the second sites application.
We we we see in our in our next lesson.
Okay.
So.
Then in the home page they tell you about the different types of data you can use with lambda index.
And this is the most frequent data, uh, you are used to what is called structured data.
These are tables, spreadsheets and databases.
Databases are just tables right.
And spreadsheets are just tables.
So Lambda index works with tables.
They work with what is called unstructured data.
So like text, video, etc. and they work with what they call semi-structured data, which is a combination
of the previous two.
And they highlight here like we can work with data from Salesforce, slack, notion, etc..
Okay, so hmm, interesting.
A home page with four main marketing messages.
Pitch.
Secondary message.
A user cases and then a explanation about what they mean when they say data.
Okay.
Now.
Also interesting to take a look at the A.
Navigation the top navigation where you can find the main contents of every website.
Right.
So one of the interesting things you are going to find with Lima Index, as you found with long chain,
is that they are preparing to work with Python developers and with JavaScript developers.
TypeScript, as you know, is just an advanced, uh, well, a type of JavaScript.
Okay, so why is Lima index and long chain working with Python and TypeScript developers and JavaScript
developers?
Because they are in the border between traditional, uh, artificial intelligence engineers, the machine
learning engineers, and the.
Traditional internet application engineers, the full stack engineers.
So the classic artificial intelligence engineers, the ones working in machine learning, deep learning,
etc. they work with Python.
But they used to work in, you know, universities, research projects, science data science projects,
etc. and now with the LM applications, they are starting to work in full stack applications.
So.
This approach here are working with Python.
Have a lambda index.
A Python version of lambda index is addressing this community.
The traditional machine learning engineers.
This one the TypeScript version of Lambda index.
Isa an approach to target the traditional full stack engineers.
The the typical internet application engineers.
They work with JavaScript, and when they approach Elm applications, they want to use JavaScript.
So we will talk more about these two approaches, because our full stack applications are going to be
a combination of Python and JavaScript, but we are not going to, uh, focus our attention on the whole
Python and whole JavaScript frameworks.
We are only going to use and focus our attention in the parts of Python and JavaScript that are going
to be useful for us to build professional Elm applications.
Okay, but interesting to see this navigation here.
Second thing interesting here is what they call the Lima Hub.
So in the Lima hub you are going to find as you as you can see here, the templates of Lima Index.
So templates are ways to help you build faster Lima index applications.
They have a one very interesting template called Create Lima.
Then they have data loaders and agent tools similar to Lang Lang chain very early this very new very
better still.
And then here in templates they have the Sick Insights project.
So here is what you we are going to go there in detail later.
But here is where you can find about this project.
You know the documentation about the project etc. etc..
Okay.
We will go there in the next lessons.
And uh, you also have the Create Lima which is a tool to install Lima index from terminal.
It's very interesting approach, similar to the approach that Lang chain is also taking.
Okay.
So here we we talk about the last features that Lima Index is talking about.
Interesting to see you know the new ideas and projects they are working in okay.
But.
The interesting thing.
Now let's focus our attention on the documentation.
So this is the documentation of the llama index zero 943 version.
This is the page you are going to spend a lot of time in a once you start working uh, in a, in a project
in llama index.
Okay.
So now what we are going to do is a soft landing telling you about the parts you are going to find here
and some very interesting things about, uh, some of these parts.
But when you start working in a llama index project, you are going to focus a lot of time here, especially
in this section, as we will see.
So what do you have here in the notebook?
First, you have a note about the structure of this documentation.
The different sections you may find.
And remember this is a very young framework, which means that it is going to evolve very quickly,
which means that in the next months you may find changes here.
Okay.
Don't worry.
This approach we are we are making is going to be useful for you no matter what content you find whenever
you are studying this program.
So the first, the first thing we do when we approach to a framework is like, okay, what are the sections
of the documentation?
In this case we have a Getting started section, which is the section we will study.
When we are learning about a new framework, then we have use cases, which is interesting.
Once you are starting to work in a project which enters in one of these categories, then a they provide
a section where you can learn, you know, more about Lamar Index after the Getting started section.
And then they have the main part.
The most valuable part of Lamar index is this optimizing section.
This is where you can make your applications from demos to professional.
So this is the section where you are going to invest more of your time.
Most of your time when you are working in a professional application and you have decided if you have
decided to build it in Lamar Index, which is right now a very common, uh, option, because usually
people go to long chain to start learning, but once they want to do something, uh, more professional,
usually launching is is a little bit short for that.
So they go to Lamar Index or to a OpenAI API.
Long chain is working very hard to solve that.
And they are focusing right now in improving, you know, their their level of sophistication and the
their performance and all this area that la mendes has already developed.
So this is the focus of long chain trying to keep up with a llama index in all these areas.
But right now llama index is, in my opinion, the leader for the professional applications in the rag
space.
Okay, so probably you are going to spend a lot of time here once you start building your professional
applications.
But.
Also you have here module guides section, which is uh includes guides to the individual components
of lambda index.
Okay.
So when we approach to this, uh, documentation, we spend more of our time in the getting started
section.
And the first thing we focus our attention in was the starter tutorial.
This is the part, uh, we use to prepare the, the first lesson, the previous lesson.
So this is where you load how to.
This is where you learn how to load a private document to create the vector database, etc., etc..
Okay.
Basics of rack.
Then we went to a a high level concepts okay.
So in the high level concepts this is a very interesting section where the team in Laminex is going
to go deeper into rack concepts and especially rack stages.
This section is super important because as you will see, lambda index is optimizing the rack technique
step by step.
So first they study very, very in detail each of the stages of rack.
And then they focus their attention on optimizing on each of these stages.
So these are very important section.
So they go into detail in each on for each of the stages.
And then they, they they go around some main use cases.
Then we went to customized tutorial which is an initial a approach to optimization.
So they tell you okay we have a basic rack, but then we are starting to uh fine tune it.
So we are working the size of the chunks we are working with.
We are going to work with different vector databases, we are going to use different LMS, etc., etc.
etc..
So this is a this is a way for lambda index to tell you, hey, we are much more than a basic rack platform.
We are going to tell you how to improve the basic rack approach.
Uh, in many ways.
Right?
Then we took a look to the Lambda Index video series.
This is very interesting.
Once you are, you know, into some of these, uh, areas in detail, we took a look at the use cases
section, the same thing.
So this section and this section are going to be interesting for you.
As long as you are working in a project or you want to work in a project related with any of these points.
And finally we went to the understanding section.
This is where uh, you can see the differences between lambda index and long chain.
So you will see that they are using different ways of loading embeddings.
And they are using similar ways for many other things like prompt templates, etc..
Okay, so we have our summary here, but you will probably find some notes that we didn't.
But this is the the analysis we did like.
Okay how are these two different frameworks working with LMS with indexing with la la la la la.
Every time you see vector store, by the way, it's the same that vector database.
So it's different names.
In some cases these two platforms use different names for the same things.
So instead of using chunks uh lambda index use the word node.
So that's a little bit, you know like confusing.
But you will immediately see that they are referring to the same thing.
Okay.
So this is a good section to compare both platforms.
And finally, in this one, you don't have to go, you know, in detail, but it will be enough for
you to understand where you need to return once you start working in a professional application with
lambda index.
Okay, so this is the way we approach a framework like lambda index.
And we wanted to do this with you.
So you can reproduce this approach whenever you find a new a framework yourself.
And also to tell you that your work around Lambda index starts now.
So is now when you need to start practicing repeating some of the exercises we have provided for you.
But most, most, most of all, start practicing and experimenting yourself.
So use what you have here this tutorial.
Use ChatGPT for experiment.
Make mistakes.
You need to learn all these, uh, by memory.
That's not that.
That's not the point.
The point is that you are familiar with the main concepts.
And and that you know where to come back, where to return once you start working in a professional
application.
Okay.
So get familiar with Lama index the same way you got familiar with a long chain.
We are going to work with Lama index in future lessons.
So so we will we will be back to this but for now and go ahead and get comfortable with Lama index.
For some people it's going to be 30 minutes.
For some people it's going to be uh, 48 hours.
Don't worry about that.
Each person has its own rhythm.
Don't worry about that.
Just keep practicing and keep working on this framework until you feel comfortable and, uh, and,
and then go to the next, uh, lesson.
In the next lesson, by the way, we are going to talk about the OpenAI API.
Remember that the typical artificial intelligence engineer they normally choose between these three
options launching Lambda index or the OpenAI API directly.
We have been talking about these, uh, three choices in a previous section.
And uh, right now you are familiar with the first one, with the second one.
In the next blog, we are going to start talking about the third option, which is a very interesting.
It has advantages and disadvantages, as you will see.