Okay.
So this is going to be a very, very interesting lesson.
This one and the and the following ones.
Remember that you are going to have these uh detailed presentation and also this detail notebook.
You will be able to download it.
So don't worry if in some cases the the the size of the letters are too small for you.
Don't worry.
Uh, this presentation is just an initial, uh, uh, round, uh, that you will complete when you look
at the document yourself.
Okay.
So we are going to start with this Lang Lang Smith in depth from zero to advance operations.
We have a long way to go.
And we will start with a the Lang Smith definition.
And this is more important than uh, than it seems because this is what Lang Chain says about the platform.
So this is what the Lang Chain team explains about the Lang Smith platform.
So the creators of the platform explain Lang Smith in her own words, and they say Lang Smith is a unified
DevOps platform.
For developing, collaborating, testing, deploying and monitoring LM applications.
So it is, as you can see, a very ambitious platform.
They want to cover all the steps and they are.
But you can say that this is a very ambitious project and you are seeing this a word here, DevOps,
which probably many of you are familiar with.
Uh, remember that in this program we are using LM ops because LM ops is more.
Like a word that in the LM application space are we used to.
So when you in this context, when you read DevOps you may change DevOps by LM ops, okay.
It's going to be the same for our purposes.
So this is the definition of Lang Smith from the Lang Chain team, a unified DevOps platform for developing,
collaborating, testing, deploying and monitoring applications.
Take a look at these bullet points because, for example, this one deploying is still not at 100%.
Um, so we will you we will see that this right now is like in beta.
We will see why we say that.
But right now it is a very good help, you know, for developing also for collaborating and not just
between developers, also between developers and non-technical people in your team is a, of course,
a good platform for testing the deployment.
The deploying is something that is is going to to be better in the next future, and monitoring is already
there.
So they are using a these two pitch messages in their website.
So you know that a long chain used to have a website and now they have different websites for their
projects.
So they have one website for a long chain.
They have another website for long Smith.
They have another website for long serve.
So in this website is where they are talking about the Lang Smith platform.
And they are using a couple of pitch messages.
And I think it is very interesting to pay attention to what they they say here.
So the first one is get your LM application from prototype to production.
This is super interesting for us because the main goal of this program is precisely to go from the toy
demo LM application to the production to the professional application.
So this immediately got our attention.
And they use this subtitle with this pitch message.
They say all in one platform for every step of the LM power application cycle.
Okay.
So again they are talking about the full development cycle or the full production cycle.
So the second pitch message is Lang Smith turns LM magic into enterprise ready application.
This is very important.
And they explain it here.
No more guessing or development by bypass.
Use testing to confirm that your LM application performs as desired.
So we talk about that in the previous blog.
The long chain team has been observing LM app development teams at work during one year, and one of
the conclusions they made is like a lot of LM app development teams, they they don't do testing.
They go from prototype to production.
They use just gut feeling, you know, a and we need to improve that because if you don't do testing
you are risking the quality of your of your application.
We all know that testing in LM applications is is different from a classical application.
So that's why, you know, people are are not using testing here.
But we have ways to include testing in our process.
And locksmith is a very good help in that direction.
So this is very interesting because what Lang Smith, what the the Lang Chain team is saying here is
that until now a lot of LM projects are magical.
Magical in a sense that they, they, they are not based on on science.
They are not based on testing.
They are just based on gut feeling.
Right?
It's totally understandable because LM, LM applications are still very, very young.
So we are growing.
Right.
But this is I understand this is a very, very interesting, interesting, uh, pitch message from uh,
Lang Smith.
So let's talk a little bit about Lang Smith terminology, because in the previous vlogs, in the previous
vlog you started a listening weird words, right?
We we talk about rants, we talk about data sets, we talk about playground or have, uh, we talk about
regression testing.
So.
Okay, what what does this mean in, in the context of LM app development.
So for us, I think it is very valuable to have this kind of, you know, dictionary.
Right, uh, where you can come back whenever you have a doubt.
So what is a trace traces record is a record of interactions with the LM applications.
They can contain more info beyond just input and output.
So if we go to the Lang Smith platform.
So here you have the Lang Smith platform open.
Here we have some small projects.
You will see one basic project at work in a, in a next uh block.
And then you will see also a professional project at work, which is super interesting because the more
sophisticated is the project, the most value adds uh, the Lang Smith platform.
So you will see that it's going to be very interesting to see the Lang Smith platform with a professional
project.
This is going to be a very interesting for you, but let's take a look at one very simple project.
Uh, so as you as you see here in this, uh, Lang Smith platform, we have a number of projects already
created.
And if we click on one of the projects, these are the traces.
Okay.
So if we click in one of the traces, this is a rag a simple rag application.
We see that one trace is like one.
We can we can call one operation or one one uh, interaction between the uh application and the LM model.
Right.
So here you can see the different steps from the input to the output.
You see the input, you see the output.
You also see the different steps.
And if you pay attention here in this table you can see the latency.
If this trace is included in the test data set.
If we have included some annotations, the number of tokens we have used the cost because this is associated
with with OpenAI.
So the cost is something relevant here, etc..
Right.
But the most interesting thing is to see that traces have several steps.
So you are already familiar with the Rag technique, and you know that in the Rag technique we have
several steps.
So here you have the steps you can investigate uh, in these steps, you know, if if something is uh,
uh is failing in order to find what is happening in each of these steps.
So these steps are called runs.
Okay.
So the LM call is just the call to the LM model, just input and output.
Okay.
So if you come here you see that the LM call is super simple.
So you have input.
An output.
If you go to the traces, you have much more information.
And if you go to the runs, that's a total different story.
The runs are small steps.
Okay.
So if you go to one trace, you can see different runs inside of a trace okay.
So this is important because if you remember in the previous blog we talk about the possibility to include
runs in the test data set.
Okay.
So here you have the difference traces.
Record of interactions with the LM application can contain more info beyond just input and output LM
calls calls to the LM model, input and output.
Run a single execution of the LM application to process an input and generate an output annotation queues
to add human labels and feedback to the traces.
This is more on traces.
This is more a easy to to to predict right data sets for evaluation.
Few shot prompting or fine tuning build data sets from examples, production data or existing sources.
You remember we talk about test data sets in the previous vlog.
What is the hub?
So this is the hub.
They are they are calling.
Well sometimes it's a little bit confusing.
They have and the playground because actually you come here to the hub.
You will see that here you have a list of prompt examples that you can reuse or you can modify.
For example, you want to use a prompts to extract for extraction and using, for example, OpenAI GPT
three five.
Okay.
So you have a number of them.
This is a brown YouTube transcript to article.
Convert any YouTube video transcript into an article.
Okay.
You go here and you see that this is a prompt that someone has created.
And we have it here if we want to reuse it or modify or whatever.
And if you want to edit, this is where you go to the when you go to the playground.
So sometimes it's a little bit confusing the how and the and the and the playground.
But the interesting thing.
So we define the Lang Smith prompt half uh, as uh, you know, an environment for prompt engineering
experimentation.
Okay.
We will see more about that later, because it is more interesting that in Sims, it's not just for
playing with prompt engineering, because you can come here to any trace.
And in this trace you can go to the playground and start playing with different LM models, uh, late
model feature, you know, different prompts, etc..
So we will go there.
It's more interesting than than than than it seems.
Then what do we understand.
Uh, for by collaboration in in in in this platform.
So we, we understand collaboration between developers and subject matter experts mostly.
So remember that we talk about product managers, a subject matter experts and software developers and
LM app developers.
Right.
So collaboration these are platform that can include the annotations, uh, and the feedback of this
kind of, uh, members of your team and of course, the feedback from the, the, the beta users and
the final users of the, of the application.
What is auto evaluation?
You can use an LM and prompt to score your application output, or write your own functional evaluation
test to record different measures of effectiveness.
You will see this okay.
So we will talk more about the evaluation about and about the different possibilities.
When you hear when you listen about when you hear about regression testing, and sure that the new features
or updates do not adversely affect the existing functionalities of the LM application.
So but this is saying is when when you.
When you listen or when you read about regression testing in in the following lessons.
Think about that.
So a are the changes we are making in the application a affecting negatively to what we already have
there?
We will talk more about this later.
Then we have another a terme which is online evaluation.
This is a next feature of long chain.
This is coming soon.
They say that they plan to have this new feature that will continuously track qualitative characteristics
of any life application and spot issues in real time with monitoring.
When we talk about observability, what do we mean?
Monitoring health and performance of the LM applications.
Insights into its behavior.
Performance of the LM model, and the interactions between the LM application and the LM model.
Okay, observability is a very frequent terms in LM ops, as you will see.
Then we may hear deployment and one click deploy.
So deployment to a in this context in the Lang Smith context when they say when they talk about one
click deploy or deployment, they are talking about deployment to lang serve.
As you know lang serve is the the another product from Lang chain.
But this feature right now in in Lang Smith is still in beta.
And you will see that right now Lang Smith has three versions.
You have like the free version.
Then you have the plus version and the enterprise version.
So this possibility, the deployment to Lang serve is in beta and it is only available for the two higher
versions, the plus version and the enterprise version.
So in the notebook we have included a link where you can have a more when you can find more Lang Smith
terminology.
In fact, you are going to see some other links to the Lang Smith documentation.
They they have a good documentation prepared.
So this is interesting.
And we have included some links here uh, for you as well.
Okay.
So let's talk about the most frequent ask questions regarding Lang Smith.
Uh.
So these are the six questions that, in the opinion of the long chain team, are the more relevant
or the most frequently asked regarding the platform.
So people ask, can you use language myth if you are not using long chain?
The answer is yes.
You can use language myth even if you don't use long chain.
Second, can you self-host long Smith.
Yes, you can sell horse meat, but with enterprise pricing, with the enterprise version.
Okay.
This is something that some big companies can be interested in, you know, because it involves security,
privacy, etc., etc..
So this is a feature that is available but is only for the higher version of the platform.
Is Lang Smith Secure?
Very important question.
Answer Lang Smith traces are encrypted and stored securely.
This is the Lang chain team talking.
So they are a showing.
Or they are stating that Lang Smith traces are encrypted and stored securely.
Can you limit the number of traces sent to Lang Smith?
This is very important.
Why?
This is very important because you are going to pay once you reach the free limit in the number of traces,
which I think is 8000 per month.
If you want to go over that, you will have to pay, and you pay by the number of traces you you use.
Right.
So it is very important to to understand, to know that you can limit the number of traces that you
send to Lang Smith.
So you can specify the percentage of traces you send to Lang Smith.
And we highlight here.
This can reduce your cost to Lang.
So this is very important.
Next question does Lang Smith slow your application.
Answer.
Lang Smith does not make your application slower.
And finally does Lang Smith use your data.
Answer.
Lang Smith does not use your data.
So they do not use your data for training purposes or anything like that.
So very, very interesting lesson.
In the next lesson we are going to talk about the initial operations, how to create an account, how
to create a project, etc. we will see this in the next lesson.