Okay, let's see how Lang Smith helps us solve the main challenge we have in the beta testing phase.
So you remember our main challenge now is going to be how to process and analyze the feedback of the
initial user.
Remember, in the beta testing, we are going to test our prototype with a small initial group of users
that we call the beta users.
So one of the problems that the long chain team has observed during this year, working with different
LM app development teams, is that they were, uh, they were having problems in order to understand
how to process and analyze the feedback of the initial users.
So this is how this is how Lamb-smith can help us, uh, solving this problem.
So here in in the notebook, you have a detailed, uh, explanations about that.
And we are going to take a look also at the platform.
So how to use Lang Smith to filter traces with negative human feedback to understand the problems behind
them.
Remember that this is the thumbs down button right.
So in our application we are going to have a thumbs down button.
And what we want is to understand and what is happening.
Uh, when the users click on that button.
Right.
So in the project traces project Traces view.
Click on any of the tags displayed on the right sidebar.
So if you see here we have some tags in the right sidebar.
We click on this, uh, tag correctness with ease, which is one of the tags we have included in this
particular project.
You see that we are filtering the traces.
So in this project we don't have many traces, but in a regular project in the beta testing phase you
may have thousands of traces.
So it would be very interesting for you to have this filter feature in order to filter the traces with
negative human feedback.
Okay.
So in the Project Traces view, click on any of the tags displayed on the right sidebar.
Those tags were created by you when you coded, for example, the thumbs up and the thumbs down buttons
of your LM application.
You will see this in the production in the in the professional project later.
We can use this if we have something like thumbs up or down in the LM application user interface.
Then we can filter with the tag that thumbs up or down add button.
Okay, this is this.
Then we can filter with the tag that thumbs up or down let's say button.
Okay, so I think the concept is clear.
So long as myth allows us to use filtering in order to select only the traces that have a negative human
feedback.
This is going to be very helpful for us during the beta testing phase, in order to understand better
what is happening.
Second, how to use lamb-smith to inspect interesting traces and how to add annotations to one trace.
So annotations is the feedback that we include in one trace.
So the the user of the application can add feedback using this thumbs up or thumbs down button.
But we in Lang Smith can add manual annotations to our traces.
And this is interesting not just for us as engineers, but also for the product managers or people from
the marketing department, subject matter experts, etc..
Okay, so how to do that?
Click on each trace to see the trace details and after clicking on a particular trace, click on Send
to Annotation queue.
You can do this in many different locations in the platform.
So if you go to projects then you select one project.
You will see that in this initial table you have this option to include a trace in the annotation queue.
You also have the option if you open one trace and you have here the button annotate okay.
So a so here you have the explanation.
After clicking on a particular trace click on Send to Annotation queue.
There you can add human feedback to the trace like for example you can include default language, myth
tags, correctness, faithfulness, conciseness, context, relevancy, etc. or you can also add custom
tags and notes.
Okay, so a.
You are going to see this in the example, but it's very easy.
Okay.
So here you have some of the the the tags we have included.
You can include a manual uh notes and you can add tags.
You see here the default tags that come with a lamb-smith.
Okay.
So.
You can also annotate one individual run inside a trace.
Okay, so apart from the trace, you can also come here.
And in one run you can annotate the run okay.
Annotate.
Run.
For example, if you find out that the problem with a bad trace is in the retrieval step, it could
be interesting to annotate the retrieval run with feedback that can help you to fix it later.
Okay, for example, this one, we can see that the problem is here when we check, you know, all the
operations.
So we say okay, check, check this step because I think here is the problem okay.
Annotation is especially useful in a collaboration environment when the LM app developer and the subject
matter expert can both comment on the highlighted trace traces.
Okay, so the subject matter expert is going to be the product manager, or people from the marketing
department, or a people with special knowledge of the field we are working in.
Okay.
Next thing a Lang Lang Smith does for us in this beta testing beta testing phase.
How to use Lang Smith to expand the test data set by adding runs as examples.
You can add traces as examples to the test database, and runs are the different steps that compose
a trace so you know how to add.
A trace in the data set.
Okay.
And you know how to add a run in the data set as well.
Okay.
We show you before.
But just a reminder, in the next lesson we are going to see how Lang Smith helps us solve the main
challenges we find in the production phase.
Remember, we are having a first round, a first view of Lang Smith at work, but in the next, uh,
blocks you are going to see Lang Smith working with real projects.
You will see a professional project and you will see a simple project as well.
So in the next lesson, we are going to see how Lang Smith helps us solve the main challenges in the
production phase.