Okay, so how long?
Chain locksmith.
Excuse me.
Help us during the production phase.
The first thing we can do is to keep using Lang Smith, as in the beta testing phase, to keep processing
and analyzing user feedback.
So this doesn't stop when we enter in the production phase.
We still need to keep processing and analyzing user feedback, so we will keep doing the same thing
we started in the beta testing phase.
Okay.
Second thing how how to use Lang Smith to monitor key metrics.
So in order to measure the performance of the app, we are going to use Lang Smith to monitor key metrics.
Let's take a look at the platform.
So with the monitor tab of the project, if we go to one project, we see that there is one tab called
monitor.
So in this tab we can monitor the key metrics.
So you see here trace count LM call count trace success rates alarm call success rates latency LM calls
per trace feedback tokens.
Tokens per call cost okay.
So you have streaming.
Trace time to first token, etc..
Okay.
This here this P50 is the the median.
This P 95 is the highest value or something like that.
So we will learn more and more about the different ways of presenting these, uh, charts.
In my opinion, they are going to improve this dashboard a lot in the coming versions.
Right now is okay, but I think it's going to be much better.
So you have here different possibilities.
But eh, in my opinion, they are going to improve this area with a little bit of work.
It can be much, much, much, much better in the visual display.
So key metrics are cost, number of tokens per trace, latency, etc..
Lang Smith allows for tag and metadata grouping, which allows users to mark different versions of their
applications with different identifiers and view how they are performing side by side within each chart.
This is helpful for a B testing changes in prompt model or retrieval strategy.
So when you are working, uh, in this, uh, well, let's keep reading because you are going to see
that there is something very interesting here.
So after opening the monitor tab click on the tag or metadata buttons, tag or metadata buttons that
are located on the top and select the metadata or tag that you want to use to display the monitoring
data.
Apart from AB testing, monitoring is a great way to see if your application in production is performing
better or worse with the time.
So.
One way.
One very, uh.
Good way to improve your application.
Is to, uh, prepare different versions with small improvements and then compare the performance of
these new versions with the current one you are having.
You will see that we have different strategies to do that.
We are going to see them later.
But here you see that the visual display of the monitoring, uh.
Indicators is a very good way to a compare.
The visually is very easy and fast to see if one if one version is better than another.
Okay, so we can use Lang Smith to monitor key metrics, and we can also use Lang Smith to mark different
versions for a B testing of prompts, models, or retrieval strategies.
This is the these are the three things that most frequently are we going to compare different prompts
different LM models.
So using OpenAI as LM model or using anthropic or using a mistral or Lama to.
Lambda three SIM preparation or different retrieval strategies.
So one of the most interesting uses of monitoring is to compare the performance of different versions
of your LM app.
Clicking on the metadata metadata button and selecting one particular metadata parameter will show you
how the different versions of your application behave.
It allows you to a B test different configurations, for example using different LM models, prompts
or retrieval strategies of your application and see how each of them impacts on performance metrics,
cost, number of tokens per trace, latency, etc..
Okay, we are going to see some very interesting advanced tips associated with some of the things we
have been talking here.
Okay, so be ready for the next lesson because you are going to find it very interesting.
The advanced tips we are going to share with you about Lang Smith.