Okay, so in this detailed video, let's talk about how Lang Smith helped us to solve the main challenges
we face in the production phase.
Let's talk about the production phase.
So remember what the Land Change team tells us about this phase.
Closely inspecting key data points, growing benchmarking data sets, annotating traces, and drilling
down into important data in trace view arc workflows that you will also want to do once your application
hits production.
Okay, so everything we did in the beta testing phase, we also want to do that in the production phase.
However, especially at the production stage, it is crucial to get a high level overview of application
performance with respect to latency, cost and feedback scores.
We are talking about monitoring key metrics.
This ensures that it is delivering desirable results at scale.
So let's see the first challenge how to keep processing and analyzing user.
Feedback in order to solve this challenge.
As we said, we need to keep using lamb-smith as in the beta testing phase, so no need to repeat what
we said for the beta testing phase.
About the second challenge how to measure the performance of the application.
We will use Lamb-smith to monitor key metrics.
Let's see more details about that.
Lang Smith provides monitoring charts that allow you to track key metrics over time.
Currently, these monitoring charts are, I would say in the early version.
I think this is going to improve a lot, but right now what we have is good enough.
You can expand to view metrics for a given period and drill down into a specific data point to get a
trace table for that time period.
This is especially handy for debugging production issues.
The platform also allows for tag and metadata grouping, which allows users to mark different versions
of their applications with different identifiers, and view how they are performing side by side within
each chart.
This is helpful for a B testing.
Changes in prompt model or retrieval strategy.
So these are the three areas where we.
We'll want to compare different versions.
Okay.
We have our, you know, a final version of the application, but still we need to check, you know,
different prompts, different models and different retrieval strategies.
Why?
Because things are always going to change.
We are going to have new models or better models, better versions of the current models.
We are going to have new prompting strategies.
We are going to have new retrieval strategies.
So even when we are in the production phase, even when we have an application which is live, you know,
being used by, by by real users and customers, we still want to test, you know, different, uh,
approaches and solutions.
And for this long chain is going to be very interesting uh, in this stage.
So in order to keep improving the app.
We will use Lang Smith to mark different versions for a B testing of prompts, models or retrieval strategies.
Okay, what we said and in order to measure the performance of the app, we will use Lang Smith to monitor
key metrics.
Okay, so this is how Lang Smith is going to help us solve the main challenges in the production phase.