Okay, so let's start talking about the prototyping phase and the main challenges land chain has identified
in this phase A with the LM app development teams.
So first let's talk about the prototyping phase.
It involves quick experimentation between prompts, model types, retrieval strategy and other parameters,
the ability to rapidly understand how the model is performing and debugging where it is failing is incredibly
important for this phase.
Okay, so we have four main challenges in the prototyping phase.
First when things go wrong.
How to identify what is failing.
Second, how do we iterate an experiment in a fast and easy way?
Third, how to compare the performance of alternative prompts, retrieval strategies and model choices.
And fourth, how do we test the performance of the prototype?
We are going to see how we solve these challenges with Lang Smith in the next lesson.
Next, let's talk about the challenges we face in the beta testing phase.
Let's talk about the beta testing phase.
So the beta testing phase allows developers to collect more data on how their LM applications are performing
in real world scenarios.
In this phase, it is important to develop an understanding for the types of inputs the app is performing
well or poorly on, and how exactly it's breaking down in those cases.
Both feedback collection and run annotation are critical for this workflow.
This will help in curation of test cases that can help track regressions, improvements, and development
of automatic evaluation.
This is how long chain is describing as the beta testing.
Okay, so we know what beta testing is about.
And what they have identified is that the top challenge that development teams are finding in this stage
is to be able to answer this question, how to process and analyze the feedback of the initial users.
And remember.
These beta testing phase is in many cases just.
In many cases, many LM app development teams, they simply don't go through the beta testing phase.
They just go with their gut feeling and language.
Myth is a great tool to use a much more scientific approach in the development process.
And this is how classic app development is doing.
So long chain, as we will see next, is providing us a very good way of doing beta test beta testing.
Finally, let's talk about the production phase.
So in the production phase, closely inspecting key data points, growing benchmarking data sets, annotating
traces, and drilling down into important data.
In Trace View are workflows you will also want to do once your application hits production.
This is from the Long Chain article.
However, especially at the production stage, it is crucial to get a high level overview of application
performance with respect to latency, cost and feedback scores.
This ensures that it is delivering desirable results at scale, so production is very important and
is not just about monitoring.
It's also about a key processing and analyzing user feedback.
Right.
So this is where long chain is telling us that this is the it is the first a there is the first big
challenge for LM app development teams, how to keep processing and analyzing user feedback in this
production phase.
Second challenge how to measure the performance of the application.
And third, how to keep improving the application.
So in the next lesson.
In the next lessons, we are going to see how Lang Smith is going to help us solve all these challenges
in the.
Prototyping, beta testing and production phases in order to a develop a professional LM application.
And once we finish with that explanation, we will see long chain doing it at work with real applications
first with a very basic application, then with a professional application.