In this lesson we will talk about latency and speed in LM applications.
So.
This is an area where you are going to ask yourself if your.
Current conditions are the right ones when you are preparing to launch an LM application.
In some cases, as you know, cost and speed, uh, go in different in, in opposite ways.
So you may have a faster application, but this is going to cost you more.
It's going to be more expensive.
And in some cases you may have a more convenient application, more cost efficient.
But this is going to have an impact in the in the lower speed of the application.
So let's talk a little bit about that.
Uh, can you afford for the user not to have a fast experience?
This is the key question you should ask yourself.
Can you afford for the user not to have a fax, a fast experience so you can usually afford it if the
users are employees of a company?
And you usually cannot afford it if the users are customers.
This is like the main principle.
Second question that is uh, is is important for you to to consider.
There are some, uh, applications that usually require high speed.
And where we are talking about high speed, we are talking about low latency.
So speed and latency are opposite.
You have high speed applications when you have low latency and you have high latency when you have low
speed.
So there are some applications that usually need high speed.
What are they?
Conversational agents.
Virtual agents.
Chatbots.
Content personalization.
Apps and recommendation systems.
All these applications, because of their characteristics and the usually the profile of the of their
users.
They require high speed.
So if your LM application is in this category, usually you will have to worry about high speed.
There are other applications that.
Can afford low speed or lower speed than the previous ones.
For example, when you are dealing with research applications and this is very frequent right now in
the legal space, for example.
So lawyers are using LLM applications in order to a be assisted in their research, in their legal research,
or a marketing teams are using LLM applications and when they are working in market research, for example.
Right.
So these kind of applications can afford to be a not very fast.
Other kinds of applications that can work well with low speed.
Creative writing and content generation apps, especially if they are using.
If they are used internally in the company, they usually will work well with a low speed.
So the key question you have to keep in mind is how speed is going to affect user experience, especially
how low speed is going to affect a user experience.
If your user is going, uh, to be tired of waiting in your application and is going to a competitor's
application, you cannot afford low speed.
The second factor, of course, is cost.
So how much is going to cause you to have a fast application?
So these are questions that you will have to work on and work with your customers.
For example, if you are working in a in a consulting firm or if you are working in a, in a tech department,
and you you need to discuss, you know, this kind of budget questions, etc., with your manager or
different people in your company.
These are important questions you should solve before the launch of your LM application.
We will see in the next lesson a more about cost in LM applications and cost control.
Very important topic.
A.
When you are dealing with LM applications.