In this lesson, we are going to talk about a very important topic in the LLM applications field, which
is cost control.
You can have a big problem with cost if you are not careful.
So this is a very important lesson.
When you work with LM applications right now, let's see what happens in the in the in the following
years.
But right now cost is a very important issue.
And regarding cost you are frequently going to face two questions.
Do you use a private LM model or an open source model.
And second, do you use the rack technique?
Or do you use fine tuning or do you train an LM model from scratch?
The second question I, I hope is very clear for you now, but it is not for some of the people you
are going to deal with in your professional life.
So there are many articles talking about fine tuning and training and LM, uh, models from scratch.
And you have videos and courses and all that.
And some people, which is not very familiar with LM applications, may tell you that this may be the
way to go, the good way to go.
Right.
So my understanding is that right now, you know very well the answer to the second question regarding
the first question, a let's talk a little bit about the cost of a private LM and remember, we told
you that in the first year of LM applications, a private LMS have been the choice.
So 99% of professional applications are using private LMS, especially ChatGPT.
But this A is a.
Difficult a matter.
Why is that?
Because a if you are not careful, you can spend a lot of money with your LM applications and private
LM models like ChatGPT.
So see for example, how much you can spend.
So the main problem of a private LMS and LM applications is that the cost can escalate very quickly.
Remember you pay for the number of tokens processed.
And let's see a few numbers.
So let's see.
You want to summarize one page of text in ChatGPT.
So this is going to cost you around $0.015.
Nothing important, but who wants to summarize one page in a enterprise level application or in a,
you know, open application, an application that is open to the world, right?
Right.
One page is something you do in one second.
How many pages are you going to summarize in an LM application being used in a, I don't know, 50,000
people company application or an application that is open to the whole world.
So probably you are going to summarize millions of pages.
And if you summarize, let's say.
1500 pages per.
Minute or per hour in this kind of applications.
The cost of using ChatGPT as your private DM is going to be $20 per minute or per hour.
So you have you can see the big.
Jump, you know, between one page and 1500 pages.
The cost of using private LMS can be very, very painful.
It is one of the main considerations for a startups.
So there are many startups that, uh, do not launch their projects or are extremely careful, uh,
into when and how launch their project, uh, because of the cost, uh, problem.
So this is one of the main things you will have to consider.
You will have to calculate in advance how much is it going to cost?
How are you going to control the cost and who is going to pay this cost.
You will see that a private lens like, like, uh ChatGPT are going to offer you some tools in order
to control, limit, you know, uh, get warnings when you have, uh, you know, an escalation of the
cost, etcetera, etcetera.
So be very careful with that.
Pay attention with that before launching an LM application, because this can be a very painful, uh,
very painful, uh, matter for you.
Regarding the second question, we already know that fine tuning and training an LM model from scratch
are things that, uh, we cannot do.
These are things that only the very big companies or very big institutions with very big budgets and
very big, uh, computational resources can afford for the rest of us, the rack technique is the way.
Okay.
So we we talk about that in a previous lesson.
The the cost consideration is one of the main, uh, advantages of the rack technique.
Uh, it's not the only one, but it's a very important one.
And we will see how fine tuning and and training costs evolve.
But right now is, uh, is a no brainer.
I mean, there is nothing to, to to consider here because the difference is so huge that, uh, rack
is being used in 99% of the professional LM applications.
Okay.
Where you find this kind of, of question.
So.
Cost control.
When you are preparing the launch of a professional LMS application is something that you will have
to study and prepare in great detail.
So this is one area where you want to spend time and be very sure before a making an important mistake.
In the next lesson, we are going to talk about other topics that you will, uh, face when you are
preparing the launch of a professional application.
LM ops.
Let's see what it is in the next lesson.