Now, let us discuss about Cost and Bisson, so we have already discussed what hostis cost is, the
sum of all the errors which are present in the predictions which you have made.
So cost function is something which you want to minimize because we want to minimize the edit, which
is present Dinamo.
For example, our cost function might be the sum of squared errors or their training set.
So in this case, we will be reducing the sum of squared error.
So we will have to reduce the error of the entire model, which we have now.
Gradient descent is a method for finding the minimum of a function of multiple variables.
So we want to find out the minimum value of multiple variables so we can use gradient descent as a tool
to minimize our cost function.
Now, let us discuss that.
Why do we really need gradient descent?
So one way of solving the problem would be so let's say we have this data, we have this line which
we have to create and we will have to find out several values for this.
So one week with me back, I can randomly select some bigger values and keep on trying again and again,
again and again until I find a very nice cost value.
So I will put in a lot of the values and I will compare different models which I have created, and
I will select the model which has the minimal cost value I will based on those minimum cost value,
whatever values I would have used, I will use those values.
So this will not be a feasible solution, because what will happen is I can use it, but only if I have
endless amount of time and a lot of patience and I can keep on trying a lot of values.
And then by using those detainees at that point, when the values stop changing and then I don't really
have to modify the value to reduce the cost, at that point, I will be stopping and I will select those
with the values.
But it is not a feasible solution.
So that is why we need gradient descent.
Now, gradient descent is completely an alien form for us and we don't really know what gradient descent
is.
So let us try to understand it and let us try to get in depth.
So.
When we are finding out the cost value, so this GLW is nothing but the cost of the entire function.
So here W can be vate or begin in simpler terms of the integration.
We can call it the BWC, so we can either say B.W. or we can say that it is one in the same thing and
G is nothing but the cost function.
So we can say gee W or we can C C or we can see F of.
Excellent.
So it can be called any of these things.
So what will happen is we will randomly select value, so we will randomly select a value for the Beita
and based on the value we will select, what is the cost?
And at any point of time, at any point of time, this is the function which the essentials.
So this is the function.
The basically the cost or the error will be.
President in a linear regression.
So let's say we have this cost, we have this function and there can be multiple lines.
So as I change the B values, as I change the slope of the equation, the line will come closer to this
particular line.
So initially, I will be creating this line, which will be a lot different from the correctly, then
I will create the line.
So when I create this line, the cost will reduce slightly, then I will create another line.
Now the cost will produce a lot less.
Now, then if I again at one point of time, the cost will be very low.
This is the minimum cost, which I can.
Then after that, once I just change the values again, then the cost will again increase.
And after that, the cost will keep on increasing.
So these are the different cost values based on the values which I can have, so same thing has been
depicted in this particular school.
So initially, the VW will, as the VW will change with the change in the VW, the cost will keep on
changing.
So the cost will keep on reducing.
Initially, I offered a point of time when the cost will achieve the minimum value.
After that, when we again change the VW, the cost will start increasing.
So we want to find out the optimal values and thus we want to stop at the point of time when this cost
is minimum.
So when we look at this particular slope, we can find out the line of this slope.
So what is the slope if we look at a very minor point on this particular golf?
This Gulf is actually nothing but a slope, which is between this small line will be depicted by the
change in the if the change in the function.
And this small section, this horizontal section will be depicted by the change in the veto.
This is the change in the double.
So let's see for the.
So this change in.
The function and this change in the B.W., when we divide this function by the B.W., we get the T,
the V, this is the equation of the band, Hedo.
And what is done theater than theater is nothing but the slope of this line, which is the change in
the function with respect to the change in the value.
So hence, we can see that the change in the function with respect to the is equal to the slope of the
line.
So what we can say very early in this investigation, we will get this done.
Multiplied by this storm.
This storm is actually the cost of the function at any point of time.
And this is my new change in the B.W., which means that I can find out the gradient at any point of
time, I can find out the change which I want at any point of time by multiplying the cost.
Of the function at any point of time with the change in the BW.
This is what the president is, so now when I see this particular function, I know that whenever I'll
be changing the B.W., so as the B.W. will change accordingly, the value of the cost will change.
And as you see, the slope is slowly and gradually decreasing.
So when we see that the slope is very high, the value of the slope is very large, but after that,
the value of the slope is decreasing.
The value of the slope keeps on decreasing as it reaches to the minimum point of.
The cost, so at this point, the cost is minimal and the slope is also minimal.
So when we are changing the values of the league, when we are changing the values of the weight, that
is one more scenario which would.
So if we change with respect to the.
So if they are changing with respect of the B.W., so what will happen that initially when we are changing
with respect of the B.W., the slope will be higher?
So we can make Foster's changes so we can change the value of W Foster's, so we will change the make
changes to the values of the W very bust.
We will make the changes in the value of the B values very fast.
But when we reach towards the minimum value, what will happen is if we are making the changes very
fast, then it will go like this.
There will be one change which will be very fast.
Then there would be another change.
Then the last change would actually overshoot this value and then we would try to get back and then
we'd make the same large change again and then we will again overshoot the value.
So what this really causes, it will again and again and again overshoot the values for us.
So what we will have to do is we will have to reduce the change which we are making as we reach the
minimum value.
So now if you make the changes very fast, then there is a chance of overshooting.
Now, let us look at another scenario.
The second scenario would be if we are making changes very slowly.
So if you are making the changes very slowly, then what will happen is that the values will take a
lot of time to reach the minimum value.
So it will be so slow that it might not even reach the global minimum and it might take in a finite
amount of time to reach the minimum value.
So that is the reason why we will have to choose not a very high value or not a very small value, we
have to make sure that the value which we are selecting is a moderate value so that we can reach the
global minimum effectively.
So now let's get for.
So let us try to formulate the equation so this is the equation which we will be having.
So we are trying to generate a function of X. So the function of X will have to be in the form V dunnart
plus B, the next one, plus B.W. X2 and so on.
So if we have this particular equation, this X one value will be one of the variables.
X will be another input variable and we are expecting the output of this to be Viag.
We are expecting the output to be.
Why value, but actual output, which is expected, is why and the output which comes out to be is by.
So the added value will be the expected output.
Minus the actual output, which we get.
After putting the BWC.
So what will be the added value, the added value will be via minus why have we had this?
Nothing, but we done all plus with our next one plus B that we do now because B is a constant value.
Can I multiply one with it?
If I even if I multiplied with one, the value will remain the same.
So I can multiply it with.
It's not.
So, like, we multiplied with X and let the value of X, not the one, so I will have an X, not you.
And the value of export will be one.
So I will get the value of EDAR as a minus be done or not.
Plus, we don't explain Glasby that we do will be done.
What is the constant value X not is the one be the one we do on the slope values.
So what will be the cost, as we discussed, costs will be the sum of all the other values, so the
cost can be described as the square, the sum of all the added values or the.
Some of all the absolute better.
So cost can be any of these two.
So let us look for.
So begin.
Vitamin D cost value by finding out the beta values, because we already know the X values and we already
know the value.
The only thing which we do not know here is the value of the betas so we can find out the value of cost
by putting in different values as Vianne X values are constant.
Hence, we will try to find out we don't value such as this minimum.
So what do we get?
This is the matrix of X, that is the X one X, not value as one.
And the value of X one, two, three, four for all the rules of the.
Similarly, we will have the Matrix y, then y will be the value of life for all the roles of data.
Remember this, if we talk about the loan data, then X1 will be the amount of the loan, the number
of dependents, the gender of the person, the number of legacy, the salary of the person, so the
number of children.
So these should be the values of the X.
And what will be my life will be the value which we are trying to predict, which is the interest rate
you.
Right.
So this is what our data will look like and they'll be the matrix is what we are trying to find out.
So the equation which we will have here is the sum of Aitor Square.
So what will this be?
It will be the sum of it or square with it.
It is nothing but VEI minus V does not.
Minus one X one minus VW x2 getting the minus sign inside because all the values were plus all values
will convert to a negative and then square all of these.
This is the equation.
Just as it is just the same thing as we have written here.
Just the same thing, we have just opened the bracket and put minus all inside.
So this will be the value of the cost.
So how will we update those values so when we are trying to update the week or when we are trying to
update the veto values, the update, we will be the we will change the weight by W minus the change
in cost with respect to change in the that just.
Change in cost with respect to change in.
The W or the V values.
So the update rule is new, the blue is equal to W minus.
A learning break, a learning break in changing in with respect to W. Now, what is this learning grade?
What is this?
This is nothing but how fast we want to go down this route.
Remember, we just discussed that if you will go very fast, then we will overshoot and we will go very
slow.
Then we will and we might not even reach this point of time.
So that is the reason why this value of aid is very important.
So we will have to decide what the value of our should be.
Getting Florida now, why will be will be training the later.
They could read different scenarios possible, so you remember this kind of slope which we have created,
so this was the data points which we had and we assigned a line to it the right to predict these values
based on the line which we have read.
Now, when we are assigning this line, when we are creating this line, this equation of the line,
then this line might not be able to predict these values very properly, because if you see the function
is not really actually a linear function.
This would have been predicted properly if they had a line like this, I'm datapoints would have been
something like this.
And then they would have created a line like this.
Then we could have said that this is a perfect line which we have created for the moment, but right
now the data points show a completely different picture.
So this scenario is called the footing.
When we are not able to predict the values properly, we are not able to learn from all the data points
because the language we have for it is very simple in nature, because the function which we have applied
is very simple in nature.
So it is not able to capture the fact present in the data.
The other side of this here, we have this data, the same data, and we have it OK to it instead of
a linear line, we have the data of the line here.
So when we go through this data, this is able to understand the patterns of the data.
It will know that when the value of X is very low, you have to have some Y value and the value keeps
on decreasing till one point the value of X, and after that, again, the value of Y starts increasing.
But it does not understand this fact that.
And on the other hand, this this particular function which we have created is just right for this type
of people.
And on the other hand, when we're fighting a very complex nyein, a very complex function to do this
data, it is trying to learn from all the data points.
It is trying to learn from all the data points.
So what happens is that when a problem comes and when we are testing the data and when we are actually
using the model in your life, then what happens is if we have a data point here, the model will try
to predict this particular value, which will be very different from the actual value.
But if we use this model and we have the same data point, so the error will be less in comparison to
this one.
And again, if we have a data point here.
Then this might not be.
Predicted properly.
But if we have data point here, then the error will be less.
And if we have a data point here, then again, the error will be a little later, but it will also
be kind of fine.
So what is the difference between under and over 40?
Under 40 is when the model does not really learn enough from the data which is present.
And overvoting is when the model tries to learn a lot from the giving data, it tries to find out very,
very, very complex patterns from the data so that it creates a complex function.
And because of that complex function, it is not able to predict properly on the testing the.
On the life data, on a different data from the training data, it will perform very well on the training,
say, but it will not perform well on the realtime or testing the.
So that is the reason why we want to find out a line or we want to find out the function which is able
to predict values and it is generally in nature, there's a generalized model and not a very complex
model.
That is why we are looking for a simple model.
A simpler model.
OK.
So this is an example of over 40 under 40 and a model that is just straight.
Now, let us see how we will actually train the data.
So for training the data.
What we do is.
We will divide the data which we have in the tool box.
One is the Prani sic and another five is the best thing.
So what will happen is we will train our model on this training set so that the model will not see this
particular part of the test set of the data.
So what will happen is when the model has not seen the best data, we will train and model again and
again and very properly in this on this training set.
And when we are satisfied with that model, then we will check if the model is performing well on the
tests and if the model performs well on the training side and the best both, then we will see that
that model is trained properly if the model performs poorly on the training set.
That means that it is understood.
You can see the model will not perform well even on the training data here.
That is why it is called underfeeding.
And if the model performs very well on their training data.
But does not perform well on the testing does.
Then it is going to be over 50.
And when the model shows same performance on the training data and testing data, a good performance
on both training and testing data, then we can see that the model is doing just great.
Now, let us look at another method.
So this method, which we have seen where we divide the data into training and investing, said this
is called hold out method.
So the out method is the simplest kind of method, the dataset is separated into two sets that is draining
SEK.
And the testing said.
And the more the the owning the using the training cycle and after training, the model is checked and
validated on the testing.
Now, what are the disadvantages of this, the evaluation when we are doing the evaluation on the testing?
It may depend heavily on which data end up in the training set and which end up in the essay.
Now, if we have divided the data randomly and somehow the data patterns which we want, the more they
do end up in the testing data, then our model will not really learn much from the training dataset.
So we want to make sure that all the patterns which we have, that those should be presented the training
data also so that we can find out and learn from the training.
Does the evaluation may be significantly different, depending on how the decision is made?
So what do we do here?
To prevent this, we use the cross-validation method and what is cross-validation method.
We basically divide the data instead of dividing the data into training daytime testing data, we divide
the data into different foods.
Now, these a number of roles could be any number of holes, five, seven, then 15.
It is completely up to you.
And based on these falls, we allow each fall, each and every fold to be a testing data at one iteration.
So in the first iteration, the first split, the first hole will become the testing data and all of
the folds will become the training data in the next iteration.
This one, the fall two, will become the testing data and all of that will become the training data.
Then in the next iteration, this one will become the.
Finding the testing data and all of that will be the training data.
So what will happen once we have then all of this, then we can take up average of all of these and
find out what is the performance of the models.
This way, the model will be able to learn from all of the faults.
So same thing, so here we have three fold.
So what we do the first time we make the blue one as distinct, next time we make the Greven as distinct,
then in the third time we make the red one as distinct and we will use them accordingly.
So this is what.
Cross-validation is so thin now we have learned about linear regression and how we will perform cross-validation
on that, so let us have a look at the code so we will look at the code in the next session.