In last session, we saw that how we can implement linear regression, but have you given a thought
when we will be implementing linear regression?
Then each and every column which we have in the data will get a confession.
Now, this would be a chance then one of the problems does not really need a confusion.
That is not sufficient.
Should have been zero or not present at all.
So in such cases, it is very difficult to find out which needs to be removed or we cannot really find
out how the value should be present for that particular column, because somehow linear regression would
always try to find the for the columns which we have provided, which will lead to overfitting.
The models would have low accuracy in case the model or fit the model to try to capture the noise in
the training data and will try to find a confusion even if it does not present.
The noise, the data point that does not really be present, the two properties of the data, but random
chance will actually impact the proficient values and lead to larger coefficients.
So learning this data makes the model more flexible at the risk of overfitting.
So the line language we saw earlier.
We will get something which is resembling to this, although the line will be linear, but still it
will try to do all of it to the data.
Hence, we need to do something about it, and that is what Regularisation is, the organization thinks
the coefficient towards zero and it discourages learning a more complex or flexible model so it will
avoid the risk of overprotect.
So to avoid overvoting in linear regression, we will apply regularisation to it.
Now, let us see what are different methods of legalization.
There are two methods of regularization.
One is the repression and the second one is the last ignition.
The formula for retrogression is summation of the town square.
And for the last regression is summation of the absolute value of the vehicle.
Hence, it will actually add the original value, the original linear equation, and add a little value
to it.
Now here we have this confusion, which is actually causing these regularization.
So this is the regularization which has been added to it.
Now, one thing to remember is.
That rigid aggression will try to minimize the proficient black vegetation, will not be able to bring
the sufficient to exact Zeder, hence there will always be a little sufficient value left.
While last regression will try to bring the coefficients to zero if possible, hence the volumes which
are actually not required will get a sufficient value as zero.
So there is no impact because of those of efficient on volumes on the regression model which we have
provided.
So let's compare both of these.
So rate will shrink the coalition for the least important predictors to very close to zero, while also
has the effect of forcing the conversion system to be exactly equal to zero when the winning parameter
lambasts sufficiently large.
So it will never make them exactly zero.
And final model will include all the predictors in case of regression by in case of loss or regression.
The method will be performed on the body and it will go on the efficiency to zero.
Hence it will also allow us to do variable selection so we can use this for feature selection also.
So let us look at the board itself.
So here is the code for rage and regression.
So for implementing, religionless, we will import and lassalle from psychic non linear models and
we will import grid so we could have implemented religionless or just the way we implemented the linear
model.
Simply by saying.
More dog food and then doing pretty here.
We want to try different values of LAMDA because we are not aware what value of lamb there should be
there.
So that is why you are using TV.
Now, what we will do is it will create different models with a different value every time so that we
can compare between different models with different values and select the one which gives the best results
to us.
So here what we are doing is we are creating a variable lambda, which contains the values between one
two hundred.
The number of values which we are generating is one hundred.
Remember the space function?
The space function generates the number of values that is in case of space.
We provide the number of values we want to generate.
And in case of acreage, we used to provide the step function.
So that is what we are doing, we are generating values from one 200 and we are using those and how
we are using those, we are creating a barometer.
Value, this barometer is a.
Dictionary, and it contains the alpha values, these alpha values contains all the lambda values which
we have created, that is values from one to hundred, and we are creating an object of the rich model.
And in this rich model, we can define if we want to intercept or not.
So here we are saying we want to put in the zip and we have creating a model object of the regression.
Then we provide all the details to the grid search.
So in the grid search, we give the type of model we want to run.
Then we give the parameter values which we want to put in.
Then we give them cross-validation value.
That is the number of forwards which we want to have.
Now, earlier, we have used the string split, which will simply split the data into a training and
existing dataset.
But now we are actually using the cross validation method.
Then we will create a certain number of calls and out of those words, each fold will get to be the
testing data and all of the data points will get to better training data at every point of time.
And we are giving this scoring method, the scoring method, which we have chosen now is negative,
mean, absolute error.
So we have just given the declaration of the grid search.
So this grid search will run this model with all the values of the meter, all the combination of values
in the parameters, but then false.
So total models which will be created would be 100 models and all the hundred models will be done ten
times as they are.
We will be 10 cross-validation go into the model.
Running will be 2000 models.
So we are doing now the grid such thing, so it is draining the model on each of the four values which
we have provided, and after that it will give us the best estimate.
So here we can see that the best estimate is if it is equal to one.
So we will look at the see the results and we can also create a function.
So here I have defined a function which will basically take the desired and it will.
Compare the results and provide us the different values for which the model has performed the best here
I can give the number of models which I want to have.
So I want to have the top three models.
So I've given both physical beauty as a default value.
You can change the number of models when you run the method itself.
So here I am running the meter.
I'm seeing report.
It will the grid search not see the results and the number of results I want is five.
So here I'm checking the model.
So the first model has alpha value one.
The second model has a value to the third best model has the alpha value three and the fifth best model
has alpha value for you.
And it is not sequential in nature usually, but it is just by chance.
It is coming out to be one, two, three, four, five.
And you can compare the performance or the performance with the least mean validation.
School is minus six point six zero seven.
And after that, we have minus six point six one two when we are comparing the mean validation school
of mean absolute error.
In that case, the value has to be near to zero.
So the best one would be the one nearest to.
Now, here we can get the best estimate, the value and from the best estimates, we can put the model
again and get the coefficient is.
So here you can see the full details.
It is five point eight zero minus two point five nine minus five point seven one one point one one.
So let us compare these corporations with the ones which we had earlier.
So here we are, five point eight minus 2.5, minus five point seven.
Yes, five point one one minus two point six.
This is four point forty, so you can see the fishing values have actually reduced.
So it has improved the proficient values, now we can run the last regression again.
So in regression, the process would be entirely the same.
We will again give the lambda values.
We will again do the grid search, create the grid search object, give the model detailed, give the
barometer details, give the number of cross-validation that we want, then the spotting criteria.
Then we are giving the greater short faith, we are flipping the model, then we are finding out the
best estimate.
After this, we are finding out the ranking of the models.
You here, you can see the best model has value as minus six point seven three, four, five four range
is it was minus six point six zero seven.
So here's the registration has actually performed better than the last regression.
And again, we can find out the.
Int. values and efficient values, and based on the disciplined Confucian values, you can see that
the values would be down to zero in the professions are not really required.
So we'll compare those with spend.
So let's compare those.
So here the model name is.
Would be.
So we compare that and find out the official tally, so I'll just run those piece of quotes here.
You can see I have created Lassalle on the school model and I've given the best estimate to it.
Then I have footed the new model, the value of the best estimate.
And then I've got the list of the.
Columns on the position.
So here you can see that the positions have actually brought down a lot and the efficient one, five
and six have been thrown into exact zeros while the coefficient X three has been reduced to minus zero
point zero zero five, which is very low in comparison to the coefficients which we had earlier in case
of orange.
So here you can see that the values were right to be reduced to a lower value.
But LASO has done a great job of reducing the column names to zero, the efficiency to zero.
So this is how we can run the linear regression, the regression and actually find out which columns
are going to be dropped so we can drop these columns and then make up for further analysis and how we
want to go ahead with the process.
So after this, we will be learning about a logistic regression, which is a classification model in
the next session.
And once we are done with the logistic regression, you will get to know how we will run that and then
we will discuss about three methods.
So that will give us another model which will help us to find out similar kind of vision values.
Which we can use for there to find out which columns can be dropped.