Hi there.
Now, we have discussed about linear regression, we have discussed about logistic regression, we have
talked about the decision tree and the informal methods and methods we have learned about the Vikings
and how we can implement bigging.
Using random forest now under the informal learning method is boosting machines.
So let us learn about boosting machines and see how those can be implemented.
So let us get Ferdo.
So this is what boosting looks like.
So as discussed earlier, we will be having.
Sequential models where each model, each of the first model will be learning from this particular data,
will be learning from the X and will try to predict value.
After this model, the output from this model, that is the error which we will be getting from this
particular model, will be pushed into the next model to be predicted.
So the target for the second model will actually be the error from the first model.
So it will keep on improvising the previous model in the next iterations.
So the last model, we would expect that the last model has no error presentiment.
Now, if they will have no error present in the last model, then that would mean that we have somewhere
over fritted on the training data.
So we should know that where we need to stop and we should also know that there has to be some kind
of regularization which needs to be applied on this boosting and got it to.
So the major points to be remembered about boosting is that these models are generated in a sequential
order in comparison to bigging where in bagging the models are activated by.
I'm boosting is created by using different decision tree stumps again.
Now, these decision trees dumb does not try to feed data to any specific pattern, like an in model,
which would always try to figure out what linear equation.
So here there is no specific pattern in which we are trying to fix.
We are just trying to create small models.
That each of these one models would be explored on something we just want to create more and more,
which will be able to emphasize on one specific, this specific pattern.
So each model will learn something and push it forward to the other model, and the model will try to
improvise on the patterns which were missed out by the previous model.
So there will be a certain amount of voltage which will be given to the model when the items in the
model one, which model one could not predict.
So the values which were misclassified or not predicted properly by model one will be improvised in
model do by giving a higher ratings to them.
So this is how each model will try to learn some part of the patterns which are present.
And somehow the entire combination will be able to learn all the patterns this way and will be able
to predict better in comparison to a normal decision.
Three.
So here the input variables remain the same in all the cloners, the input variable will always be X,
Y and Z target variable will be changed to the added value from the previous model.
The target value for the first model will VI and the target value of value for the next model will be
Y minus W one w w well three and so on.
Where the well one is the prediction which is made from the.
Each week learner, that is model one more do more to transform.
Now, let us see how the algorithm works, so to find vehicule, we apply these learning algorithms
with a different distribution.
Each time the best learning algorithm is applied, it generates a new prediction.
This is an iterative process.
And after many iterations, the boosting algorithm combines these weak rules into a single strong prediction.
What are these weak rules?
These weak rules are the small decisions dump's or these small weak learners or the small modules which
we're connecting here.
Each model is a vehicle which we are combining together.
Now for choosing the right distribution, here are the following steps, the first step, be the best
lunatic's or one of the distributions and assigns equal big order attention to each observation.
Step two will be if there is any prediction error caused by the force to be stolen base Lerner then
b b higher attention to the observations having prediction error.
Then we apply the next based learning algorithm.
So in these step three, the step two will be keep on repeating until and unless we reach the higher
accuracy or the limit of the based learning algorithm is reached.
So does that mean we are able to find higher accuracy and we are able to reduce the error?
We will just keep on improving on the previous based learner.
So we will keep giving the higher priority or more attention to the previous misclassified points.
We are able to improvise once there is a position with.
There is no improvisation happening then we can stop.
So finally, it combines the outputs from the vehicle and creates a strong llona, which eventually
improves the prediction power of the model boosting pays high in focus on the examples which are misclassified
or have higher errors by perceiving the group.
So each model here is a vehicle learner or is called a vehicle, which are combined together to form
a strong learner and each of.
When misclassify something.
So there is one rule.
So this rule will be able to classify some part of the data correctly and it will be misclassifying
some other part.
Now the part which was misclassified will be given more weight, it will be given more attention, and
this particular model will try to classify it properly.
Now, again, there will be slight improvisation, then we will combine more than one and there will
be more number of rules which were created here.
Now, again, these monitors would have missed out on something.
So those predictions which were misclassified, those will be, again, taken up and those will be given
higher rated by this particular model.
And this model will again try to improvise.
So this is how each and every model will be a small rule or some have some property which will be capturing
some different patterns so that when combined all together, they will work as a strong learning.
Now, let us consider what is either boosting, either boosting is one of the basic algorithms in boosting,
which will allow us to understand how this entire thing works.
So let us start with this method.
So let us consider the box one.
This is box one.
This is box two.
This is box three.
And this is box for.
Now, in case of box one, you can see that we have assigned equal weight to each data point and applied
our decision stamp to classify them as plus or minus.
The decision stem from the one has generated a vertical line at the left side to classify the data point.
So this is the forced decision stump, which is classifying the data now.
It classifies these two points correctly, but it has misclassified these three.
Plus, signs and classified all the negative signs correctly, right, so we see that this vertical
line has incorrectly predicted the three plus items minus.
And in such case, we will assign higher ratings to these three plus signs.
So we will provide more higher ratings to these three plus signs and apply on their decisions.
So what will happen in the next room?
In the next room?
What we do is because we have provided more ratings to these three storms.
So these are bigger in size as compared to the rest of the data point.
Now, in this case, the second decision, staff will try to predict them correctly, the three plus
signs correctly.
So that is a new vertical line.
They do, and the right hand side has been added.
So what this has done is it has misclassified these classify these three misclassify plus signs correctly
and also these two plus signs correctly.
But now it has misclassified these to these three negative signs.
So now what we will do is we will now give a higher rate to these minus signs.
So now what we have done is we are we have given higher ratings to these three minus signs.
So now what will happen?
We will have our decisions down the three, which will predict these misclassified of points correctly.
Now, this time, a horizontal line would be generated to classify the plus and the minus and based
on the higher weight of the misclassified observations.
So now what will happen?
It is generated this line, which would have classified these three plus signs correctly and these three
minus signs correctly because they had a higher rate this time.
So now we have these two plus signs which have been misclassified and one minus nine, which has been
misclassified.
So now what we do is in the box four, we have combined the one, the two and these three to form a
strong prediction, having a complex rule as compared to the individual V cloners.
So on combining these three decision stems, we can see that now these points have been classified properly.
So these negative signs have been justified as negative and these positive signs have been classified
as positive.
So the algorithm has classified these observations quite well as compared to the individual vehicle.