In this session, we will discuss about Osama bin Laden, so we have already discussed about this case.
And they know that decision trees have certain drawbacks.
This is really thanks to Overthought.
Hence, we need a bit more news, which we're really not all Wolfert.
So let us have a look at ensemble and try to understand what ensemble can do for us.
And so learning combines multiple Zik algorithms to form a strong.
Using ensemble methods allows to produce better predictions compared to a single model.
So in case something happens, filming the event, combining Swanzey algorithms or small weak monitors
to form a strong.
And it will allow us to produce better predictions, come back to us in this role model.
So instead of creating a large decision party, which is very, very prone to being overfitting, we
can create smaller decision, please.
I'm pulling them together in such a way that they even have a strong machine learning model which will
not really overfit.
So let us go for.
Now, that is this one condition, Vias and Gideons trade off.
Now let us try to understand what bias invariance trade offers.
Bias straight off is when we have a lot of bias, when the data is on the.
Then I'm wondering, does not bring enough and it's loans only from one particular partner and ignores
another important fact, and it is going to be biased towards that particular fact.
So let's see, we have the python suppression that is being featured in one feature makes us know that
the value of the loan, what should be.
Yes, while there is another feature which tells us that the value can be Nolensville.
Now, what happens is that the more activity known from the featured a lot ignores the feature, which
is the reason why the more it makes up usually prediction that this always approving the.
This kind of situation leads to a biased result, that is, it will always make a prediction as the
norm to be approved, but actually the law would have been.
Injected in this case.
So this is all bias when a model tries to capture only one type of fighting, it is highly biased and
this media needs to understand that this the model is able known only one type of fighting from the.
Another situation is when the data has high readings of more morning, they have high ratings if it
will try to capture all the pythons from the data.
If a more price Bryzgalov from all of the violence, from the data and even loans, small conflicts
of these from the data, then the data does not really stays generalized enough.
So now the more data is not generalized enough that this is sort of creating a straight line for the
question.
It draws a zigzag line to actually captured each and every data point, which actually leads to overfitting.
So this is called Hibernians, when the model tries to learn a lot and extra from the data point.
Now, what happens is when the water starts to try to unlock, the bias decreases.
So here, the orange line depicts the bias, so as the model starts to learn from the back, then the
bias starts to decrease.
And when the morning starts to learn extra things, it starts to get over overcrowded and the variance
increases.
So because of this, when the bias is decreasing, the ratings is actually increasing.
Similarly, when the variance is decreasing, the bias is increasing.
So an awful lot more than it would be when there will be no high bias, I know high ratings, a point
in between these will be an optimal balance of bias and ratings.
Hence, the test error will be minimal when the via's I'm valiance both are in an optimal level.
This is what Via's various trade office, that is when the bias decreases, the variance starts to increase
and then the variance decreases, the bias starts to increase, and we want to find out an optimal balance
between the bias and variance so that we can make good predictions and have a generalized model which
does not have a bias towards a few features or does not try to capture the extra information or even
noises from the data.
Now, this image perfectly defines how bias and variance looks like.
So here in this image, you can see that the actual target value is the red one, but because the model
is biased, it tries to predict something else and it completely misses the target.
This is called high bias.
That is, the model is not able to identify the correct target.
Now, next is high variance, high variances when the model actually knows what the target is, but
the values are so trying to capture the noise and different complexities that it does not hinder dog
but gets scattered around.
So the value all this keeps on deviating from the actual target value.
Sometimes it will be less than valuable and sometimes it will be greater than good value, but it will
be really hitting the actual target value.
It will be really predicting the actual value.
This is all Hibernians.
And the optimal solution will be no ratings and no bias when the answers would be detailed and will
not have a lot of variance, that is they will be to the point.
So what we want to achieve is no bias and no ratings.
Now, let us discuss about ensemble learning methods within what we call making them together to create
a stronger, more like we still don't know what ensemble is doing in.
So let us try to work this time about ensemble learning based on the ensemble learning methods that
we have.
So bigging is the forced ensemble method.
What bigging does is that bigging will create small models, so this model, one more to do, more than
three will do for more than five.
These are small modules which will be created randomly from these models.
We will be bringing them individually and then we will make them wards.
And from there, what we will decide the majority, what will be the answer or the average of the values
to be the answer of the problem?
So the verdict about you will be the average or the majority walk from these five evenly created models
because these models are completely unrelated with each other.
So they can generate these models fairly and then simply combine them by taking a majority from them.
Next is bigging.
Banking consists of building different battery models, so in case of bigging, we will create a different
parallel models and each of these models has a different set of input sampas, which has to be a unique
one.
Now, think about a simple decision.
Now, then we will be creating a decision if I will provide the same input value and the same output
value and the parameters which I am giving to it.
That is their next of the tree.
If all of these are given they by.
Then the mortgages which will be generated will be.
So if the input data is safe for a you for decision, then the water which will be generated will be
the same as the other decision trees.
So if we are taking wood, we are taking a majority vote or we are taking the average of the results
from these decision trees, then it will be of no benefit.
It is something like, let's say we have five friends and all five friends have same opinion.
Then the answer, which will be will be getting would be same as before Streambed or so, there is no
need to have all the five things asking and giving us the answers if the answers are going to be the
same always.
So we want to have friends with differing opinions so that they can give us different ideas and different
ideologies so that we can have a better outcome.
This is the reason why we will be building these modules by visiting datasets.
They will be creating these models by selecting the data, the me, I'm setting the date that I them,
so that the models, which are generally they're not unique models.
And then the result which is generated, it is generated by taking the average of these positions predictions
or by the majority of them.
Bigging is also useful when we want to create a model by decreasing the variance, by keeping the bias
seem so bigging is usually used when we have models which are highly overinflated in nature.
So when a model is highly overrated in nature, then we want to decrease the variance error, which
is why we will be using the bigging method which decreases the variance while keeping the bias in.
It works this way because bigging is kind of averaging technique, so it will take up all the different
values which it will get and all the variance which will be present in the different models, it will
average that out and get a single target value.
So.
When we have high ratings, the data points are scattered, but the bias is perfectly fine.
So in this case, we will take an average of these values which will allow us to get the accurate answer.
That is why we use bigging method, because it averages out the variance and reduces the variance without
reducing the bias for the.
Then it does not help much with models which have high bias now when we will apply of the bigging model
where we have high bias.
So in that case, the averaging will be that in this particular location will not actually improve the
accuracy because the accuracy will improve when the values will move towards the target, when the value
of the bias will reduce.
So as bigging in order to reduce the bias so it cannot be applied to such kind of problem, it can be
applied to a problem where we have a high variance issue, not where we have a high bias issue.
Next type of method is boosting.
So what is boosting is a state, they are creating sequential models.
So first we will create a model one, then we will create model after that model three, then more before,
then more than five.
Now, what these models will be doing is all of these modules will get X value as an input.
The first model will get X and Y as input, and it will try to predict the value Y.
But the model being a VC model, it will not be able to identify all the patterns and it will simply
predict something which is not really close to Y.
So the prediction which will be made by this model, one late in the Vivan.
So now that EDAR, which we will be having from the model one will be via minus Vivan, which is the
actual value, minus the predicted value.
So this is the error.
So for the next model, we will be predicting we will be getting X values as input again.
But this time, instead of predicting Y, we will be predicting the error value, the value, which
is the difference of V and Vivan.
So now we will try to improve this model by simply predicting the difference between the actual value
and the value which we are getting from this particular model.
So we will try to improve that.
So what will happen from what we do, we will get another prediction.
I'm that prediction will somehow improve the.
The value more so now, what is the error, which is now the error, which is left this value minus
Vivan plus vital?
So there is a little lesser amount of error left.
Similarly, Model three will try to predict this value minus violent, less vital.
Then Model three will also learn a little more.
A new pattern will be learned by more than three, and this model will again improve the.
This prediction, I know the editor will be further reduced to by minus Vivan, plus by two plus white
three.
This same process will keep on going on and finally, we will have a model which we have via minus Vivan
plus why do plus, why three plus four and so on, all the values still this entire film becomes.
Now, this entire town will become zero because we will be improving the model one after the other.
Now let us see the difference between buying and losing in case of buying the models I created by Lily.
Here, the models are created sequentially.
In this boosting model, each model is trying to improve the previous model.
In bagging, these models are independent and do not have any relation between each of them in bigging
Veiga Arbitrageur majority vote.
In boosting the right to predict the error of the previous model, so each model is trying to predict
the error of the previous model.
What does bagging do, bagging tries to reduce the variance of the smaller model.
And what does boosting do Woolston will that I do to reduce the bias of the models, let us know more
about boosting.
So boosting consists of building different sequential models one after another.
Each model has seen X as input, and the first model predicts the value Y.
Then after that, each model predicts the error value left from the previous model until the error is
zero.
Now, this particular value will be down to zero.
Now, this model is actually used to decrease the bias and building a strong predictive model.
So they may sometimes overdo it on the training data.
Now, look at it.
We are trying to reduce the error more and more.
And so there it will try to create a finite number of models so that it can have the added value as
well.
But while we are reducing the error value to be zero, we are somehow moving towards over.
So.
In this particular.
Figure, you can see we have high bias, so when we are using the.
Boosting method price to improve the value of the prediction slowly so that it moves towards the target.
So for each iteration, boosting a the weight of the samples.
So that some folks that are misclassified by the ensemble can have a higher rate.
So what will happen when they are making some prediction and there are some values which are misclassified
by this model?
So the next model will make sure that the values which were misclassified by Model one have a higher
rate in this particular situation so that this model will actually try to improve those wrong predictions.
So the samples that are misclassified, the ensemble can have a higher weight and therefore a higher
probability of being selected for training in the new classified.
OK.
Now, Bargain will mainly focus on getting an ensemble model with less variance then its components,
so Bargain will try to reduce the variance by boosting and stacking.
We mainly try to produce a stronger model, which is less biased than the component.
And even if the ratings can be reduced, then it will try to reduce the output.
So boosting will try to reduce the bias and variance both, but majorly the bias.
While Beijing will try to reduce the variance with the Vyvyan keeping the.
Biase.
Now, we just stated an algorithm stacking here, but we don't know what this is.
So let's discuss about stacking.
So what is stacking?
Stacking allows to create a linear combination of multiple nonlinear models.
So what are known models and what are union workers, we discussed both union aggression and logistic
regression.
So both of these models are actually linear models.
And when we talk about decision three or random forest or boost or bragging or boasting algorithms applied,
these algorithms are actually called non-union models.
Any model which is trying to create a non-linear relationship is called the nonlinear model.
So stacking creates a hierarchy of models using the outputs from the previously.
So stacking will try to combine different models.
So how does that we will look at that after some time.
So first of all, we need to understand a few important things before we actually dig into the bigging
and boosting, mordent.
So let us understand what these modules are and what we know nurser.
So we are talking about the models which are being used here, these small models, which we have just
discussed in case of bagging and boosting, these are called the base models or cloners.
Now, what are these, the building blocks for designing more complex models.
And they do not perform well because they have either high bias or too much variance, because these
are very basic and very small models.
So they will not be having any complex nature and hence they will either have a very high bias or they
will have very much variance.
So the ensemble method will try to reduce the bias or the variance of such cloners by combining several
of them.
So we will take a lot of small models which will have either small high bias or high variance, and
then we will try to reduce the variance or reduce the bias by using boosting or bigging method.
Now, these small modules will be helpful because they will help in creating stronger modules or ensemble
modules that achieve better performance.
So when we create a single strong model, what happens is that the model might it or it might try to
capture the complex pattern from the data.
But we do want to capture that small, such strong patterns.
And hence what we do is we create one vehicle owners or one based models, and then we try to combine
them using the informal learning methods which actually allow to reduce that bias or variance which
those small models have and does not actually cause the problems.
Which one having we were having for creating such complex models, which did not work that.
So we have a better option of using an ensemble aloni, which allows us to use small models which are
very less complex in nature and easy to create and simply combine them so that we can get strong models.
Now, what more can be, oh, the clone of a linear model, which has a very high up and nobody can,
the vehicle owner, for example, a linear model which has very high or very high end range applied
to it, can work as a vehicle or another vehicle Linnean model with a subset of variables.
So a linear model with not all the features available, but which is created by only a few subsets of
variables, can be used as a vehicle.
Or we can use a decision tree which is very shallow or is just a stump.
That is the depth of one.
So we can use that.
Now, why do we use the cloners, we you use vehicle owners because they cannot learn the Netsch five
didn't handle it, hence they cannot overfit.
Now, a combination of these Viglione has been captured on General Bigman.
So let us try to understand this.
So let us consider a decision three, which we are creating now, if a decision tree is a strong decision
tree and it has a depth of legacy of six or seven or let us say 12 in that case.
So this decision tree will have learned from a lot of features and it will have also learned from the
noise which is present in the data.
Now, because the decision tree has a high depth, so it will be having sort of bias or certain variance
to it, it will have some amount of data which it will have learned from the noises.
Now, the noise will allow the model to actually make wrong predictions.
But when we create small quarters, what will happen is that these small V models will not know about
the complexities of the data.
It will not have long the complex patterns of the data.
So the vehicle owner will have learned a small, generalized patterns.
Now, when we have a hundred or five hundred such vehicle owners, they all would have learned something
different and still something which is very similar now because the models have learned things which
are similar also.
So.
It will allow the model to actually create a generalized model because they all would have blown the
prominent features from the data but neglected the impact of the noise which would have come in the
final decision because if they were.
So a very large decision tree might have got impacted due to some extra features, some noisy features
or due to some noisy data.
But these smaller decision systems may have got impacted, but not all of them would be impacted.
So because of that, because we will be taking an average and because we will be taking up a majority
vote.
So those small wrong learnings which these models would have done, they will be completely neglected.
So because they will be neglected.
So the combination which will be launched would be a general factor.
So as a result, it will have not come by the noise.
It will not have combined the noise and it will be ignoring the noise completely.
So this is the reason why we use the cloners, because the vehicle owners will not learn a lot from
the noise or a lot from the features.
And when we combine them together, they kind of diminish the impact of the noisy features or noisy
variables.
In next session, we will begin with the implementation of bagging and boosting algorithms, so we will
learn one example of bigging algorithm and another example of boosting algorithm.
So let us look at that in the next.