In this session, we will discuss about the third informal learning method, which is stacking now,
then all we have discussed about a lot of supervised learning methods, all of these supervised learning
methods perform decently at their own level.
But sometimes the problems are so complex that one cannot really capture all the patterns which are
available.
So for this particular situation, we have a solution for stuck in this solution, what we actually
do is we pick out some trained methods, some green algorithms, and create a linear combination of
these and partitas.
The combination of these algorithms actually helps us in getting a better outlook, in getting a better
performance from these markets.
So let us see what stacking helps us in doing so.
When do we actually choose?
So in case we want to decrease variance, then we will use -- in case we want to decrease the bias.
In that case, we will use full steam.
Now, in the in the case, we want to improve the predictions, then we will see to stacking Alberto.
So what is stacking, stacked generalization or stacking for short is an ensemble machine learning algorithm
and combining the predictions from multiple machine learning models on the same data set.
So we will train multiple machine learning algorithms.
So let us see.
I have one linear regression.
I have one random forest, I have another she to another SVM model.
And all of these models are performing decently on their own part of the data, but they are not able
to capture one or the other part of the pattern that is present in the data.
So in that case, we will combine the powers of all of these good performers and get a very strong performer,
which is called Ustad Model, which will be created by combining the powers of all of these algorithms.
So this is used to explore a space of different models for the same problem, learning with different
types of models which are capable to learn some part of the problem, but not the whole space of the
problem.
So what we will be doing here is we will take these different models and create a linear combination
of these models so that they will perform better from the other models, from these individual models.
So the models to be stacked should be non-linear in nature and the models are combined using a linear
method.
So we will be choosing several long nonlinear models such as SBM vs Kinen or Qaboos, Random Forest
and these nonlinear models.
We will combine these nonlinear models using the linear method.
That is, if we have a classification problem, then we will combine these using logistic.
If it is a linear problem that it is a regression problem, then we will combine these using linear
model.
So this is the entire process, so we will let us have our training data, which is engross in that
is the number of rules and is the number of columns.
So we will train these four modules or five modules, any number of models of our choice and train these
models on this particular dataset.
Now a new training set of second level models consisting of the prediction from the first level of models.
Now we will have some predictions from these models.
So this model will let us say give I had one.
This will give by two, by three and I had four.
Now we will be predicting we will create a logistic or linear regression on both of these very high
values, which we will obtain from these models and try to reduce the error on on these by actually
predicting the values from these very high values.
So at the first level, we try to predict the values from the input being this enclose in matrix that
is, this includes NBPA.
Now, the input will actually be the for the second layer, the input will be the output, which we
will get from these four moments.
And then we will try to reduce the error from these forward monitors.
Now let us see.
Still, it is not performing well, then we can introduce another layer and that there will actually
be trained on the output of the second layer so we can have any number of layers present in the stacking.
So based on these levels, we will get the final prediction.
So what is the process, the process is that the initial training that X has and observations and features,
so it is a and so there are different models that are trained on X by some method of training like cross-validation
beforehand.
So we have already trained some different models.
Now each model provides the prediction for the outcome.
Right.
So we will have some very high value from each of the models.
Now, these are forced into a second level training data and which is now across in size.
That is M number of rows I am wanted.
Now, the M predictions become the feature for the second level and value remains the same.
So now we will apply on the second level of model on top of it, and then they could further.
Now what we can do is if the output does not surprise us, then we can apply another model on top of
it so we can do it that way.
Now, after the second level, which is a linear model, we can again apply all non-union model on top
of it so we can try different variants, try different combinations and get us that model over.
Now, how do we do cross sampling now, the stocking will be using a similar idea as to the key words
cross-validation to create out of sample predictions.
So if we were to use predictions from the models that can fit through all the training data, then we
will not be having any out of sample data to bring the second level of the model.
So for this, actually, we want to have us out the work sample.
So that is why we don't want to be biased towards these models.
So we will also need to train the models on T minus one foot so each model will be trained on legacy
for food and prediction will be made out of this food so that we will have an out of box prediction.
This will allow the predictions to be out of something and useful for the beginning of the next year.
So let us see how this would happen.
So let us say we have created three for here.
So what we can do is we can bring the data on legacy first two blocks for us to fold and we will train
all the models on the force to force and make a prediction on the third one.
Now, again, we will train all of these models on the next two four and make a prediction from this
green one.
Lastly, we will bring the model from the green and red Ford and make a prediction from the blue one.
Now, the predictions which we will be having here from the Models X one, two, three will be of if
one F2, F3, these are very high values, which we will be getting from these models.
Now we have these very high values.
These very high values will actually test the X values for the linear model, which we will be bringing
in the next level.
So these will be these will go on in two into the X values for the next model and this will go into
the value.
So we will bring a linear model on top of this particular data and get the output.
So like this, we will be able to get the stacking implemented.
So we will have a look at the code of this particular implementation in the next session.