In this session, we would implement stacking, so for stacking, we will be using different non-linear
models in the first tier and we will be providing a linear model in the second year.
You can use several layers in a stacking model and decide upon yourself how you want to stack.
Stacking is completely a creative thing.
You can try different combinations, you can try different things and create your own model using stacking.
So let us begin with the first thing which we will be doing is we will import find us a number and we
will import of whatever columns, whatever libraries we want for the implementation.
So I will be using the nearest neighbors of I will have random forest and logistic regression.
I will be evaluating it using oracy uses school and for modern selection, I will be using full time
frame display.
Next, we will be using the census income data set, which we have used previously also in this dataset.
We have different values as greater than five or less than equal to five.
And based on the details, we have to find out if we have to keep or if the class has to be greater
than equal to five fifty thousand or it has to be less than 50.
We have already seen the data set.
So let me show you the data set again.
So let me run this.
And let me add the.
So this is the data that we have.
And here we have converted the vehicle.
Next, we will be deleting the education column because it contains same information as the one presented
in the Ordell column, which is education.
No.
After this, we will be creating sub columns using the categorical classes like Worldclass, finally
we marital status, occupation, relationship, race, sex, native countries.
So these are the columns which we will be converting from categorical column to dummy columns.
So these are the steps associated.
So first we have deleted the education column.
After that, we have selected all the columns, object columns and from these object columns we have
run.
Allu.
This logic's checks, if a particular category contains more than 99 values, then only it will be converted
into the variable.
Otherwise it will skip that.
You can choose any frequency based on your dataset, based on the number of values present in your dataset.
And after that, we have made this string split and retrieve the dataset.
So we have got the train and the test data set and the CD train would be divided into extreme and vitrine
further by dropping the volume and keeping only the right column.
And similarly, the test will be divided and do X test and write this data frame by removing the Y column
or an.
Keeping only the white collar.
These are the columns which have been converted into categorically from the ordinary classes, these
categorical classes have been converted into one hot and Cold War dummy variables.
Next is the list of classifiers which I will be using.
So these are different algorithms which I will be using.
The first one is gay neighbors classify it.
Next is the random forest, then X to be classifiers.
These are the details of the same one is having 50 neighbors.
Next, I have created random forest with two hundred meters and balanced with another random forest
with hundred estimated.
And then.
It's to be classified, I've created both with one of the investigators and the learning rate being
zero point one and zero point zero one.
And this is the list of algorithms which I have.
Next, I will simply use these rules of the day I have created counted the number of rows of detail
present in the training data set, so it has twenty six thousand and forty eight rows of data.
Next, I will create a layered one, this layer one will contain.
All the zeros with.
The number of roads being twenty six thousand forty eight.
This is the data, this will basically contain the.
Predictions which we will be making from year one.
So we will be training more those on X train and by train and then making predictions in the scale of
one, two, three, four and five using the field method.
And the full method will allow us to have out of box predictions in this particular dataset so that
we can further train this on a linear model.
So the next thing which we are doing here is we are creating giffels, so here we are creating then
split out of the total data and we are beginning with the fall to being one full value, being one.
And out of the Kiev split, we have taken the a number of Rose does, so this is the best efforts we
have taken in it and we have taken out the train and left out junk from the Kiev split.
So give the split will basically create folds and it will create thin folds out of the entire dataset.
So it will divide this entire dataset into training dataset and left out a chunk.
Now, what we will be doing is it we are bringing the full detail, the number of fold, the if it is
the first or second fold or which fold are we running this for?
And further internally, it will move on all the algorithms that we have.
That is these five algorithms which we have with us.
So this law will loop over the split and over the illegal items, so it will be called the dream data
on the.
And it will start and get the.
Of value and the value of the training data set, and then it will decide upon the left junk that is
the size of the junk and based on the left out junk, it will keep an eye on the entire data set.
Then it covers all defense split.
And finds out by fitting the algorithm and predicting the probability of of the left outcome.
So it will drain on the extreme drain and white brain drain data.
And predict the extreme left out junk values.
And after we have predicted these extreme left out junk values, we will fill the little one data frame,
which contains these blank values.
So as of now, and we will fill in the left junk part of the details name the the the values.
That is the values which we have predicted.
And then we will obey the value of fall to fall through so that it will again loop over the second fall.
So first of all, it will consider the first world as the best value or the value which needs to be
predicted later.
That is the left out.
Then it will consider the next world as the leftover junk and so on.
So it is keep a note of default and then work on it.
So here it is working on the first hole and predicting the values for the five.
Algorithms, then it is.
Going over the second full and.
Again, working on the final verdict, so this it will be doing for the oil then folds, which we have
in the data, and here we have all the predicted values which we have from the.
Folds, would we have run?
Now, what we will be doing is now that we have all of these values, we want to create the now this
layer will be three on top of this entire data.
So now this becomes our values and the value remains the same.
So now using these values, we will apply a logistic regression.
On top of this, you can apply linear regression in case is a regression problem.
But because the problem which we are solving is a logistic problem, that is if value is greater than
equal to 50 or less than 50.
So that is why we are applying logistic regression.
So we will consider these as the X values for the next clip.
So what we will be doing is now, again, we are considering later to decide which is taken from this
particular date frame.
We have combined this entire thing.
And now for layer two, these values become zettl values.
So we have created a new data frame layer to this which will contain zero values.
And these zettl values will now be fulfilled by the predicted probability, by the.
By the model which we have created here, so they'll know what we have done is.
We have voted on all of these forms and from all of these words, we have predicted the values here
and now what we are doing here is.
Here we are again, creating a model of the debris where we have zero values and we're vibrating on
the algorithm and we are generating the layer to debate.
Now, in case of little, we will generate a logistic regression and this logistic regression will be
fixed on layer one.
And this little one will now be this model, which has been generated by fitting the model on only one
will now be used for predicting the probability of the layer to be.
Now.
Once we predict the probability of layer two data.
Then begin for the.
Check the probability of it, so from the data, we can see that the accuracy score comes out to be
92 percent.
Now let us apply a simple classifier.
So this is what needs to be classified, which I have for.
So this has an estimated one hundred law needed one, and this classified unfitting only one and little
and wide data, it gives us the same accuracy, which is zero point nine.
So here you can see that instead of applying all igby classified, we have applied a very simple model,
which is logistic regression, and it is also giving the same accuracy.
That is not much difference in both of these.
So we can use any of the models.
We can either use a simple model or a linear a linear model or nonlinear model or in the second layer.
So it is completely up to us what we are choosing to implement here.
So this is about the stacking implementation, so all you need to take care of is that the data which
you will be using needs to be done properly.
And after transformation, you will simply applied layer one very thin layer when you will use different
nonlinear models.
And in year two, you will use a linear model.