High in this session, we will revisit random thoughts and discuss about the issues with decision trees,
how we can implement random forest and different details about the random forest and what else we can
actually gain from random forest.
So let us have a look at it now.
What are the different issues with decision?
Trees, decision trees help capturing the nation on patterns in the data.
That is the modern basic training data very well, but may not do well with the newer dataset whenever
we build a particular model.
We wanted to do well with the future data.
Also, we don't just want to learn from the current data and predict the current values properly, but
also to predict the new values, the unseen values better.
So hence, we need the model to be generalized in nature, that is, we don't want the model to be overfitting.
So how do we really handle this?
So on one hand, we want to capture the decision through this amazing ability to capture not only new
data and I'm the same time we do not want to be susceptible to noise or the need patterns from the training
data.
That is the decision tree, because when that decision will grow longer, what will happen is that the
decision tree will try to learn from the noisy or the noisy features with which it has.
But then we will have random forest.
What will happen is because we will be averaging out the votes from different decision trees.
So in that case, if there is a particular variable which has some noisy data, or in case there is
some feature which is noisy features and we don't really want that feature.
So what will happen is the impact of those particular data points will get diminished.
So it will, in tone, perform better then the decision.
So it is a very simple and powerful idea which resulted from the popular vote, some for the random
forest where we use the bigging algorithm, which is present evidence for Malony.
Now, what do we have in random forest, the random forest introduces two levels of randomness, so
the we know what forest stands for in random forest.
The forest stands for the multiple number of trees.
So in random, what is we will have one hundred, five hundred or thousands.
But the randomness, which is the name, the randomness is actually included by two levels.
That is the building process, which will help in handling the noise of the data.
Now, what are the two types of randomness?
The two types of randomness is that it stays on them subset of all the observations in the training
data.
That is the first level that we will be having the entire data, I'm considering all the data points,
we will consider only a subset of the data points.
That is a sample of the data point, which will be used for creating one of the three.
So we will have hundreds of these.
So each three will be created from a different subsample.
The second one is it will also take random subset of all the variables from the random.
Training data.
So now what will happen is that initially what we did was we did a sample of the entire data.
Instead of taking all the data, we took only a part of the data for each three to be constructed.
The next randomness which we introduced was by taking up subsample of the.
There is also so we have a subset of the dollars and we have a subset of the soldiers on active duty
also, so we are not considering all hundred or twenty variables, which we have.
We are considering only a subset of the variables at the time so that the rules which will be generated
will also be a subset that will not be all rules will not be created because for all the rules to be
created, a lot of variables have to be present.
Right now we have a subset of variables.
So now the rules which will be generated will also be lesser in number.
Now this will help in creating different decision parties.
If they had all the variables, then it will automatically be able to find out the best split.
And the best split will any how come to be seen if Jenny was calculated, so because we have taken a
subset of variables, the guinea value will again be different for different for these different split,
which will be competing created by comparing different rules.
So now that isn't three, the levels in the decision, three will again be different.
So one decision three might start with the age as a rule and the decision three might start as number
of children as a rule, because now the variables might be missing in one or the other doesn't.
So in order to understand this, let us assume that we have ten thousand observations and the 100 variables
in the dataset in random forest, many decision trees are buried.
Instead of just one single tree, for example, it is considered a 500 trees are being.
But now if it were these trees using same ten thousand observation and 500 times, we will end up with
the same 500 trees.
So now we will subsample the data and we will subsample the number of variables.
So instead of considering all the thousand rows of data, we will be glad to see a thousand or five
thousand rows of data for each tree.
Or let us say we can take only two thousand barrels of data for each and for the variables.
We can just let us say one hundred variables at one presentation or say twenty was what each generation.
This is how we can actually create randomness in our trees.
So this will in on that every node will have a fresh, random subset of variables.
And from this, the best tool which will be picked up will be different from the others.
Now, what will happen is because earlier the large decision tree, which was being created, it have
a lot of noise and it would have captured pythons, which didn't really exist because of the presence
of the noisy that was already noisy data point.
So now it does not capture that because anyhow, it will have to be average value.
It will have a bigger majority now because some noisy pythons will not be present in all the trees.
The the the diminish impact of the noisy neighbors will occur.
So the noisy variables will not show up or will not have high impact on their decisions which we are
making.
Hence, the performance of the random forest will be much, much better in comparison to that of the
decision.
So in short, the first randomness to remove the effect of these noisy observations and the second randomness
will remove the effect of the noisy variables.
So the final predictions made by the random forest model will be a majority vote of all the trees in
the forest in case of classification and for regression.
Again, the prediction will be the average of the predictions made by the five hundred decision trees.
Now, once we have discussed all these letters, go ahead with the implementation of random forest.
Now, what do we have now?
We will build the Diamond Forest model.
So for this, we will use some hyper barometers for the decision trees, amongst others.
Since that time, the forest is ultimately a collection of the decision.
So we will use all the hype meters, which we have discussed earlier.
We have discussed certain hyper parameters which were present for their decision tree.
So we will use all of these hypovolemic as a bathroom, that we will have some additional hyperbola
meters, which are an estimated maximum features I would strap.
Now, what is an estimate and estimated is the number of trees to be buried in the forest.
So the default value is then and the good starting point can be Hundert, so we can have a little bit
of value as hundred or let's say one hundred five hundred thousand seven hundred two hundred.
These would be different values of an estimate and then we can decide after using the model.
Which number of value, which an estimated value actually gives a better result?
Next is.
Maximum features, so now the maximum featured is the number of features being considered for selecting
the best at each split.
So the value of this barometer should not exceed the total number of features available.
So the maximum number of features should be kept up as a subset of the total number of values.
So initially, the maximum number of features, let's say we had two hundred features, right?
So it maximum the value could be two hundred.
But we should keep the value something between say one hundred one fifty fifty.
We should try different values between these so that we could find out the optimal number of maximum
features.
Next is step now bootstrap allows for sampling with replacement or without replacement, so it takes
a boolean value.
If it is true, then the sampling will be done with replacement.
And if it is flaws, then the sampling without replacement will be taken.
That is, the sample would be taken only once.
That is a particular value will be sampled only once.
So no rules of detail we would be doing when we will be sampling the data.
Now, let us begin.
So what we will be doing is we will import the Iron Dome for this classifier from the Escalon ensemble.
Next, we will create an object.
Cliff of the land, what is classified?
For what we will be doing is we will create the dictionary.
Now, this dictionary has different values of parameters that will be trying to figure out the model
giving the best performance.
Now, apart from investigators maximum features and bootstrap parameters which are specific to the rest
of the hippopotami, those which have been chosen, these are the hyper parameters which belong to the
decision tree.
So this is is the base classifier.
So we will be using the base classifier and we will be passing these hyper parameters to this.
Now, you can choose any of the values that is used, either randoms or TV or TV.
So it is completely your decision which method you want to use for this.
And I have already explained random.
So it's even good TV separately.
So you can go to that particular video and decide yourself which method you want to take.
Next, we can simply get a list of bottom distributions, you can use this extensive list of bottom
feeder distribution or create your own list of parameter distributions.
I will not be running any in in the sessions because the more fitting takes a lot of time and that cannot
be considered in the video recording.
So I have simply skipped the training part.
So you can simply run these trainings.
And when the model is training, it might take a lot of time.
So you need to be patient for that time.
So for this particular combination, there are around six to nine thousand one hundred twenty combinations.
You can see there are probably nine sixty six and six in between two combinations present.
So we will not be doing all that, we will be fighting only a small little combination.
So what we can do is we can simply the run the estimation from the bottom of the distribution.
So let me remove this, this is not needed.
So you can simply run the estimation and then get the best classifier from that, and after getting
the best classifier you can, simply put the values of the best classifier inside the line of what is
classifier.
And after that, you can do the odd, if not for.
After doing a lot of good faith, you can obtain all featured importance now.
What is feature important, feature important is.
In Ashkelon, one of the ways in which we feature importances described is by something called mean
decrees impunity, that is this mean decrees impunity is the total decrees in the north impurity averaged
over all the bodies buried in the random forest.
So what this will be doing is this will basically allow us to compare all the trees so it will compare
all the trees and from all the trees that it has come back, it will find out the average of the impurity
reduced.
So for each and every feature, it will compare how much impurity this feature has, of course, to
reduce and then based on the maximum impurity did used.
By a particular feature, it will be sorting that and providing the same as the feature importance.
So if there is a big degrees, an inability, then the feature is important.
If it is not important, not in.
Now, in this particular report, what we are doing is.
The idea of banning the feature important attribute.
So the use of this ad will make sense only after the model has been --.
So we store the future importance along with the corresponding feature name in the frame and then sort
them according to the importance.
So here you can see.
The different values, so you can see that the importance for marital status is zero point one, two,
three, the importance for capital gain is one zero point one and one for education number is zero point
one five six six.
So we can see different important values so we can observe that the marital status of the spouse is
identified as the most important variable while the column.
Marital status underscored separate is of the least important, so random forest can also be used as
dimensionality reduction technique.
So we have discussed about the last regression where we actually used the feature, the importance of
being from Lassalle.
Similarly, we can use the random photos to obtain this, feature the importance and then remove the
columns which are of less importance from this particular list, which we have got.
So in the case of high dimensional data, we can run on a random forest and get the features sorted
by importance and we can go ahead and choose the top three hundred or twenty or twenty five features
for for the processing.
Now, using this technique, we would not be losing relevant information because the relevant information
is present in these columns on the.
So at the same time, we will be drastically reducing the number of features considered.
So what we can do is we can let this be considered, though not the top 15 or 20 columns and get rid
of the.
The rest of the columns, so if we do not use the line of what is more as the final model, we can at
least use the random forest, what, reducing those features?
So what they can do is as a process begin, first of all, implement the linear model that this minute
regression or logistic regression and once we have implemented that.
We will get to know what columns are important and what features are important, and then we can keep
those features.
Again, we can implement decision three and decide upon which features have higher importance and keep
only those features.
This will allow us to reduce the number of columns.
It will allow us to reduce the number of features.
Now, once we have reduced the number of features, then we can implement Igbos or any other algorithm
of our choice and compare the performance of Random Forest with Boozed and SVM and Navys.
So all of these algorithms can become big and then we can find out which one has a better performance.
In case we are not satisfied with any of these performances, then we can go ahead with using methods
like stacking.
Then we will have a bit dunwoodie so we can use that one, but at least implement random forest online
models to at least obtain the feature of importance so that you will have unknowledgeable for what feature
is actually important, what feature is not important.
So it can be used as one of the methods for dimensionality reduction.
Now, let us discuss about fashion block, so random forest Wandel is one black box, obviously, because
we will not be able to understand how each and every model works inside this, how each and every place
created inside this.
And it will be really difficult to find that out.
So in this, we do not get any officiants what anybody will, and hence we are unable to make any interpretation
of different variables on the response responsivity in case of linear regressions or this regression.
It is a normal model.
It is not a black box model.
We are able to see the video.
We shoot more transparent.
So what will happen is in order to meet this particular line of what is interpretable and to make the
different interpretation of the variables which are used in that, I know what is we need to make a
prediction on the entire data and average it for the variable, which we are interested in.
So, for example, let's consider the case of education in this so we can create the variable name as
education and get the predictions for extreme and get the predicted values.
Next, what we are doing is we are getting the video data that we are just simply concatenating the
variable that is education number and the corresponding response with it.
Now we are simply printing the alleged plot for the variable and the response.
And we have not just for the discussion for this one.
Now, you can see that when applaud the two columns education number against the response as shown in
this below.
It is not very informative in nature.
That is, there are a lot of variations since the response contains the effect of other variables.
Right here we have a lot of dots and we are not really make out what this really means.
So we will plot us, will then go on top of this, which will give an approximate effect of the variable
education and little on the response.
So we basically will average the response at each education, no value.
So what we will be doing is we are simply importing these DOT or API as Essene and we are creating the
essence that dot non parametric Lewis Blätter, which is a small data plot and they are plotting it
using a lot.
So now if you see this is the goal, which we get.
Now, in this plot, we know this, that the educational value goes up, the chances of having income
greater than fifty thousand dollars go up.
So as the value goes, as the education goes up, the response also goes up.
Right.
The education number variable does not have much impact on the response.
Feel the value around the inaudible deal.
Now, there is no much impact on education, no on the response.
But as the education response goes above 11, you can see the response value is increasing very much
right.
There is a steep indicating with the higher education, the probability of owning more than fifty thousand
dollar increases so it can be done for any other variable that we wish to check of, although it does
not make sense for dummy variables since it has only two value.
So we cannot really compare because the values will be only to here to here.
We have eight values present, so it is more clearer picture, but what a categorical variable.
We might not get a clear picture, but it will work really fine for the.
Continuous variables so we can create this financial independence plot also.
So you can create this financial dependence plot and use the feature of importance obtained from the
abandoned forest as it will help a lot for dimensionality reduction.
So that is about the forest.
In the next session, we will be covering the extreme boost, which is based on the boosting algorithm.
So let us go ahead with that.