Hi, Dad.
Now, we have learned about how we can play different roles for categorical and numerical variables,
and we have also seen that how decision trees are created.
We know that the decision trees are based on the different rules which we have, and these rules only
define what decision node has to be placed next.
So now we will learn the next most important criteria, which is the splitting of data.
So each split is made would be aligned to an axis with each feature present an axis.
So what does this mean?
So here this means that each split made would be aligned to an axis where each region is in axis itself.
So let us have a look at this particular data where we have values or we can consider these values being
different classes.
So these classes.
I present in four different regions, so the first class is present here.
This is the second class.
This is the third class.
And this is the fourth class.
Now, if they want to subdivide and try to find out where what class is present, then if we want to
make such a decision, then how we can do that.
So we can do that by creating rules so we can create rules by legacy.
Having one rule as.
It's this being the x axis and this being the Y axis, so we can have the first Ruelas X greater than.
Zero point five, when X is greater than zero point five, then this particular line would be selected,
which means that X is greater than zero point five, then it would probably be a part of Class one or
class three.
Now, after we have decided if X is greater than zero point five, then we will have under the condition
that Y is less than.
Zero point five.
So for tree, where we have X greater than zero point five and Y less than zero point five.
That would mean that the glass is one.
Now, remember, the condition we created X is equal to zero point five, X is greater than zero point
five.
So for the condition, yes, the condition which we just discussed about would be prison.
I'm on the other end, we will have X Leiston zero point five, so now for X less than zero point five,
there are two types of conditions present.
So now we will again have another condition that is X is greater than minus one point one.
So if X is greater than minus one point one and this was the previous condition.
So again, the same situation would come up where we will have to create a split on the Y axis.
So based on the y axis, we will again see that if the value of Y is greater than minus zero point five,
if the value is greater than minus zero point five, then it belongs to Ford.
If it is less than zero point five, then there could be a good idea where it could be to.
So we will get to this.
Now, again, there would be another split at this particular point that is being done, Y-axis again.
So it will check if the value is between minus one point two and zero point five.
And it is less than.
Zero point.
Two or three.
And so it will decide that and based on that, it will decide that this would be the last to.
And finally, when the condition was that the value of X is less than or greater than zero point one
minus one point two.
So based on the negatives, that action would be placed for the oil values being for.
So you can see that each split which we are making is in concordance with the X and Y axis, so each
rule which we will be creating here, will be belonging to the X and Y axis.
So let us see how we can actually select different booths so there could be several booths present.
I could have had a room for minus one point five, Wojo I could have had four plus one point five total.
But these rules will actually not give us any good results.
So that is why we have to choose these rules which have been highlighted here.
So how will we actually get to know that Richard Rule is actually useful?
So for that, we will look for a split which gives the most homogeneous chain.
So when we say most homogeneous try node, we don't really know what homogeneous means.
Let us say we don't know the meaning of homogeneous.
So let us try to think about it and what it actually means and let us formulate.
So the different ideas for rule selection are the guinea index, entropy and divisions.
These are three formulas which are used for creating the rules and selecting the rules.
Now, we still don't know how they actually work, but we will look at it in something.
So the Guinea next formula is one minus summation of the probability squit the formula for entropy is
minus probability of probability and the low probability.
So similarly, we can have these formulas in hand and then use them how we will use them.
We will look at it in some way.
So for that, just wait for a moment and let us first of all, understand terrifically what this means.
So let's say we have this entire dataset.
This data contains four glasses, these four glasses of blue, green, red and yellow.
Now, out of these four glasses, I have equal amount of data presenting all the four plus.
Now, the problem here is that I want to decide that what is with.
So let us try to do that.
So for this, I need to create a split in this particular data so that I will be able to recognize what
glasses, glasses it.
If if someone would ask me which a place, a point they were and ask me which point would be that particular
class, I might not be able to answer based on this particular image.
So I need to make a split on this.
Image.
So let us make the split.
So the first bridge that we will be making is the horizontal strip.
So when we make this horizontal split, you can see that data has been divided into two portions.
One is the top portion and the one being the bottom portion in the top portion.
We have us some part of red points, a few blue points and all the yellow points present while in the
bottom split we have.
Some of the great points, all the Green Point and some of the good points.
So here we don't have a clear majority of any points present.
Now, if we have a look at the split, too, you can see they have made this vertical split, which
divides the data into pool spaces where the left hand side space has only to be the point, which is
blue and green, and that to all of the blue and green points right on the right side, we have all
the red and yellow points.
So if you see in this split is done, then if we try to make another split, then what will happen is.
Let us make fun of this.
So for this particular split, the next to which we will have to make is at this point.
Which will divide the blue and green point and the next split which will be created would be at this
point, which will divide the red and the yellow point.
Now, if we talk about the four split and we make another split here, then the split would have to
be made at this particular point and under the split at this point.
Similarly, once it would be made at this place and another the split.
So now if you see they have to make only one, two and three split in the case of the second split,
they've had to make one, two, three, four and five split in the case of the split one.
So it is declining, lesser number of split in case of split two in comparison to split one.
And also the data is being divided in such a way that we have all the data point of one class present
in one position.
So it is better to make a split two in comparison to split one because it divides the data to be in
a more homogeneous location.
So the data in the two is more homogeneous because we get only two classes on the left side and only
two classes on the right side.
While in the case of split one, we have three classes at the top and another three classes at the bottom.
So hence it is less confusing in case of split two.
If I have to make a decision about this at this particular point, then I can easily say on the basis
of this split, which we have made, that I like this place.
Either it would be a blue point or a green point, but if I make the same point in the four split,
then I will have to say that the point could either be blue or yellow orig.
So the confusion which we already had still exists.
We were able to eliminate only one class using this particular split out of the four classes which we
had while we were able to eliminate two classes with just one split in the split.
So that is the reason why we will to now let us discuss what information gain is.
Now for information, Jeanne, let us try to find out the probability first, so for finding out, probably
let us look at this particular data.
So here we have 12 points of data.
So out of these 12 points of data, we are making a split.
I'm driving this down to two different sub nodes.
So the full split is creating this particular division and the second split is making this division.
Now, let us try to find out the probability of the balance for finding the probability of the finding,
we will let us consider the green point.
So what are the probability of getting the green points out of this baneful?
It will be six divided by the total of 12.
That is zero point five.
Now, let us have a look at the probabilities of both the split so far split one at the left hand side,
the probability of getting the green point will be three by the triple.
Seven is equal to zero point for the higher the probability will be three divided by five, which is
zero point six seven.
For the second split, the probability of getting the green pointis four divided by five, which is
zero point B here, the probability is two divided by seven, which is zero point to it.
Now, let us try to find out the probability of getting the Redpoint out of this block.
So that would be five divided by seven, which will be almost 72 percent.
So you can see that when we are trying to decode values from the first split, the four split is less
homogeneous in nature because this is having more mixture.
While in case of the second split, the values are more towards more bias towards a particular class,
and hence we can easily make out that this is a great class study, this is a green class and this one
is a red class.
So when we pick out the point from this particular portion, it will always give out a green point in
majority.
And then we decode points from this particular split.
It will give out more red points.
So hence it being more homogeneous in nature.
We will consider this split as are actual and correct split for the next splitting.
So let us find out the information game using guinea pigs.
So what was the formula for Guinea next?
The formula for guinea pigs was one minus summation of probabilities.
What you can use any of the indexes, you can use Guinea, that you can use entropy, you can use any
means.
But for now, we're explaining it using the Guinea index.
So let us calculate the value for the Fed, including.
This note will have Ghneim Index as one minus.
Six by 12, all squared plus six by 12 square.
So we will be taking the proportion of each class in consideration here.
So it will be one minus six by 12 of the Greenglass and one minus and plus six by 12, all of the grade
class.
And when we solve this, we will get zero point five.
Now, let us get the guinea values for the full split.
This is up for a split.
So the Guinea index would be one minus.
Four by seven.
Plus three by seven all square.
So it comes out to be zero point for evening.
Now let us guys wait for the light inside of the four split, it will be one minus two by five.
Plus three by five.
Holds square and it comes out with zero point forty.
Now we will calculate the weighted average of the game.
Now how will we calculate the weighted average of the weighted average will be calculated by taking
the left hand side.
So how many points we have here?
We have seven points here and in total we have 12 points.
So it will be seven by 12, including gaining of the left hand side, plus the five points here.
So five divided by the of 12 in the beginning of the right hand side.
So it comes out to be zero point forty six and the game comes out to be zero point five dollars million
of the beginning of the BEREC, minus the beginning of the child, which is zero point one for.
Now, let us calculate the need for these second.
The beginning of the second split comes out to be for the left inside the guinea will be one minus four
green.
Divided by the total faith and the whole square of this plus.
One read divided by the total five and one square of this, so it comes out three zero one three two
zero.
Now for the right insight, again, one minus.
Five divided by the total seven plus two, three divided by the total seven and the whole square of
this, so it comes out with zero point four zero eight two.
Now, again, we will take the weighted average.
So this campaign's five out of 12.
So five by 12 into the beginning of the left and say zero point three to zero and plus the.
Seven out of the 12 Endou zero point four zero eight two now we will and we got some of this, will
we.
The two point three seven one.
Now, the gain value will be zero point five minus zero point thirty seven, so it comes out to be zero
point one the week now when we compare the zero point zero one four with zero point one two eight five.
We can easily see that the gain for the second split is more in comparison to the four split.
So we will consider the first the second split for the splitting of this particular pairing.
So we will split a particular node and then we will get these.
So then after getting the split, so once we have split this, then again we will consider how we want
to make the split and which is the best split and again, split this particular node.
So it will keep on going on until we reach a point with each of those conditions, which we have already
set for stopping the splitting.
Now, let us have a look at this particular image.
So here we have the data where we have this these lines of separation.
So let us try to create the rules for this or decision for this.
So the four split, which we can create this X X one less than three point five.
So if X one is less than three point five, then we get this particular region.
And if one is greater than three point five, then we get this particular region.
So let us consider X1 less than three point five.
So the next one is less than three point five.
That is in this particular region.
Then the second condition, which we need to check is if X2 is less than seven point five or not, if
X2 is less than seven point five, then it comes out to be X, otherwise it comes out to be zero.
Now, let us consider the X1 greater than three point five, so if X1 is greater than three point five,
then there could be this particular area which we want to divide.
Now, for this, we need to check if the value is between four four and seven point five.
If the value is between four and seven point five, then we will again check if the value of X1 is less
than seven.
If the value is less than seven and the value is between four and seven point five, then we get this
particular area, which means it is X, and for all other conditions, the market does zero.
So this is how we will create a decision.
Now, once we have got these homogeneous pieces that does this particular space, which is completely
homogeneous and has all the points there is no meaning of creating for the split because the Guinea
index of this particular area will be seen.
The index of this particular area will still be one hundred percent.
And again, if we make another split, it will still be one hundred.
So there will be no information gain present.
So unless and until we have a preliminary information gain, we will not make a further split.
Because if we keep on making up for the split, the three will keep on growing and it will not support
anything, it will just keep increasing the size of the tree without giving much information.
Now, let us talk about regression vs regression trees are what we use in the case of regression.
So in this case, we use the mean and median of the records, I believe not to find out the value,
the average value of the predicted value.
Now, let us try to learn about what regression Tresa.
So for selection of rules in election scenarios, we were using the Guinea Lexar entropy, but in case
of regression, we use the sum of squared error.
So how do we do that?
So we will find out the sum of squared error of the different type of split.
And then find out the minimum split, the value which has the minimum sum of squared error.
So let us consider this particular split.
So this is the original parent or the parent node contains values five, six, four, six, 11, 12,
13 and 12.
Now, if they will protect the values, the predicted value will be the average or the mean of these
values.
So the average or mean of these values comes out to be eight point six.
Now, if we will surprise this eight points, six from the target value, we will get these ordinator
values.
Now we will find out the sum of squared error as described in the.
And the definition of the regression tree, so let us where these values so on squaring these values,
we get thirteen point one seven six point one nine one.
So these are different values of the squared values of the.
And then finally, we are the all of these squared values and get me some of squared error, so what
is almost square and some square in it is we will find out the error, then squared all the error values
and then find us some of these squared error values.
Now we need to make certain splits, and by making the splits, we will have to find out which one has
the lowest sum of squared error.
So this is the first type of split, this four split divides the data such that the target value has
five, six to 11, 12 in the left hand side and 11 six thirty nine four on the right hand side.
So here the predicted value will be the mean of these dot good values.
So for the left side, the mean value is eight point seven five, and for the right side the mean value
is eight point five.
Now we will find out the error by finding the difference between the target and the predicted values.
And then we will square the values and then find the.
Squared error, the sum of squared error.
So now we the add the sum of squared errors of the left and right side of this particular split.
So we get forty two point seven, five plus fifty three, which is ninety five point seven.
And the original sum of squared error of the betting mode was ninety five point eighty seven.
So hence we can see that there is not much improvement by doing this particular.
We have not really gained much from this particular split.
Now there is another split which divides the data such that the target value values on five, six,
six four in the left hand side and 11, 12, 13 and 12 at the right hand side.
Now, again, we will find out the predicted values using the average of these data, good values.
So the average of these would be five point five and four.
These it would be 12.
Now, again, you will find the added value by finding the difference between the target and the predicted
values.
Then we will square these air values and find us some of these squared values.
Now, when we sum the sum of squared errors, we get two point seven five plus two, which is four point
seventy five.
Now, let us compare these sum of squared errors.
So the second squared has a all four point seven five.
The false flag, five ninety five point seven five.
I'm the Fed incommode had a city of ninety five point eighty seven pence.
If we are to choose between different splits, we will decide upon this second split, which reduces
the sum of squared error by almost nine to one.
So this is how we decide on which split we should be choosing now, let us have a look at this particular
image.
This blue line actually shows like what?
How we have created different decision nodes and created split so that we have a decision three in place.
Now, this decision tree is able to predict the training data very greatly and all the values are predicted
very finely, like when we are actually trying to predict certain values, such as the value at this
point and a value at this point or value at this point.
It actually gives a lot of it because for this point it will.
Predict this value for this point, it will predict this value for this point.
This will predict this value.
So now.
If you see the error, which we have is higher.
In these cases, but instead, we could have simply solved this problem by using linear regression.
So if we would have applied linear regression, then we would have created a straight line and it would
have predicted these values could.
Or with the minimal edit.
So this is the reason why we should always start with the very basic models, always implement the linear
regression or logistic regression, forced to get the benchmark value, and then slowly and gradually
get to the next models and see what is the level of improvement we are getting.
If we are not getting much improvement using the higher level of algorithms, then we should always
use the simpler model that is either linear regression or the model which gives a better accuracy.
So that is how we should decide on which model should we be choosing never to go for a complex model,
always decide by having a simpler model which gives a good design to us.
So that is the reason why instead of having a decision, it would have been better to decide and select
the linear regression here.
Like this does not mean that we will always select the linear regression or logistic regression, always
go for a model which gives you a better performance.
Now, we have been creating this decision tree, now this decision tree would have been better if we
would have just dropped earlier point of time, but how would we know that there should we stop?
So to stop, we should either just or select of these five conditions, either of these five conditions
would help us to decide when we should stop.
The first condition being when the node becomes homogeneous.
That is when all the norms of this particular data becomes the same.
When the old data becomes homogeneous, then we should stop splitting because if the nodes are already
homogeneous, then on splitting, there will be no information gain.
Next is when a maximum depth of reporting is reached, so we should decide on that value, a maximum
depth value and based on the maximum value, we can stop at five, seven, 12 or 15 Dep't, which is
considered a good bet for a decision.
Three, and we should not allow the tree to grow a lot.
Next is we should alert, allocate a maximum number of leads, no limit that is reached.
So if we do not know what that we should be having, then we should at least set a limit to the number
of leaf nodes can which can be created.
The next clay is the leaf node has observations less than the loan limit of evolutionism.
So let say I have one hundred rows of data and I am making a particular split.
Now, for this particular split, let's see, I already have the lead up and not having four values
of one class and five values of another class.
Now, if I will be making a split, then it will be creating and giving me one node with forward values
and the node with five values of the other plus.
So this kind of split will be giving me one node which has less than five values.
Which is a very low number of values of No one to muddle through of data.
Now, I could have set the minimum number of Neef, no observations do 20 in case I had a larger amount
of data.
But when I have only hundred rows of data, five would be a lower.
Let's we have a thousand points in the entire data.
Then in that case, we could have said 10 or 20 as the lower number.
So we have to decide on the basis of how much data we actually have so that we don't end up with a leave
node with very minimal amount of data, which does not actually count for a good majority.
Then the split results in at least report having less than no lower limit of value.
So either the D.A. has a lower limit of value or the split after this, particularly if not, will result
in a lower limit of value, then we should stop the splitting.
So there are several hyper barometers, which we have the fullest type of barometer is criterion, which
defines that which method, which way we should be using, which is either guiney or entropy.
The function is to measure the quality of the supported criteria.
Zarghami for the intro before for the information.
Now another is the splitter.
The speaker is either we will choose the best splitter or random speaker.
So which split should you be choosing is what speaker decides.
So by default, the value is best and we should always go for the best.
Next is maximum depth, so maximum depth is the maximum depth of the which we want to have.
So if none of these values, if no value for maximum Deftest chosen, then the Nordon that expanded
till the end of the entire three, till all the leaves are pure or and they all contains less than the
minimum sample.
So we should define a maximum depth or we should define the minimum sample split for.
Next is minimum sample split, minimum sample split has a B fold value to so it will not allow the minimum
sample split to be of value to have more than less than two values.
So the minimum number of samples required to split an internal node is two now.
Next is minimum sample believe that is the minimum number of samples required to be identified.
So there has to be at least one value at the leafleted.
Now, next is maximum features like this.
What is the maximum number of features we should consider by looking for the best split now we could
either choose automatically, choose automatically, or we can choose different values or or square
root or law.
To now maximum feature allows us to select the maximum number of features because we cannot really use
all the features all the time, because then it will create a very huge bucket of rules.
So that is the reason why we don't use a max feature.
Next is maximum leave node.
So it allows to grow a three with maximum leaf node.
That is the maximum number of leaf nodes which could be created.
And lastly, we have last week, which allows us to define if the classes are equally present in the
data or not.
So what does Glass Week help?
So in case, let us say we have a double.
We have two glasses.
Yes, I know.
And there are hundreds of data out of which five hundred rules have.
Yes.
And five hundred rules have no as they're about a good one.
So in this case, we have equal amount of data protection for both the classes.
So hence, we don't need to provide any any details to it, but in case, let us see.
We have a scenario where one glass has only 10 percent of the data and the glass has 90 percent of the
data, then we will have to provide a voyage to the second glass because it has a very less amount of
data.
So what we will do is we will give a high the higher wage to the class, which has less amount of data.
That is the one which has only 10 percent of the data, such that each point of that particular class
is considered as 90 days of the actual or actually nine times of the actual loss so that there are balance
created between the glasses.
So we will be providing that in the way for.
So this is how we will create the maximum split.
Next is Overfitting, next is to define the overfitting, so the decision carries over.
When we have very high depth or there are certain observations which are noisey in nature, and while
we have considered those observations or if we have some features which are not really required and
they are moisy in nature.
So in those situations, overfitting can and thus we have other algorithms which can actually help preventing
overvoting in decision 3s, which we will be launching later.
For now, we will be moving towards the code of decision three and how we can actually implement decision
trees.