In this session, we will discuss about decision trees, so now we have discussed about the Linnean
models, we have seen that the linear models try to create an equation of a straight line and try to
fit the data out with.
In case of decision trees, we create a sequence of decision nodes or a tree like a structure where
we will make decisions at different point and at the end of all the decisions, we will get a clear
winner.
What should be the target value?
So let us have a look at that.
So in this particular tree, we will be having different.
Values now we have values like dependent variables and the independent variables in this kind of a tree,
the dependent variable is the data which is present inside the three.
That is our data, which will be pointing to either a child should play or not play and the independent
variables or the features or the attribute become the decision points.
So each decision point will be created from the feature columns, so in this particular data, the feature
column contains the outlook of the day, the humidity of the day, and if the day's windy or not.
So this kind of structure would be generated and based on this structure at the end of the tree, I
believe we will be getting if a child should play or not play.
So let us have a look at this thing.
Types of bombs, which we have in case of decision.
So the first bomb is to promote the rule of law, to represent the entire population or the sample.
And this further gets divided into two or more homogeneous sect.
So here we have this entire three and the place where the tree begins is called the root node.
This room contains all the data that is all 14 rows of data which we have and for each decision, which
we think divides these 14 rows of data into smaller chunks.
So these 14 rows of data on the basis of the decision node that its outlook is subdivided into five
rows of data here, five rows of data in this section and another five rows of data in this section.
Now, that child structure, the story structure where we have one child at the top and then a. at the
bottom is called a parent child relationship, where the node above is called the parent node.
And the node which is created from it, is called the child.
Similarly, this is a parent node I the node which is created from it is called the child.
And in the field of parent child would be this as the parent and this as the child.
So let us look for the now what a splitting splitting is a process of dividing a node into two one more
sub not.
So here we have the first node where we have 14 rows of data.
This note is subdivided into three nodes.
So this node is divided on bases of one rule that there's one condition.
So what is the condition?
The condition is what is the outlook?
Is the outlook sunny or is it overcast or is it rainy?
So based on this decision, the note on the door has been subdivided, this subdivision of Naude on
basis of a coalition is called splitting of the node.
Now, the next is decision not.
When a sub node split into four those sub nodes, then it is called the decision node.
So this is the road north.
Now we have another road, which is a sub node, this sub node is again divided into different Jagels.
So this is subdivided into different Miloje based on one condition.
Now here the condition is humidity, which is either the humidity is less than equal to 70 or the humidity
is greater than 70.
In this particular case, this sub node is subdivided into two sub nodes where the condition is Vendy,
the decision which is being made is windy.
So these two modes are known as the decision nodes.
Next is the leaf node or to the node that Doolan's plate is called the need for determining node, that
is the node at the very end of the tree are known as the leaf node or the dominant.
Now, what is a branch or a subtree, a subsection of the entire tree is called a branch or a subtree.
So this is a subsection.
Again, this one is another subsection.
So both of these are called branches or subtree.
Next is parent and child.
So, again, the node which is above the sub node, which is above, is called the betting node and
the node which is created from the better node is called the child.
Now.
Let us have a look at the process of the decision tree generation, they now what we have seen is how
the decision tree looks like.
Now we need to learn about what is the process of creating a decision tree now for creating a decision
tree.
We need to have the rule of law.
We need to have an initial node and all the rules of detainment.
Now, after we have all these rules of data, we need to define one particular rule on the basis of
which we will be splitting the nodes.
And once we have selected this particular rule, then we will be doing this split.
And after this particular split has been made, then we will again evaluate all the rules which are
present and we will compare all the rules and find out the next best rule on basis of which we will
be making the split.
Now, how we will generate these rules and how we will compare these rules is a secondary thing and
we will discuss that in some in some time.
Now, let us have a look at how we will make the decision.
So we will have certain rules on the on the basis of which we will be subdividing.
So I'm the final stage when we have this leave node.
So based on the majority values of the leaf node, we will decide if the child should play or not play.
So here we have two values for play and zero values for born to play.
So what we will do is we will decide if that the child should play because the majority is with play.
So let us look at the process so the trees are generated by selecting the best route possible at every
split.
So we will have a lot of rules, we will create a lot of rules, and we will have to select the best
rule to make the flick at every point.
Now, there are two types of problems, which is classification, and another one is regression.
Classification is where we want to find out a particular class, for example, if a loan should be approved
or not, if someone will be defaulting on the loan or not, if a person should play or not, if I should
be cooking today or not.
So these would be classification problems.
Now, what is a regression problem?
A regression problem will be what is the height of the person?
What is the income of the person?
What should be the interest rate for a particular loan?
So these are all values which are continuous in nature would be a regression problem.
And Decision Tree will help us to find out solutions to the classification.
Also, I do regression also.
We can solve both type of problems using a decision to.
Now, how will we do that?
So in case of classification, we can find out either the probabilities or the hype, plus if we want
to find out the probabilities, then we can find out the probability by opening the proportion of the
value that the methanol.
So we have these leads not so we can find out the probability of.
So here the probability of playing would be one here.
The probability of playing would be zero here.
The probability of not playing would be one here.
The probability of not playing will be zero.
So this is all we can find out, the probability of the classes.
Now, how can we find out the hard class?
If you want to find out the actual high gas, what we can do is either we can select on the basis of
the majority.
So here we have two words for playing, zero words for not playing.
So we would play.
Here we have zero words for playing and three words we're not playing.
So majority is for don't play.
Next is applying the cutoff on the probability score, so many will be finding out the probability value.
So we will apply a cutoff.
You see, we can have a cutoff of, say, 60 percent.
So if the probability is zero point six, then go out to play.
If the probability is less than zero point six, then don't play.
So we can do something like that.
Now, in case of regression, so in case of regression, what we do is we make the decision and it is
dependent on the average of the target value, either Dominello or leave.
Now, in case of classification problem, we have values as play.
I don't play present in the leave.
But in case of a problem of regression in the lymph node, we will have continuous values.
We will have values of, let's say, 10 rows of data or 20 rows of data, which will have different
interest rate values or different salani values or different height values.
Now, based on all of those different values which are present here.
We will be taking an average of all of those values and that average will actually be the result of
this decision that.
Now we need to create a decision tree and we will be creating this decision tree using a lot of splitting,
which will be doing now.
We cannot keep doing this splitting endlessly, because if we keep on splitting the decision tree,
then at the end we will have a very huge decision day.
With all the rules of data in a different block of liefooghe, each data rule will convert into one
leaf node and the tree will be very large in structure and also it will become very complex in nature.
And the forest guideline, which we have for machine learning, is that the simpler, the better.
We will always decide on going towards a simpler model in comparison to a complex model because a complex
model tends to overfit.
This is the reason why we will choose a simpler model and hence we will have to stop splitting at one
point of time.
Now, hi, when will we stop this bleeding and how we will decide when should we stop?
The thing is based on a few great ideas and a few set of conditions, which we have already decided.
So what are those conditions?
We will have a look at them once we actually know that, how we will start creating the decision to.
Now, let us have a look at the Roy Creation's.
So for creating rules for the new music video, we will split up then the ranges.
So we will have to split up the random positions, so let's say we have a numeric variable each.
Now this numeric variable, each can have any value.
See five fifteen twenty four.
Twenty nine fifty six hundred.
Anything would be of value of each.
Now there is no specific range or any specific distance on basis of which we will see.
Now here you can see the rules are like if the humidity is less than equal to 70 or greater than 70.
So there is any value can be created, converted into a rule, but the rule will be created randomly
so it is not fixed.
We will have a rule at zero, then under the rule at five, then next rule as then, then 15, then
20, then 25.
No, that is not a criteria.
We can have one rule at age greater than 10 and under the rule at age less than 15 and next at 16.
So there is no specific criteria or no specific range or interval which is present in creating a rule
out of the numeric variables.
Any value can be used for generating this rule.
Now, let us create a rule for categorical video we have seen for numeric video, but the next rule
will be generated on the categorical variable.
How do we create the rule?
Uncategorically, video will be created by splitting on the value created then zero point five.
So let's say we have one categorical column as gender, another categorical column as Sipi Wichai values,
let's say New York, Paris and Moscow.
Now we can avoid the categorical column into numerical column by the creation or by one Hawtin pooling.
So the gender column has been converted in the male column.
Or we could have created a female column and if the value is zero, then we are saying it is a female
and if the value is one, we are saying that it is Amedi.
So the condition which we can evaluate here is if the value of milk is greater than zero point five,
then it would mean that we are pointing towards me.
Otherwise it would mean that we are pointing towards Fehmi.
Similarly, we can create less city and New York, greater than 50 divide is greater than zero point
five.
So now, again, the value will be compared with the city value because it is one.
So it will mean that we are pointing towards New York.
If it is Paris, then we are pointing towards one.
So this is accordingly evaluated.
Now, the next thing which we will be discussing about is the splitting, so we will discuss about splitting
and how we will do the splitting and different rule generation for the split in the next session.