This session, we will discuss about different performance metrics that are present in case of classification,
so these are the performance metrics which we will be using for logistic regression and also for any
classification problem.
So let us have a look at those.
So the first one is a confusion matrix.
So a confusion matrix is a combination of the actual values versus the predicted values.
It is all table for predicted and the results that are revealed in nature, so let us consider this
particular confusion matrix.
So here we have a true positive.
BP, which means to positive, then we have Effi, which is false positive, Ifan is false negative
and the end is all negative.
Now, how is it declared how the names are given?
So whatever value that we have predicted is coming at the end of the name.
So if we have predicted something to be negative, then that is it is either false positive or false
negative or negative.
Similarly, if the value has been predicted as positive, then it can be either positive or false positive.
Now, what does this true and false mean?
So let us see that we have predicted something to be positive when the prediction is positive.
It means that that prediction will be positive.
The second word will be positive.
Now, the actual value is also positive.
So the predicted value and the actual value match.
This means that our prediction is true.
That is why it gets the name proof positive.
That is, we have predicted the volume value to be positive.
And this prediction is true.
This means that it is a true positive.
Similarly here we have predicted the value to be positive, but actually the value was negative.
Hence the prediction which we have made is a false prediction.
The prediction which we have made is not correct.
So it is called false positive.
The positive, which we have predicted is a false prediction.
Next, we have false negative, this means that we have made a prediction that the value has to be negative,
but actually it is positive.
So it is a false prediction.
Hence, we have predicted a false negative.
That is it is not a negative.
It is a false prediction.
It is a false negative value, which we have predicted next is true negative, which means that we have
predicted value to be negative.
And it is a prediction.
It is a correct prediction.
That is why it is called a true negative.
Next, we have these different metrics, which we have, so the first metric is accuracy.
Accuracy means the correctly predicted values all out of all the values.
So what are the correct predictions?
Correct predictions will be true, positive and negative.
That is the predictions which we have made correctly.
That will be the proof positive and negative.
Right.
So accuracy is proof positive, plus true negative, divided by the total value, all the values that
this positive plus negative.
So this is accuracy.
Accuracy, which means that the correct prediction out of all the values, what value we have predicted
correctly, that is accuracy makes this precision.
Precision is that out of all the predictions which we have made, these predictions are positive, which
values are actually positive?
That is, out of all the positive which we have predicted, what values are correctly predicted, so
correctly predicted?
Positive will be the positive means that these are predicted positive and these are actually positive.
So this means it is proof positive plus false positive, which means that which values we have predicted
as positive.
So out of all the values which we have predicted positive, which are actually positive.
Then we have sensitivity and the equal, which means that out of all the actual positive values, how
many positive values are predicted that that is out of all the positive values?
So here we have these all the positive values.
So proof positive and false negative are the positive values, are the positive values, the actual
positive cases through positive and false negative.
So these of the actual positive value, the one which we have predicted as to be positive and it is
actually positive, so we divide this correctly predicted positive value divided by the actual positive
values.
This is quite recall how good we are at recalling the positive cases.
Out of all the positive cases, how well can we remember the positive cases?
It's called REPL.
So it is all true positive rate.
It is also known as proof positive rate and precision means that out of all the positive, which we
have predicted positive, which are actually positive.
So these are used in two different ways when which is used that to something which we will discuss in
something, then we have specificity.
So specificity is the opposite of a recall.
So it is correctly predicted.
Negatives out of all the negatives and sensitivity was correctly predicted positive out of all the positives.
So specificity is correctly predicted.
Negatives out of.
Out of all the negatives, so it is too negative, divided by a false positive plus two negatives.
So which metrics should we use where?
So this is how we will decide which metric we should use so we cannot decide upon a single metric.
Hence we try to use a combination of metrics like decision.
And so precision is proof positive, divided by two positive plus false positives.
That is out of the values which we have predicted to be positive.
Which one not actually brought up correctly predicted were positives.
So it is decision and how many positives which we have predicted out of all the actual positive values
is really good.
So let's say we are predicting cancer amongst patients and only if we are absolutely sure means we have
a high value.
So we will have to provide value in case of to someone that they have cancer or not.
So we will let us say we want to tell someone if they have cancer or not.
And for that we want to have a high cost of value.
That is, we want to be 100 percent sure.
We want to have very, very, very sure and to tell someone that they have cancer, otherwise we won't
build them.
So this is the case where.
We are fine on missing out on a few actual positives here.
We want to be assured that we can miss out on an actual positive case, but we don't want to miss out.
We don't want to tell someone who doesn't have cancer that they have cancer.
So instead of telling a negative person, a person who does not have cancer, telling them that you
have cancer, it is better to lose on a few cases who actually have cancer.
So this is the case when we want to have high precision and risk, but that is of whoever we are telling
them they have cancer, they should actually have cancer.
And we should not tell someone who does not have cancer that they have cancer.
So this is the first case.
The second case is when we want to have a look at the value.
That is we are fine to tell someone who does not have a cancer of telling them that they have a cancer
instead of losing out on someone who actually has cancer and not giving them medication.
So we don't want to have a case when all actual cancer patient does not get to know that they have cancer.
So here we want to have a very good recall rate.
So here we want to have a high quality, low precision.
So hence, as we lower the value, the value of the call will increase and precision will decrease.
So here you can see that when we decrease the value, the decline will increase and precision will actually
decrease.
So if we want to miss out on any actual positive cases, if it is fine for missing out on a few actual
positive case, then we will have a high precision.
But if we don't want to miss out on any positive cases, then we will have a high value.
It is very important to understand this, because these are the metrics which we will be using and are
very important.
So let us have a case where we have a five one matter, what is a five, four, five one error or false
positive rate is when we are finding out the number of false positives out of all the.
Negative cases.
So this is also about specificity.
Now, let us see that data is balanced, and in that case, we will be calculating the accuracy rate
and the formula will be true, positive plus negative divided by two positive false negative plus false
positive plus two negative.
So for a balanced dataset, we will be using accuracy rate as the metric.
But for any imbalanced dataset, we will choose from the equal precision one and decide on the basis
of these criteria if we want to have a high precision or we want to have a high equal value.
Now for a false positive, if in case false positive is important, then we will go towards precision
and in case false negative is important, then we will build the world's record.
Now, that is another metric called if we test for this and the score is calculated by one plus the
in two times precision, including one divided by the Beita into two in the recall divided plus precision
value now in gives false, negative and false positive.
Both are important.
That is, we don't want to lose on either false negative or false positive.
Then we will give the better value as one.
But in this, we want to reduce the false positive rate.
We don't want to have high false positive values, that is, we don't want to tell someone that they
have cancer.
When they don't really have cancer, then we will decrease the VW.
And in this false negative is important, that is we don't want to tell someone that they do not have
cancer when they actually have cancer so that they don't really miss out on the treatment and get the
required treatment as soon as possible.
In that case, we will increase the B.W..
So based on these ideas, you can actually decide if you want to increase the value or do you want to
decrease the value and will you be considering the high precision all of you will be considering, Heidi.
So these are a few points which you might want to keep on Notthoff and try practicing again and again
so that after some period of time you will get a hold of which metrics do you actually need to have
a look at?
So let us have a look at a few more details, so let us say we have this model, one which has a precision
zero point five under the model, has a precision value, zero point nine, and that is a model T because
of precision zero point zero.
Now, here we have model one.
Rules of equal value is zero point for model two has a very low equal value.
That is zero point one.
Model C has a very high legal value.
That is one.
That means that it will never miss out on any.
Positive cases and having a high precision means that it will never miss out on a negative case.
Now, the average of these comes out to be zero point four five zero point five and to point five one.
Why is we have a look at the F1 school, you can see for model one, the F1 score is zero point four
for models to drive one school to zero point one need for more than three, though, if one score is
zero point zero three nine two.
So here you can see that if one school will actually give a better view of the entire precision and
recall metric, because then if one scored when either of the precision and recall is very low.
So in case of Madrid, you can see that the equal value is very low, while in case of model three,
the value of precision is very low.
So for both of these cases, the F1 score comes out to be low value.
These both of these values are low.
While although precision and recall are not very high for model one, but it does not really allow the
run of the value to be very low, hence if one scored is higher for this.
So you can see that F1 score is a better metric when we want to have a balanced precision and vehicle.
So next is a U.S. school, so let us try to understand what a U.S. school is, the U.S. scored indicates
how well the probabilities from the positive classes are separated from the negative classes.
So.
It tells us how much a model is capable of distinguishing between different classes from a U.S. school
perspective from this school, which we have, we want to choose a particular school which has the most
amount of area under this.
And then we have a look at this.
U.S. scored a model which has a threshold value, which is like we have different values plotted for
different social values.
Now, the closer it is to this diagonal line.
The closer it is to this diagonal line, the poorer the performance of, the more the less we want the
model which has the value closer to this top left corner of.
So hence, we can decide upon either this particular threshold value or this threshold value, something
which is actually closer to this particular point.
So the higher the you see, the better the model is at predicting zeros as zeros and ones as ones.
The sensitivity which we have here.
This is also called true positive rate.
And one minus specificity is known as the false positive rate.
And these are the formulas which we have already discussed of of specificity and sensitivity.
So the main task here is to select a particular goal, which is near to this top left corner.
And what we can do is we can plot different charts which are present for different threshold values.
So these are different threshold values for which we are plotting this.
So here we have special value, zero point zero one zero point to zero point three zero point five seven
zero point seven six point nine five.
So all the threshold values are present here.
So out of all of these special values, we want to select a particular threshold value for which this
score is closer to this particular corner so we can decide accordingly.
Now, let us have more information so it is used in the declassification problem, it is suppose we
are implementing a logistic regression.
I suppose the model has predicted some probabilities.
Then we need to decide what is a threshold value and what is a value.
So out of these values, which we have here, out of these values, we want to decide which value should
we select.
So the threshold value or value is decided by the problem statement.
Now, for example, if we need higher false positive ID or law to positively based on that, we can
play with the cutoff value.
Next, we have the sequel, so the Odyssey for calculating the movie required both false positive rate
and positive.
Right now, based on the threshold value, we get the positive and false positive rate that is this
rate and this rate.
So I can select any threshold value legacies, zero point five seven.
So based on this, I will go towards the goal.
So this Goba has a positive rate as zero point six.
I feel positive.
Does zero point for now based on this, I can decide what I actually want to have.
So from this, you can see that here at this particular part of.
I have a higher false positive rate and a higher positive rate, so in that case, I will select this
point to be my cutoff value.
That is zero point seven six can be a good threshold value for this particular.
Now.
It plots a graph joining which point we get the goal of the 80 and the goal is for the U.S. goal.
So 80 under this goal is called the U.S. goal.
So what we can do is we can plot one different U.S. goals and from the different autoclaves and U.S.
goals.
First of all, using different U.S. scores, we can find out which model is better.
So let's say you are comparing different models.
So for each model, you will have a different goal.
So let's say I have one model for which I have a goal of like this and another model for which I have
a goal of like this.
So out of all of these goals, I will select the model which has the highest that this one this one
has the most amount of area under this.
Right.
Next is if I want to select the cutoff, then how I just stated the cutoff has to be the one where the
value of.
False positive rate is less and the true positive rate is high, so we will want to have something like
this particular point.
Or maybe this particular point would be a good decision point where the false positive rate is zero
point four and two positive is zero point six.
So we can have any point selected, which is closer to this particular scenario, and we can then compare
what values we are getting for the bus was for the precision and recall and then compare accordingly
and decide upon the model that you want to select.
But if you want to select upon different models, which we have created, so we will go ahead with the
one which is having the maximum schauder that.
A U.S. school we basically domain how would we are classifying and predicting zeros and zeros and ones
as one, so that is what we are looking out for here.
So this is about the matrix, which we have for classification.
In the next session, we will learn about the implementation of a logistic regression.
And I will have another session in which I will explain about different metrics of a job as intended
so that you can understand more from them.