High in past few sessions, we have discussed about depression and logistic regression in logistic regression
and regression, we have used the most used metrics.
The metrics are basically the quality, quantifying or quality of prediction, how we can actually make
sure that bad models are performing well.
These are what the metrics have been designed.
Now, there are different metrics available for regression problems and different metrics which are
present for the classification problem.
So Ascalon is the library, which we have been using and Escalon on Metrick is actually the module which
helps us to select certain metrics which we will be using for these different algorithms.
Now, let us have a look at this particular library and see what other metrics that are available.
So this is the link, which is cyclone dot org slash stable slash module's slash Mauduit evaluation.
From here, you can actually get the details of different evaluation metrics, so here we have certain
sporting barometers like cross-validation.
Great TV cross-validation score, so these are different validation scores, which we can use, then
we have metrics which are the values such as you see a mean absolute score and all different scores
which we have now.
This link itself provides different metrics like this.
If we want to have a classification metric here, each class is individual class and that is no scenario
where one class would be or multiple classes could be do that does not present in the classification
metric.
So classification metric only helps when we want to find out something like either something is a cat
or dog if it is or a sees a problem where we have either two sides, either it would be yes or it would
be an either someone will pass the exam or someone will fail the exam.
So these type of problems will be considered in of classification metrics.
Next set of metrics is being multilabel ranking metric, these type of metrics are something which are
used when we have multiple levels which are applicable.
For example, we have classes like or labels like Don and Chubbie, and we have something like labels
like short, chubby and not chubby or they're not fat.
So these kind of metrics are present and we want to find out which labels are attached to a particular
person so a person can be either a boy or chubby and the person can be chubby or not chubby.
So something will be a combination of multiple things.
So it cannot select only one type of food.
So it will be there can be multiple labels which can be selected that we can have someone else job boot
so they can label selected out of all the five labels or their neighbors, which we have.
So that is the multilabel metric, then we have regression metric, which is where we have the regression
problems.
These are the metrics which are used for continuous value evaluation.
So this is something like some of squared error out of just your data square.
So we will discuss about those next.
We have the clustering metric.
This is something which is used in case of unsupervised learning.
So we will discuss about it separately during the unsupervised learning module.
So let us discuss about classification metric, multiband metric and regression metrics.
So here you can see the definition of these.
So these are different pre defined metrics.
So scoring accuracy could be accuracy, balanced accuracy, average precision.
If one if one micro, if one macro of some negative log loss, then for clustering.
This is what classification metrics.
See, these are different classification metrics.
Then these are different clustering matrix.
Here we have the different regression matrix, so regression metrics are explained variance, maximum
error, negative, mean absolute error, mean negative means squared error, negative root mean squared
error, negative means squared log error, negative media, an absolute error of scored.
This R-squared score is a very majorly used to score because it is having one speciality.
That is when we use our square score and we try to find out different type of columns.
So let us see.
I'm working with a specific let me discuss about it separately.
So this is the R-squared Coalition, this is sufficient of determination, so this is regression.
So the best possible score is won and it can be negative also because the model can be arbitrarily was
now a constant model that always predicts the expected value by will have the value one disregarding
the input feature would get our squared value of.
So the best model will be.
Having value one now, what we will be doing here is that is in case of R-squared, what happens is
legacy say we want to find out.
We have been very busy.
And so out of those 10 variables, let us say I want to find out which we are actually helping me in
finding out the predicting the value which I want to predict.
So what I can do is I can keep my scoring mechanism as our squit.
And then I can try the try finding out the R-squared value by draining my model on the first feature.
Now from the first feature I will see what is the R-squared value if the R-squared value is improved
by introducing the second feature.
Now, then it means that second feature is a useful feature.
Similarly, we can test for the third feature if introducing the third feature actually improves Darmody
performance or not.
So this is how we can actually decide upon multiple features, because if the feature does not have
any impact on the R-squared value, then it means that the feature has no impact on the model.
So this is how our squared value is very important and it is a very useful metric in finding out the
feature importance.
Also, although we have different methods which we have discussed about like feature importance, which
we will be discussing in case of random forest or the opportunity we are using L2 evaluation or evaluation,
which we have discussed in case of meaning and logistic mortgage, so we can use any of those methods.
And other than that, we have different methods as bonders, profiling and ViiV method and the correlation
coefficient methods.
So there are several methods and you can choose any method which you like and maybe use a combination
of all of these methods.
So it is completely your choice.
But I would suggest try all the different methods one by one, maybe in one velzy or one model.
You can try one method under the model, try another method so that you will get to know how to actually
use this.
Because, I mean, unless you try and use all the different methods of finding out the best coefficients,
you won't really get to unless you will try different methods.
You will not get to know which method you are most comfortable with and which methods you are not comfortable
with so that you can work on the methods which you are not really comfortable with and improve your
concepts on that.
So let us go back to the Matrix, which we have.
So these are all different metrics which are available.
So from all of these metrics, the the of using these metrics is just the same.
You will simply import the metric scale on dot, matrix and dot the name of the metric that you want
to use.
That is how you will it.
And then you will simply use it like a cross-validation score for the model for the of the particular
X value and the Y value and then check which one will with.
So this is how you will use them.
So let us go to the classification matrix.
So these are different classification metrics, so you can see we have precision of accuracy, of balance,
accuracy score and cupper score.
So there are a lot of metrics.
The most frequently used metrics are accuracy of.
Then we have confusion, matrix.
Then we have a U.S. school, so here you have the definition of the methods also.
So what you can do is you can read these descriptions and then decide upon that which method you want
to actually use.
So there are a lot of metrics and different methods provide different kind of benefit or under the method.
So that is how you can actually use them.
Then we have this recall score you use, you score, then we have average precision school love loss,
so there are different kind of scores which are available.
So there are scores coming score.
So these having score is basically use when you have something like the actual data so you can decide
upon what kind of data you have and then use that kind of metric.
Then in case we want to have a multiclass kind of a problem, so then there is a different kind of matrix.
So see if you have by any metric, then you can improvise.
And in case you have multiclass or multilabel problem, then you can use a different version of The
Matrix.
So there are different versions of Matrix, such as macro rated micro, which have different kind of
benefits.
So let's say Micro will simply calculate the mean of the binary matrix giving equal weight to each class.
So if we have, let's say, a class which is balanced in nature, then we can use macro.
Now let's say we have some kind of imbalanced classes.
Then we can use the weighted version of the particular metric which we have.
Then for micro, it gives each sample last year an equal contribution to the overall metric except as
a result of some fluid rather than some make the metric.
It will simply put some calculation on top of it.
Now, micro averaging, maybe preffered in multilabel setting, including the multiclass classification.
So in case we have multilabel setting.
So in that case, we can use this microstamping.
So there are these different versions which are available to your matrix.
Then we have accuracy score, which you can use, and violence accuracy score, these are different
versions of the scores which we have here.
We have the confusion matrix, which is, again, very highly used.
And if you want to see the further documentation, you can just click on this.
And the more detailed definition would come up.
Like you can see here, I have clicked on confusion matrix, so it simply tells me the method of using
it.
So I can simply see that, see that it needs the true value, the predicted value.
And after that, all these things are actually are not really useful.
If you want to use them, then you can use otherwise.
These are optional in nature so you can provide the labels or the sample rate or if you want to normalize
the date or not.
So these are different things which you can use in the confusion matrix and it gives the details of
the parameters.
Also what I present here, and it also gives the examples of those.
So this is how you can actually check how you want to use a particular metric and how to actually implement
that.
So in case there is a metric which you want to explore further, you can simply go to the documentation
and have a look at it.
Similarly, let's say you want to use any different version of any algorithm.
So it is not a sure thing that you will always use the same implementation of Decision three or same
implementation of linear model.
So you might be interested in a different type of implementation.
So in that case, simply go to the documentation and have a read.
This is the best prescribed way, which will not just help you now, but also in the future when you
will have to explore different models.
So this is one practice which is very useful and it actually helps and saves a lot of things because
it will not happen that you will always remember how you want to write the code for the confusion matrix.
So you don't need to worry about that.
You will learn that by time.
But for their timing, you can simply go to the documentation and read it out, go to the example,
see how it is being solved, and then do the implementation.
OK, so this is how you will learn slowly and gradually, it is like a best practice, what you can
follow.
So here we have a different classification matrix, which I was talking about from the classification
matrix.
You can see this classification report, which is provided.
This is a very useful tool because it gives the precision detail.
Also recall also if one score also.
So here you can actually compare and how much precision is then how much recall is that?
And then based on the problem that you have, if you want to improve the precision, then you can look
at the precision of the glasses.
If you want to improve the recall, then you can look at all of the glasses and then decide which one
do you want to use.
Right.
So this is what you can do then, apart from that, you can see the details of the autopsy also being
given here.
So you can use that.
But these are the major methods which you will be using other methods you might not really use that
often.
So here you have the autopsy documentation.
So this is the documentation of Orosco.
Here you can see you just need to provide the zip code, which has been imported from on dot matrix
and give the value and divide predicted value and the label details, and then it will give you the.
Oh, itself.
So you can bring the code for yourself or compare the values and then you will get to know which value
you want to pick for this particular.
And you can generate different goals for all the models which you have prepared.
So let us see.
You are applying a classification problem, then you can implement the classification problem using
logistic regression, also decision, but also random forest also exposed also.
And then create Orosco for all of these modules and see which one has the maximum.
You see the maximum area and the goal and decide which one you want to pick up.
So these are different things which you can use.
There are a lot of tools which are present.
You just need to learn and explore those.
So here you can see here we have all the models which have been come back and you can clearly see that
the blue light blue one is the winner, so which is for the plus one.
So this is the winner.
This is the best model which has been created.
So similarly, you can go for the multilabel problem then, in case you want to find out the regressions.
So here you can see what a different regression matrix.
So we have this explained variance metric by which you can see how much variance is explained by a particular
column.
So this helps in using this helps in finding out if a particular variable is actually important or not,
or is it actually having some impact on finding out the target or not how useful the variable is?
That is what this explained variance code will tell you.
Similarly, you can find out the mean absolute error and compare different mean absolute errors, the
mean absolute obscure, the mean square logarithmic error, all these values need to be closer to zero.
So if the value is close to zero, then we're good to go.
And regarding the school, the school has to be close to one.
Apart from that, all the other regression metrics need to be close to zero.
So that is all you can actually find, though, then use different metrics on here.
We have all the implementations so you can try these implementations and check the implementations,
which I would be giving in my code so that you will have enough documentation and you can learn a lot
from this.
So this is what this particular session in the next session, we will go ahead and learn about the decision
to.
Thank you.