In this session, we will discuss about the mathematics behind logistic regression.
So first of all, let us look at a few items.
So the first stop is ORMs.
The odds are the chances that something will happen now, if you flip a coin, the odds are 50/50,
that is you'll get hit.
Now, if something strange happens, people will often see what were the odds of that, which means
that I can't believe that happened.
The odds were against it.
So how would we find that?
So, for example, the odds in favor of a ruling or two or three or six cited by one is to five or one
divided by five.
That is odds against will be and against an even equal number of unfavorable outcome divided by the
number of favorable outcome.
So here, let us say, the number of times our team has already won is one that a team has lost the
game four times.
So the odds in favor of the first team winning is one is to fourth with five wins.
The total number of games.
This is also written as one is to four.
Now, what is probability?
Probability is the odds of something happening or what?
All the popular cases, all the possible cases.
So probability in this case will be one to five where one is the chance of this team winning, divided
by the total number of cases with just five.
While Vilvoorde is not equal probability itself, it is the ratio of something happening over something
not happening.
Probability is ratio of something happening over all the possible cases.
So this is the difference between odds and probability.
OK, now let us see, we have odds of winning as five to three.
That is five times IBM has one and three times the team has lost the game.
So what will be the odds value?
The value will be five divided by three.
That is one point seven.
Now, what will be the probability of winning?
The probability of winning will be five, but just the same as the giants of the team winning, divided
by the total number of values that this five plus three comes out to be eight.
So the probability of winning will be five by eight here.
Which is zero point six to five now, because probability of winning is equal to opposite of probability
of losing, that is one minus probability of losing.
So the probability of losing will be three by eight.
That is zero point three seven five.
We can also write as a probability of losing equal to one minus probability of winning.
So which will be equal to one minus zero point sixty five, equal to zero point three seven five.
Right, so probability of winning is zero point six dubi, but just five by eight probability of losing
is three by eight, and probability of losing can also be written as one minus probability of winning.
Now, probability of winning divided by probability of losing candidate A. probability of winning,
divided by one minus probability of winning.
We have just replaced probability of losing by one minus probability of winning.
So this gives us five by eight, divided by three by which comes out to be five by three, which is
actually what it does, the odds of winning.
So you can say that probability of winning divided by probability of losing is equal to or winning,
right?
So if the goal, the probability of winning as fee, then we can also be divided by one minus the.
You can take some moment for the slide and try to understand what I just said, that is the what is
the odds of winning?
What is the probability of winning?
How probability of winning is related to the probability of losing and all of this mathematics.
Now, the odds value, which is five, is to three.
This odds value will range from zero to one.
For against scenario, they range from one to infinity for favorite movies.
So let's see if.
Our theme is the value of is if the above value, if the odds of winning.
Is a larger value rate, the odds of winning is a larger value here.
So in this particular case, the favorable case will be of greater value divided by the smaller value.
A favorable case will always be of greater value divided by the smaller value, then only it will be
the in favor of the being right.
So it will always be a greater number divided by a smaller number, which means that it will always
be greater than one.
So the value greater than one can range from one to infinity.
Now when we are discussing both against scenario so four against scenario, it will always be a smaller
dome divided by the larger.
So when this would happen, then the value will be between zero to one and a smaller number divided
by a larger number, will always give us a value which is ranging from zero to one.
So this is what we get from or so of value, which is ranging from zero to one, is always be present
for or against scenario and for favorable scenario.
Odds will always range from one to infinity.
Now.
Now, when you see this.
Four hours of winning.
So although veni.
Will be a favorable scenario right now.
Let's go for the.
Now, let us see, we have these equations which we have formulated, which is probability of Y is equal
to one given the excess true is B they we just said we will say probability of winning.
What is being that this is probability of Y is equal to one given X is equal to B, then law will be
divided by one minus V will be better not let's be double next one plus X.
So here what we are doing is we have this equation.
Where we were seeing that the probability is equal to be done or plus with our next one, plus we do
X to be the extra and so on.
So we are seeing that this is the probability value now because this probability value was ranging from
zero to one.
And this equation has a range from minus infinity to infinity.
So what we can do is we can convert this equation into the same range value by taking a log of.
So if we will meet the goal, all of it, then the range mismatch will be resolved.
OK, so what we do here, we will take a look and now for taking a look at the situation and on solving
the equation for VA, we get the equation.
The VA is equal to one upon one plus E to the power minus Vetinari plus vedova an excellent as that
went on.
And this is actually the logic function which we have discussed about.
This is the logic function we have formed for this equation, which will convert this linear line into
this goal line, which will be able to map to the.
Two classes, which we have.
So why do we do this with this equation now?
The objective here is not to correctly estimate the value of log the by one minus B, so this is not
the objective V one.
Do either maximize or minimize the value of B such that we get either a low probability or higher probability
that it's either somebody would just close to zero or somebody would just close to one.
So here we will be having the probability value as some value as zero point one zero point three zero
point five zero point seven.
So these would be different probability values, which will be ranging from zero to one.
Now we have these probability values.
Now we want to change the values such that when we take this log of this, the this function will actually
tell the values closer to zero and one.
So the values will be if the values closer to zero, then it is more prominent that the answer is zero.
That is because of zero.
Plus, if the value of the probability is closer to one, that means that the answer is one.
That is the probabilities of the first class.
Now we have this likely war to function, but just create it so that we can formulate the same thing.
So here, the objective is not to correctly estimate the value of log be divided by one minus five.
OK, so we want that barometer's to be values which reside in such a score or a probability which enables
us to have a good cut of value.
So if the values will be closer to zero or closer to one, then we can have a lot of value which will
be more prominent, which will be more helpful.
If the total value is not created properly, then some points might be misclassified.
And that is something which we don't want to happen.
So meaning this school should be high for one class and low for another class.
We want to have a good distance between both of these school values.
Now, let us consider this particular equation.
So here, when Y is equal to zero, when Y will be zero at that point of time, the value of likelihood
this function is called likelihood function.
So when Y will be zero, the equation will go on and do one minus B.
Because this value will be zettl, so this dome will turn into one, this value will be one minus one.
So this will turn into.
This this will be one minus zero, so it will turn into one, so we will get one minus B as the equation
now when the value of Vye will be one.
So what will happen to the other one will be probability.
Now, this will be one minus one zero.
So this will give up one here.
So that is why the entire tone of likelihood function will go on and do the.
So in other words, we want to maximize this likelihood function, we want to achieve this value because
if we will maximize the likelihood function, then the value of likelihood will go into either B or
my one mind.
The speed, which is not a good hit that is then is equal to zero.
The likelihood equals probability of being one then by is equal to zero.
The likelihood of probability will be of Y will be zero.
So we want that probabilities to match with the real outcome.
Hence we want to maximize the value of L.
So that is the value.
When Y is equal to one, the likelihood will be probability.
That is, it will go on and do one value of maximum value and then the negative form will diminish.
The storm will diminish.
And when the value of Y is equal to zero, then this entire goal will be reduced to almost zero.
So this is what the likelihood will do to us.
Now, this function which we have created is for only one observation.
So we will have to convert it into a combination of all the observation.
So for this, we can do this by basically multiplying all of these storms together, because in terms
of probability, then we want to combine different probabilities.
We will be multiplying these probabilities.
I in earlier scenario, when we were combining the different medications for different rules of beta,
we do the same mission.
But in in this case, we have probabilities.
So we will be taking a product of the probabilities.
So we will have to maximize this collective form.
We will have to maximize this entire thing.
So for maximizing this and the entire thing, we will be.
Then voting this into a cost function and we will be taking the logo of this entire thing and this becomes
the cost function of this entire.
Of likelihood function for logistic regression, this is the cost function of the logistic regression.
So next is different performance metrics, which we will be launching later.
But for now, what do you need to understand is that even if you don't understand the more details of
the entire concept here, which we have just discussed, you just need to remember that our main purpose
is to throw in the values we want to get the probability value.
Which is closer to zero, and one that is because.
When we have this kind of plot.
When we have this kind of plot, our main target is to set up a value which is a cartel value.
Now, the current value will supposedly be in the middle of this line.
So when the value is in the middle of this line, what we want to achieve is if the probability value
will be very low and very high, then what will happen is wherever we will put the cost of value, these
points will not get misclassified.
The line will be more towards if the cutoff, if the distance is more, if the distance between these
is more, then the line would do something like.
This.
So this will be able to make a better decision, give a better decision boundary.
So that is what we want to achieve instead of having the the values of the probabilities very close.
We want to have the value of probabilities.
Finally, when the values of probabilities will be far away, then the value can range from anywhere
to anywhere like this.
And we will still be able to classify the data properly because now the data is far from each other,
the probability values are far from each other.
So any cutoff will still make that magnetism.
But when those values will be closer like this, when the values will be closer like this.
In that case, if the values are closer like this, somewhat like this.
Then the cutoff will have to be a very strict one, then it will be very important to find the perfect
cutoff for you.
But in this scenario, because the values are already far away, we can have any of value between this
to this end of value will give a good result.
So that is what we want to obtain.
And the only formula that you want to remember from this entire thing is the formula for the logic function
that is being equal to.
One upon one plus E to the power be done on Glasby, Delmonico's one, Buzby that way too and so on,
when this be done will be the one that will be the professions which you will be generating an X one
extra extra hour, different values of the features which we have, the feature values, the attributes
which we had.
So it is similar to the linear regression.
But the only thing is we have applied this logistic function instead of the normal linear regression
so that we can get this kind of transformation here.
This is the only thing which is important for you to understand here.
Now, in the next session, we will learn about different types of metrics which are available, and
then we will go ahead with the implementation of this so that you will get a better picture of what
you are trying to do here.