In this lecture we are going to look at an alternative and better way of building a logistic regression
model in PI talk as usual.
You can look at the title of the notebook to determine what notebook we are currently looking at the
motivation behind doing this is that when we perform the sigmoid we have to do an exponential operation.
The problem with that is that exponential is our numerically unstable due to the fact that they grow
very fast with respect to the input.
In addition the long term in the binary cross entropy is also unstable for similar reasons.
Well it turns out that because the binary cross entropy loss has a log in it it almost cancels out the
exponential in some sentence although we won't go through any derivations in this course.
The end result is that if you know you have to do both the sigmoid and then the binary cross entropy
loss right after there is a way to combine them into a single function so that you avoid numerical instability
in particular.
You can express the loss only in terms of the activation which is also called the logic.
In this case if you've ever studied statistics then you may have heard of this term before.
Otherwise don't worry.
Basically the logic is just the input into the logistic function and the logistic function is just another
way of saying the sigmoid function.
So in this script pretty much everything is exactly the same except in two places the first place is
when we create the model
so before we use the sequential and combined the linear layer with a sigmoid layer.
But in this script we no longer need the sigmoid since that's included in the cost calculation and therefore
our model goes back to being just a linear model.
The output of this model is the logic.
Then when we create the lost function we use BCE with logic loss instead of BCE loss like we were earlier.
As its name suggests this function calculates the binding across entropy loss directly from the lodge
it's
as you can see everything after this is the same as well.
So our training loop is the same and we get the same results.
One more difference we have to pay attention to is when we make predictions.
If you recall the sigmoid always outputs a number between 0 and 1 and we interpret that as a probability.
So when we want to make a prediction we simply round those probabilities.
Anything greater than zero point five is a prediction of one.
And anything less than zero point five is a prediction of zero.
But what do our predictions look like now since the output can again be any number.
In this case we just go back to our geometrical picture.
You can also think of what the sigmoid actually does.
And if you follow that logic this should make sense.
If you recall anything on one side of the hyper plain defined micro model is positive and anything on
the other side is negative.
The sigmoid simply maps all the positive numbers to probabilities greater than zero point five.
And similarly it maps all the negative numbers to probabilities less than zero point five.
Thus it's very easy to make predictions.
We simply check if the model output is greater than zero or less than zero.
Anything greater than zero is considered a one prediction and anything less than zero is a zero prediction.
Note that we can use the greater than sign directly even though the greater than sign returns truths
and falsities as we discussed earlier in python a true is 1 and False is 0.
Therefore we don't need to do any further processing.
Once we have our predictions we can check the model accuracy and again we can verify that the result
is the same.