So in this lecture, we will be continuing our discussion about the intuition behind logistic regression.
Now one thing to be aware of is that logistic regression can be used for multiclass problems as well.
This is relevant for this session, since the dataset will be looking at has more than two classes.
Sometimes people call this multi nominal logistic regression, which is, in my opinion, a poor name,
since there is no point at which we work with the multi nominal distribution.
There are other names as well, such as maximum entropy, which are all equally as bad.
My preference is to simply call it multi class logistic regression or simply just logistic regression.
In any case, here's how this method works.
Suppose that we have classes, then we will have Quade vectors, let's call them W1 and W2 and so on
up to W.K.
Note that in this case, these are not components of W, but rather each of these is a whole de size
vector.
Furthermore, suppose that we also have corresponding biased terms B1 B2 all the way up to b k.
From these, we can then compute K activations using the usual formula w transpose X plus b.
So this gives us A1 A2 all the way up to 8K.
The next step is the critical part, instead of applying the sigmoid function, which is only used for
the binary case.
We use the soft max.
The soft max function has the formula you see here.
As with the sigmoid, we interpret the output as a probability.
Specifically, if we have little K in the numerator, then the output is the probability that Y belongs
to class little K.
Now, if you have any math phobias, please don't worry as you don't have to solve any equations.
The important part is just to be able to see how it works.
Firstly, notice that we exponentially all the activations.
Therefore, every element becomes positive, which is a requirement for this to be a probability.
The second thing to notice is that on the denominator, we take the sum over all possible values of
the numerator.
The result is that the sum of all the probabilities will equal one, which again is a requirement for
this to be a probability distribution.
The third thing to notice is that there are such probabilities.
As you can see, the little K in the numerator is an index in the possible values for it are one of
two Big K.
So I hope you're convinced that the SOF Max is an appropriate function for converting activations into
a probability distribution in the multi class case.
So due to the facts about soft Max from the previous slide, we should be able to infer what the outputs
of the multiclass logistic regression should look like, in particular for each of our samples, we
should get key different probabilities that all sum to one.
Thus, if we have any samples in key outputs per sample, then our overall output will be a matrix of
size n by K. As you recall, this is also what we get when we call model, predicts Prabha inside and
learn.
Now let's consider what we would do if we wanted to get the actual class predictions instead of probabilities.
The answer is to take the AMAX over all the probabilities for a given sample.
That is, we choose the Class K star that yields the maximum P of Y given X.
Note that this will yield a one dimensional array containing the class labels.
So let's do a short quiz to test your understanding of this concept.
Consider this model output.
It's an array containing the numbers zero point two, zero point five, zero point one and zero point
nine.
The question is for multi class logistic regression, is this a valid output?
Please pause the video and take a moment to think about what the answer should be.
OK, so the answer is that this is not a valid output.
Firstly, notice that this is only a one dimensional array, which doesn't make sense.
As you recall, we expect a two dimensional array of size end by the number of rows is the number of
samples, and the number of columns is the number of classes.
Furthermore, note that these values also don't sum to one, so they couldn't have been the distribution
for a single sample.
OK, so let's do another quiz this time we do have a two dimensional array.
It contains the numbers that you see on the slide as before.
We want to know from multiclass logistic regression.
Is this a valid output?
Please pause the video and take a moment to think about what the answer should be.
OK, so again, the answer is no, this is not a valid output.
Note that this matrix does meet one requirement.
It is a two dimensional array, which is what we learned about from the first quiz.
However, note that in the second row, we have a negative number, which is not allowed because probabilities
can't be negative.
Furthermore, note that there is no way the soft max can output a negative number if its inputs are
real numbers.
OK, so let's do another quit this time you can see that not only do we have a two dimensional array,
but all the values are positive as before, we want to know from multiclass logistic regression.
Is this a valid output?
Please pause the video and take a moment to think about what the answer should be.
OK, so again, the answer is no, this is not a valid output.
Note that this matrix does meet several requirements.
Firstly, it is a 2D matrix, which is what we learned about from the first clips.
Secondly, all the values are valid probabilities, which are between zero and one.
The reason why it's not a valid output is because not all the rows sum to one, and hence they are not
valid distributions.
In particular, the third row contains zero point five, zero point seven and zero point one, which
adds up to one point three.