So now let's talk about lost functions and why they are essential to the training process.
So remember the slide from a previous section?
This is this is where we put we input some images here into our randomly initialized neural network
and we just got some values which supposedly would be random values that tell us, give us the scores
for each class, each image class.
This one was a six or five or six eight eight one.
And you can see the random scores generated from the outputs of the randomly initialized neural network.
So what we need to do?
We know these probabilities are wrong.
Your scores are bad, but we don't know how bad they are.
We need to actually find a way to quantify how bad our predictions are or how good they are.
So we'll take a look at the method that's used primarily used in convolutional neural networks.
Although this this could cross entropy loss can be used in any classification problem.
It's not only specific to convolutional neural networks, however, because we use CNN's for classification.
This lends itself quite well to our use case at hand.
So imagine these our classes here for an input for image.
I was input.
And these are the predicted probabilities that come out of it.
So you have point one point to point one, some point zero five point three notice that this is the
highest one point zero five and point zero five.
And look, fortunately, the ground truth for that image was actually a seven.
And the probability was highest for four seven.
It was 0.3.
So that's good, right?
Well, technically it is good.
However, we need to find a way to actually use these probabilities to work out more, a more mathematical
way to measure how good or bad these results really are.
Because even though this is like a perfect result in theory, because the actual outcome is the predicted
class, because the highest probability was 0.7.
If I find that the fact that it was a close 0.2 was quite close for a one, for four plus one, that's
a bit unsettling, meaning that our CNN might have a problem distinguishing between seven and 1s.
So let's take a look at a formula for cross entropy loss.
Cross entropy loss basically uses two distributions of a ground truth distribution here.
It's VFX and Q effects of predicted distribution.
So in this case here, where Y is, the predicted is around it and Y hat is a predicted distribution
and the Dot is in a product and this is the formula we used to do it.
I will go through this formula in the next slide here.
So let's take a look at a simple example.
Let's assume we only are looking at three classes here.
Class zero, Class one, Class two Let's pretend it's cats, dogs and penguins.
So these are the probabilities for an image that was input into this.
So if this was a cat dog penguin, we think it's it gives us that CNN gives the highest possible probability
score of six for it being a dog.
So and then it corresponds to the ground truth here.
So it's good.
This is a relatively good result.
Yeah, but let's see what else actually is.
So by plugging in those values here, this is how we apply to formula.
So we have this is the distribution here.
So we have zero times larger point three plus one times log point zero, five point six and then zero
zero, by the way.
Are the ground your classes?
That's the other.
That's the other ground to the submission, plus zero times point one here.
This gives us this value here, and because it's a negative in front of it, here we the negative value
we get in here is cancel and we get a positive.
So we have point two to two.
This means that all loss has been quantified.
If this point six was a point nine and this was a point zero five and this was a point zero five, this
score, this loss score would actually be a lot lower and think about loss is indicative of something,
but sort of lower or losses.
The better of a neural network is performing.
So also know that multicast log loss rewards or penalizes to correct classes only as you can see that.
That's because these were zero here, and this is just another way to represent that formula.
So what about other loss functions?
Do they exist?
And yes, they do exist.
And oftentimes lost functions are called cost functions, depending on I think it was in engineering.
They used to cost functions a lot and in mathematics to use lost functions, terminology a lot.
So but it tends to mean the same thing when we're doing a binary classification problem, we use something
called the binary cross entropy loss, which is basically to.
Same thing, however, the formula a small tweak because it's only one up, a good that case for regressions
regressions, meaning that instead of predicting a class, we are predicting a value like let's suppose
we were to take an image and we needed to get like a a car value estimate out of it.
So that value that it predicts is a regression.
It's a continuous value.
So the regressions since we do not have classes to compare it, but we use something called the mean
square error, and that's how it's written.
It's similar a similar type formula where we just compare the output distribution to the ground, truth
distribution and get a score here.
They are the lowest functions.
Do there's L1, L2 hinge lowest mean, absolute error.
There's actually a whole host of lost functions, triplet loss and many others that probably escaped
my mind right now.
But contrastive loss is not a popular one I used.
But for now, the biggest focus you're going to be going to be looking with is cross entropy loss.
So now that we have a loss, what do we do with that value?
Well, we need to have the DeWitt's.
But updating the weights for when CNN isn't a trivial task because as you can see, there are thousands,
sometimes millions of widths.
How do we know what value to change those random weights do?
So that in the future, in the next iteration, we minimized the loss.
So we need to do an update that allows the loss in the future to be lower.
So how do we do this?
Well, we use a technique called impact propagation, which is quite brilliant technique, which we'll
discuss in the next section.
And we used a lost value for this to those values essential for making back propagation.
So let's stop there, and in the next section, we'll continue onto back propagation.
So thank you and I'll see you in the next section.