In the section on corrective measures.
We shall start by commenting on our loss and validation plots.
So we saw already that our model was underperforming, and looking at the loss versus epoch plot, we
could see clearly that our loss values were much higher than what we could potentially get.
And so this kind of scenario where we have both the training and the validation.
So here we have train and then here we have validation.
So when we have this training and validation plots um or training and validation loss values which are
way higher than what we would expect.
In that case we'll call this um under fitting.
And this is because in the worst case scenario, our model at least should be able to learn from the
training data.
Well, here we could interchange this.
We could have the validation and the training below.
But the point here is that the model underperforms in both the training and validation sets.
Now generally when we have this kind of scenario where the model underperforms even on the training
data, this means that the model is not suited for the problem, or the model may be too simple.
And by simple here we mean that this nine parameters m one to m eight plus c that's nine aren't enough
to learn from our 1000 samples data set.
That's actually to get all the patterns necessary to be able to get, um, a single sample like this
one and then predict a reasonable price, which is close to what's expected.
So when our model is underfitting and not able to learn these patterns from our data set, what we do
is we build a more complex model.
And now this more complex model will be made of many more layers, hence rendering our deep learning
model much deeper.
So let's take this here and just pull this up here.
You see we have all these inputs.
We have this weights and then we have this output.
Then instead of just taking this as our output we're going to add this other three two outputs.
So now we have three outputs.
Let's pull this up here we have this one C we have this one.
And notice how each one has each input is linked to um every other output.
So take this here and put out here C.
Again each and every um input here is linked to all three outputs.
And so what this means is we are going from eight weights here.
That's keeping aside the bias.
We're going from eight because we had um m one to M eight now 224.
So instead of um eight we now have 24.
And then on this other side, what we'll do is we'll take this again.
So let's, um, drag this here.
There we go.
We have this one.
There we go.
And then we have this other one.
Well, we also have this other one.
So let's just pull this up here.
And that's fine.
Again we have all this linked to um one another.
So you see that.
Now we have um here and this um intermediate layer.
We have 24 parameters.
And in this one we have three times four.
That's um 12 parameters.
Here we have 12.
Here we have 24.
And then for this last layer we have um four times one that's three.
So let's get back.
Let's drag this up here and there we go.
So you see that now we have a much deeper model with several hidden layers.
We have now hidden layers.
And then we have our inputs and the output.
And one great thing about TensorFlow is that if here we had dense that's we had this dense.
And then we placed uh, we had a single output.
And, and this was in a list.
Now we're going to have that same list with a dense.
But now we have um three outputs.
So here we specify three.
And then for the next because we, we go from three.
That's this here we this layer, we have three outputs for this next dense we're going to have four
outputs.
And then we'll end up with um a dense with one output.
So we have four.
And then we have a dense with just a single output.
So here we have dense and we have one output.
So we're going from this line of code to this other one, which doesn't change much at the level of
the coding because we just have to add this.
But actually our model has become much deeper by deeper here by increasing the number of layers as we
go, um, in this right direction, we go deeper and um, our model becomes much more complex.
And so it's able to learn, um, the patterns in our data set, much easily now as compared to this
very simple model, which wasn't capable of learning those complex patterns in our data set.
So far we've been dealing with just linear functions.
So, um, essentially all what we've been doing is MX plus C, which is a linear function.
But to increase the complexity of our model, we could add some non-linearity, which we'll call big
N.
So if we add a nonlinear function or if after getting our outputs here, that's for each and every neuron
we have here, we pass this into a nonlinear function.
Because obviously this this is a linear function here.
If we pass it into a nonlinear function, we shall make our model overall much more complex and hence
could learn more complex patterns in our data set.
Some of the commonly used activation functions are.
Nonlinear functions are the sigmoid, the tanche and the ReLU with a ReLU, for example, is just um
y equal x if x is positive and y equal zero if x is negative and x is e x minus e minus x divided by
x plus e minus x going from -1 to 1 for the sigmoid.
We saw this already.
Um, it's one on one plus e to the negative x and it goes from 0 to 1.
So when we compute m x plus c.
This year becomes our X.
And then we pass this in here.
So we pass this in here.
And then uh we now obtain the sigmoid.
Same as with the tangent.
We take this x, we pass it in here.
We pass it in here, we pass it in here, pass it in here.
And we obtain the tangent.
And then if it is um, the with the ReLU it's going to remain the same.
Um, if all of this is positive and if all of this is negative, then it's going to be zero.
Dive into the code.
We're just going to change this and say 128 for example, we could have 128.
We're going to stack three of this.
And then with one output.
So we have 128 128.
And then our output must be one because that's the nature of our data.
So because our data um expects to have some input which is of the shape, uh, and also the output must
be of um, this shape.
So if you don't have this kind of shape, then you would get an error.
So that's it.
We just stack this up like this.
This is very easy.
You could stack as many as you want.
That's all.
Uh, you need to do.
So there we go.
We have three of this, and then we run that.
And you could see now that we have 34,304 or 305 trainable parameters, we still have our 17 non trainable
parameters from our normalizer.
We could reduce this.
Let's say we take um 28.
Or we could also have different values.
So we could take 53 for example.
And you see we have fewer parameters.
Anyways let's get back to 128 128 and 128.
And we run that.
We have our summary.
We check out our plot.
You see now that it's going to be different, um, from what we had uh, previously because we've made
some changes.
Then we go ahead and compile the model and we start with the training.
Let's train for about, um, 200 epochs.
After the training is complete, you could now see that we have a much better performance as our loss
values have, um, dropped substantially.
You could also check out the root mean square error.
You see, there's this great drop compared to what we had previously.
We evaluate and then we see check out this um plots.
So you see now that the actual and then what the model predicts are not too far from each other as compared
to the previous model.
Now let's go ahead and add the non-linearities and then retrain again this time around just for 100
epochs, as we see that after even about um, 20 epochs, the model doesn't change much.
So let's go ahead and train for 100 epochs with.
Now the non-linearities added to other non-linearities is simply activations.
So you have activation and then you specify ReLU.
That's all.
So here we have activation and there we go.
We have activation and we have ReLU um activation and ReLU.
Now for this last one we don't need to have the activation as our values should be um all positive.
So we just, um, let that and that's fine.
Let's run again and recompile and start with the training.
Let's change this to 100.
Um, there we go.
And then we run all this so we could check those out after.
Now we actually forgot to note the the the the y test values for the root mean square error.
As you could see, we have, um, better results.
Uh, we could go ahead and check out those, uh, charts.
Let's run this again.
And there we go.
We have our charts indicating that our model now performs much better as compared to our initial model.