Hello.
Welcome back.
In this lesson we're going to write a function to help us initialize our weight and our buyers.
The B parameter.
I'm going to delete the comment from here to create a bit more space and then I'm going to put a function
up here.
I said def
initialize
and a sure initialize zeros and then it's going to take the dimension.
Simple as this.
And then what we want to do is use num pi.
We can simply say W B which is screenshot which are going to be the return.
And then we say over here num pi dot zeros.
And over here we can simply
pass dimension by one over here and then zero.
Once this is done we can simply return you b so return w come up b here and then we can test this out.
I can come over here and see.
Way to come up by S E course.
Just name of the function Initialize zeros and then I'm going to see a dimension I'm gonna pass a dimension
of 3 and then what I'm going to do next and I'm gonna see a print
wait e course plus SDR wait
then print
by as equals
bias like this.
Okay let's run this and see
why are we still printing this.
Anyway as you can see the weights have three dimensions.
The bias has a single dimension right.
Okay looks good.
I forgot to clean this print.
Okay
I'm gonna clean this now.
Okay.
So once this is done um we can go on to write our forward propagation function but we're going to need
a sigmoid there.
So we have to write a function to implement the sigmoid in Python.
So let's do that now.
I'm gonna come down here or write a function called sigmoid.
This function is gonna take a variable X and then what are we going to do is say that um we can see
why the cause 1.0 divided by one plus num pi exponential XP minus X like this and we can return y
right.
So this the sigmoid function and we saw we saw this array This is the equation for compute in the sigmoid
is the same thing with implemented here.
1 over 1 plus e rich to the poor the argument pass their function over here it says c we are passing
X it s fine.
Okay.
Right.
So once this is done we can go on and write our forward propagation function.
Actually you're wrong.
Don't you know.
Right.
Forward and backward propagation you know separately.
How about we just put it all together.
We can implement it in one function of C diff.
I'll call this propagation
and then we going to pass the weight the weight vector.
The bias which is s color value our input and then our labels which are the y values.
Right.
And let's start by extracting the um the M value also M because X shape and then we're gonna extract
it from here like this.
Once that is done we can perform the activation and we saw how to compute the activation.
Let's take a look at I was
right.
We said we compute ze by taking the dot product of weight and input input x here the Dutch product of
the input x and transpose this t here implies the transpose of the weight.
Once we've taken the dot product of the input x and the transpose of the weight we are by bias P and
then we get Z.
When we take Z and we pass it as an argument to the sigmoid function then we get the value over yet
it is showing y hearts because it is the final a value because it's the final one but this is how we
get activation.
This the final activation that is why it's written here as y hat.
Okay so we're going to implement input right plus B and then pass it through the sigmoid and then we'll
find a right.
Okay.
For those of you who don't understand I said Of course various images here.
So I'm saying it here and Y hats are the same.
I've written y hearts as we saw in a theoretical class it's y hat here because it's the final activation
of the neural network.
I'll show you a slide that shows that this is the same as a show.
This is the slide from our workout neural network where we're compute in hours of work out and hours
of rest.
Over here as you see we have a here and then we have y hat is because over here we had we had a hidden
layer before the final activation there.
That is why we have a and why how to bind our logistic regression.
We have just one activation.
Okay.
So this was a neural network or a logistic regression you would know what looks like this is what you
make we convert it to a vector like we've done already and then we feed it into a neural network we
feed it.
This the inputs layer and then we just pass through a single activation and then we predict if it's
greater than 5 if it's greater than zero point five meaning it's a cut if it's less than zero point
five.
It's not a cut.
So this was a our neural network looks like that is way over here.
Why hat is the same as a.
We don't have a one and then a 2 which is the same as Y hat if we have a hidden layer.
Then we would have a face activation and after that activation they activation after that one which
could be the final one will be our y huts.
But over here we simply have our input layer and then we multiply by the wheat and the buyers and then
we pass it through our activation function.
Right.
So this on your network let's continue with implementing the back propagation.
So we came over here we said we've got to compute the activation which is a.
So I'm going to say the course we're going to call the sigmoid function we wrote sigmoid and then we
said we we compute the z value by taking the transpose of the weights and then compute in a dot product
with a weight transpose and the inputs.
Once we've done that we added bias and then that gives us Z to get E we pass Zi to the sigmoid function
and the output is a.
So over here I'm gonna call our sigmoid function and the argument here is NPR the dot dot for the dot
product operation and when I do the beauty for transpose of weight and then X over here.
So this will give us you know this will compute the way to transpose dot product X after that plus b
here and then all of this will be applied.
We apply the sigmoid to all of this what comes out is the value.
Once this is done we can compute our cost.
We saw the equation for our cost function.
Let's take a look.
Let's take a look let's look at it.
So does the equation for our cost function.
We are going to write this equation in Python right.
Okay so um the return is gonna be stored.
Here it is.
We said let's see let us see again.
So it starts with minus 1 over m m is the number of training examples and then we take the summation
of this entire thing.
Okay.
Oh 2.
Sorry Buddha gotta go minimize this a bit.
Please or equation here.
Okay so I'm gonna come here and I'm going to say minus 1 over M times and then I'm going to use num
by NPD or some that would give me the summation and Pete at some and what I'm gonna do is open brackets
and within this brackets I'll start off with Y times log a
and it is the same as Y hut.
So remember that.
Yeah.
Y 10 times look a plus 1 minus Y times and p dot log 1 minus B.
Like this.
Let's verify.
We have minus one over X times and put out some brackets Lupin y times and P does it look a plus 1 minus
Y times.
Y here.
That should be a dot and P dots look one minus a.
Okay I'm gonna open this again.
Right.
So this what we have this going to be this is our cost function like we saw in a theoretical class.
Okay.
So once this is done we've got to perform the back propagation we've got to compute D W and W A.B. and
we can take a look at this slide again for those y.
So we compute D W by one of these z times X transpose over here.
Right.
And then um okay.
I should mention if if you've not seen this slide before then it would be that I may have re arranged
the course but I would try to keep the current arrangements of the cause the current arrangement of
the course implies that we have studied this in the theoretical class before get into this practical
part but often at times after I apply the Course students and suggestions regarding how the arrangement
of the course should go and I tend to end up rearranging stuff but our endeavour not to add outputs
all day three or day three core lessons at the top so that you go through them before you get to the
practical part.
So it's gonna be some form of long fourth coalescence before come into the practical part.
Um yeah to see um this code in lesson but I hope you get a point anyway like we saw earlier.
If you've seen the theoretical part about how to perform back propagation this is how we compute empty
w this equation for D W and we know DC we compute DC by simply doing a minus Y.
Right.
So I'm gonna minimize this so I'm gonna come down here C D W E cause 1 over m times and P dot
X comma and then DC is a minus Y transpose that's what we saw in the equation and we compute TB as well
like we saw in the theoretical class 1 over m times and P dot one of our M times let's verify that DP
is summation of DC okay N.P. daughter sum and then DC is simply a minus y right.
So this is going to perform the back propagation for us to then once that is done we are going to I'm
going to squeeze it I'm gonna see cost e course and pedo squeeze and this you squeeze the m the dimensions
for us
and then once that is done we are going to store this in a in a dictionary type I'll say it grads this
so that we can simply use the word W and it will fetch W for us and we can do the same
for D.B.
okay who started this key valued pair.
Yeah.
DP WDC and then
we're going to call this gradient.
So then we can simply return gradient in the cost
to receive an arrow funded station here
right.
So this is going to be our propagation function.
We go through forward propagation we calculate the cost and then we computed gradients.
Okay
so in the next lesson we're going to write a function that would learn from the data and update the
weight and bias to sort of reduce error.
We're going to write a function that will take the gradient descent and sort of Len or create our outputs
more model for us.
Okay.
So in the next lesson we're going to continue.