Oh welcome back.
In its lesson we shall create our model.
So first of all let's create Constanta for the um the various attributes of the model.
So I'm going to say annex number of pixels.
Number of pixels is going to be it's going to be um we know our image pixels was a sixty four.
I'm going around the script again just to verify when I run this bit to control us and then I'm gonna
come over here run module.
Okay.
Sixty four by sixty four by three.
Okay so a number of
number of pixels is going to be this total number of pixels.
It's gonna be this by this by three.
Okay.
And X so I'm gonna print and X to see what I get
simply going to comment this out.
Control is run more module Mm hmm.
Interesting.
Why do I get this
the
is because of my typo here.
That should be multiplied by three.
So with by height by three Chanos.
Okay.
This is what we have.
So this is Centex.
This total pixels of a single image.
This is quick.
This is also the size of our column vector.
Okay.
And over here we're going to set the the number of hidden um the hidden layer length.
Meaning number of nodes in the hidden layer n h here.
So you can just set it to any number.
I'll start by six so you can change this value if you want the number of nodes in the hidden layer to
be 10.
You can set it to 10 and output layer is going to have a single node so I'll just pass one over here
okay.
And this.
So these um yeah we should use this in our function.
Also we can set lower dimensions we can create a variable called Layer dimensions
and then this is going to be
an X and then an h it in
and then in y like this
so we can just make sure the person we can just make sure the use I can pass the dimension.
This list can be passed or functional we should write
I just plural.
Okay.
So now we're going to write a function to take everything we've created
Okay I'm going to come over here and see if I'm gonna call the function to Leah and Ed
Leah and then Modo
you sped up with this okay.
So we're going to take our input on what training data sets their labels.
Which is why.
And then we're gonna take layout dimensions over here and then we going to take the lending rate
this learning rate to we can set it to a constant in case the user doesn't provide a learning rate we
can start with this and then number of iterations.
And when I say no underscore
iterations set us to three thousand five hundred.
Okay.
Right.
So we start off by um let's create some some list to hold um some of the the things that we shall compute.
So this function is going to return our parameters which are the m the weight w 1 the bias to be one
way to W2 and then by is to be two.
I'm gonna hold a gradient over here.
I'll have Gratz and open close and then have this total costs
and then I'll go up the number of training examples storage in a variable called M O say m e course
shape of X 1 like this and then I'm going to extract what is past this is going to be a list when X
extract what is parsed here and put into three variables.
So I'll see and underscore X comma and underscore H and underscore why of course there's dimensions
like this once that is done.
I'm going to call our initialize parameters function from our library.
Let's see what it looks like.
Okay.
So it takes an X in each and in y
I'm going to close our help but we don't need it.
Okay so I'm going to store it when I saw this in a list called parameters.
The return is gonna be stored in this variable code parameters here.
And so I'm going to say parameters equals
initialize parameters
and then the score X in on the score H.
And then the score y like this.
Right.
So after this after running this you have parameters here.
So now I'm going to take what is in this.
This actually is a dictionary.
This is not a simple list.
It's a dictionary Lucio function.
Yeah.
The function returns this dictionary called parameters.
So this is going to be a dictionary and we can extract the m the y one with a key y one y to the A key
y to be one etc. It's a key vote key value pair.
So I'll come over here and extract the results I'll see why 1 equals
parameters
y 1 be 1 because parameters
P1
I kept saying why this is totally one sorry border.
I'm losing my mind so they stop you one by one weight bias weight bias.
And I said Why.
Why for a while actually.
So w one of course parameters
w two equals to be two.
Okay so the last speed to be true cause
parameters V to right.
So now we have our weight and bias for the two layers.
This is the architecture again we used to be one over here and then we used to be two and B two over
here w one and B one over here and then over here.
Okay so now we can create a loop of a gradient descent loop or c for i in range
zero to number of iterations
I'm going to I'm gonna perform
our linear forward propagation
or perhaps rather than take a two step approach we can perform the propagation and activation at once
so we have linear we have linear activation forward.
This one takes the input or the previous a value the weight the bias and then we call our linear forward
propagation here.
Right.
Good.
So we can just call this one or copy this.
This is from our library and I'm gonna come down here and this is going to take x w one and it's gonna
take the bias
B1 and the activation function we going to use is there really and this is going to return it's gonna
return a dictionary and no let's see this is going to return to cash the cash so list is going to return
to cash and a single scalar value which is the activation right.
So I'm going to store this I'm gonna have I'm gonna come over here and see you store this in a one and
then store the cash in cash 1 right.
So now we're going to have a one and then cash one.
We're going to take a one and pass it through the neural network.
Can I take a one pass it through here.
Right.
We've computed this bit.
Now we are on this side.
We pass it through a sigmoid.
There should be a sigmoid.
Not really.
Okay.
So I'm gonna come over here and call the same function.
I'm gonna pass a one and wait matrix to W2 over here and V2 and the activation function I'm gonna use
is the sigmoid and this is Green to return A2 and then cash to like this.
Okay.
So once that is done we can compute the cost.
Let's see the function we created for compute in a cost as simple as this computer course we pass y
Hudson y value remember y heart is the same as a two y how just like the last activation so I'll copy
this compute cost and then when I come over here and pass a 2 and then Y and the return is gonna be
stored in a variable that we shall call cost like this.
And once that is done we can initialize these variables for back propagation.
So now let's compute a two.
So we say t h to you cause this is what we are it we say we use the N P door to divide.
Um n p divide over here and we do Y comma A2
and then minus n Pitot.
Divide
1 minus Y comma one minus A2 then put at minus here.
Right.
So this will give us a 2
once this is done we can take D E 2.
I mean d h you know a 2 the derivative of a 2 we can take the derivative of a 2 and then the content
of our cash 2 and then pass it through a linear um or back propagation function.
S or whether we have a function for the linear back propagation
let's see the any above what
Yeah we have this function linear activation.
So I'm gonna copy this from our library and then come over here.
And this and this function is gonna take us I comment to a 2
and then cache to remember the content of cash to let's see the content of cash to cash to is given
to us from our forward propagation
de cash to host to the linear cash in the activation cash
we want to see what a linear cautious we simply go to the linear forward function we wrote here the
forward cash to house the he has the weight the bias and the input the weight the the parameters as
well as the input which is a year the previous a course that's what we pass we do a W plus B to get
c w dot E plus be just like a W plus B to get C and then we take a we take a W and B put into cash that
is the same thing we are getting and that is what has permeated through the various functions to arrive
here so we take cash too and we apply our activation we're gonna use the sigmoid
over here and then once that is done this function is going to return three you're gonna return um three
variables the derivative of a one derivative of w two and derivative of um b Two we can verify this
from the function in our library
okay.
So these are the return values to a previous dy W and then DP.
Okay so I'm gonna come over here collect them and put them in a variable I'll have the A1 W2 cause we
say D a previous so it has to be one less than the others W2 and then D 1 B.
Okay.
So once this is done we've got to take what we have.
And then pass it through our next activation for.
Oh I really do.
So I'm gonna call the same linear activation part quote again
I'm gonna pass a one I'm going to pass D.A. 1 Sorry
a one over here and then cash 1
and then relative activation over here and this is going to return a previous data if you want and then
D uh DP 1 2 previous A's the it's gonna be D is you pretty equal to sign here and then we're going to
have the w 1 and then DP 1 Okay.
Right.
So this is going to be this the end of the back propagation we can take the gradient and store them
in a key value pair the empty one that we created here.
No no cigarettes.
So I'm gonna come over here and then I'll see crowds.
I'm gonna give this the key detail you one I'm sorry bother
GW one in crowds
dp 1
this is going to hold that dp 1 crowds
key W to
this is going to TWD to grads
DP to this is going to hold.
DP to right.
So once we've done this we can update our parameters.
Let's see or update parameters function come off here.
Update parameters takes the parameters the gradient and the lending rate.
I'm gonna call this function from our library.
Come over here paste this over here.
The first argument is parameters.
Gonna copy parameters pasted over here.
The second argument is the gradient.
I'm going to copy the grads pasted over here.
The next document is the lending rates.
I'm just gonna take the lending rate the user passes to this function.
Copy this paste this over here and let's see what this returns this update.
Parameters function.
Okay it returns.
Yeah.
Well it would return a dictionary of the new parameters.
So I'm just gonna store this
parameters like this right and then I'm going to um I'm gonna retrieve the parameters and put them in
a variable when I retrieve the new parameters.
We have a dictionary that will be retained.
I'm going to take each item out.
So do w 1
equals parameters w 1
p. 1.
Course
parameters
P1 w 2
equals parameters.
W two and then B2
equals parameters B2 Um actually we are done here or we can do is provide some visual feedback as the
model is being trained.
So we are going to print that cost for every 100 training examples.
So I'm gonna come over here and see if if by percentage one hundred of course equals zero then will
print the cost print
cost after iteration
and then I'm simply going to pass the iteration number in the cost.
So I simply do it for months and then I over here then end p dot squeeze we're going to squeeze.
And the reason we squeeze I think I explained this before I explained again the reason why we increase
is sometimes our return values have excess dimensions.
We could have a value such as this 20 after a particular you know after computation or after doing something
we end up with something like this.
But what we want is just 20.
So when we you squeeze this becomes just 20.
If we pass the 20 this way to our function it would be rejected.
That's why we sometimes you squeeze
right um if someone in the in the course has a better explanation you can share it India in the Q Any
s..
Okay so we're going to do this and yeah we're going to append this to the cost um we've got a list for
cost.
Yeah we're going to keep spending to it as well.
So I'm gonna come over here and then um I'll see if
I'll just append it in the same block I'll see cost dot append and I'm going to append our cost over
here like this.
Also we can provide some visual feedback in a form of a graph we can actually make it plotted graph.
So I'm gonna come over here
sorry I'm gonna I'm trying to set my indentation column before.
Okay so I'm going to plot I'll say BLT the plot
and then R2 and pedo squeeze again to squeeze my cost.
And this is the cost list we need to audit we need to append to various costs so that we'll be able
to use it to plot a graph then or do I'll do peel T dot y label in addition to y label is going to have
the word cost and then peel T dot X label
and this is going to be iterations
and this is going to be when I come over here and to peel t to try to
give this a try to I'm gonna cut the learning rate here and in great and we simply take what is provided
by the user
and then peel Tito show and then we can return the parameters of the model so that we can use we can
use it for prediction
right
so um let's conclude this lesson here in that next lesson we should run our model and then once it's
done training we should run some predictions to see its performance so I'll see you in the next.