Hello.
Welcome back.
So in the last listen we saw how to build a to a Europe network and then we used the functions we created
in our DNA and that's called lip version one script to create on your own network.
In this lesson we are going to create functions that would allow us to be able to use a neural network
size that we want.
The first one was just to neural networks.
Now we're going to write a function that would accept the dimension of the of the layers such that you
can have a deep neural network with three hidden layers for hidden layers five hidden layers as much
as you want.
We're going to create a function that allows us to automatically include multiple layers rather than
fix it to two layers only.
So I'm gonna make a copy of the last project.
This one here I'll just copy and paste the photo here
and I'm going to rename this.
I'm going to rename this to
multilayer and then
right.
Can gonna open it.
I'm going to rename CNN leap.
I'm going to change this to version 2 and then this over here to layer.
I'm going to call it Morty here
right.
So let's start with our library.
Going to right click right click open with I do.
Right.
So let's see over here.
Now I initialize parameters function as fixed for this size.
We have an input an input layer a hidden layer and then the output.
Now we want to be able to have multiple layers.
So what we're going to do is we're going to write a new function where you can simply pass the D layer
dimensions and based on the argument you pass here it's going to determine it's going to initialized
a number of layers you select for you automatically.
So we would have to write a new function that would do that for us so I'm going to come up here and
then I'm going to see def
initialize
initialize parameters so call it
multilayer right.
Okay.
I think there's a very descriptive name and this function is going to take the estimations as I comment
right.
So we start off by I'll see to my random number generator and P random dot run in
just to random the seed actually not run down.
Okay.
And then once that's done I'm going to create parameters here.
I'm going to start the parameters in this dictionary when we are done.
I'm going to just initialize it to be empty
and then
I'm going to get a number of layers by finding the length of this list.
So this is going to be a list and in the list we're going to give the dimension of each layer and a
dimension means the number of nodes.
So if our input node has a thousand.
If we have a thousand input we'll have a thousand and then the next layer.
Let's say the first aid in there.
If it has three nodes then it's gonna be three.
And then if the one of two that has five nodes it's going to be five.
So he's gonna be a thousand.
Come on three.
Come on five that's gonna be like the dimension.
So just by taking the length the number of items in this list that we shall call list dimension we can
know the total number of layers of the neural network.
Right.
So I'm going to create a variable capital L over here and I'll say this is equal to length of this argument
which is going to be passed over here.
Right.
So once we've done this we can open a for loop.
I'll say for l in range from 1 to Cup 2 L
and I'm going to see parameters
w plus S T R L
E course then remember we generate in the m the weights so we use random and P2 at random dot round
and decides is going to be the layout dimensions I read or list index index l by
layout dimensions
have to be careful with the name the spellings so the argument is called layout dimensions so they add
dimensions over here the estimation is they add dimensions.
Okay index they are dimensions
L minus 1
and
I'm going to divide it by a square root
of the previous layer.
The previously estimations which is L minus one right.
I'll reshape this a bit.
So once this is done we can initialize the bias and we do parameters.
We're going to give it a key of B like we did earlier
plus SDR index L
because we initialize the bias to 0.
Source I'll do MP dot zeros
and then the dimension is low dimension by 1
right.
So next we can simply return to parameters
so return parameters here like this.
Okay so we can just pass we can just pass a 20 layer neural network and this will be able to generate
the weight and biases for us right.
So does the first addition we've added to our library.
Are we going to create.
Let's see.
Well we have everything we need.
We we can reuse almost everything here.
Okay.
So over here we have a model forward prop.
Okay let's write another function for back prop I'm going to come down here and write a function I say
def model back prop
so we going to use this to perform debug propagation
and this is going to take it's gonna take three argument it's gonna take the M L which is the activation
of the final layer which is the same as y hat y our labels y and then some caches.
Yeah the cautious we pass.
Okay so let's start off by
I start off by creating some gradient.
Create this list to hold a gradients and I'm going to get the length the number of layers from the um
the length of the list that would hold a cautious associate length of cautious
once that is done.
I'm going to reshape y to be the same shape as a L or C Y equals Y dots reshape and reshaping it to
the shape of it.
In a sense we're making y the same as the shape of y hat.
Now I'm going to compute d e l e l e course
and P divide we divide in here.
Y comma AOL and then minus and a minus here
minus n Peter to divide
1 minus Y comma one minus a l like this
right.
So I'm going to create a variable quote current cash
and I'm going to grab elk minus one
crushes elk minus one.
And once that's done I'm going to um I'm going to set the content of the gradient.
I'm going to sort of collect.
I am simply going to call out linear propagation function and then collect the return value some put
it into the gradient.
So we created a function earlier known as linear activation Buchwald over here.
I'm gonna call this and this function as you can see it returns the previous the W NDP.
I'm going to store this in a while gradient dictionary here so I'll call the function.
The function is going to take a loss argument that is the previous the a and then the current cash
and then the activation we're going to use is the sigmoid
okay.
So I'm going to store the return in the dictionary.
I'll come over here.
It's going to return d w e and DP.
So I'm gonna just see gradient
D a comma D A plus cause if we're dealing with the iterator o append L over here and over here like
this this the um the last layer
and the next return value is the w the next return values the top abuse or simply do comma creates and
then when I start this I'll give it a key w plus
string L then the next one is DP
DP plus.
So this is what we lost.
This is for the last layer capital L.
Now we have to just use the we have to iterate to do all previous layers L minus 1 L minus 2 L minus
3 etc..
Right.
And we simply call in our linear activation backward function.
This function returns the B TWD a We've stored them in our gradient over here.
So I'm going to create a loop to do the rest.
Or C for l in reversed
range
L minus 1 like this.
I'm going to do current cash of course L
because cash is cautious it's going to be past us like you meant to the function to a cautious index
l will be the current cash and then once that is done I'm going to call our function again
reach call a function Lena activate DNA activation backwards.
Copy this piece over here.
The first document is to be gradients the a
secret key to a plus SDR L plus two one after that
the next argument.
We have three arguments so we've got a positive cash.
This is previous the E and then after that the next argument I see the function the next argument is
the cash the third one is the activation type so the cash is current cash and current cash is taken
the edge of the cash no l in our list of cash is so current cash
and then
activation equals random
linear activation backwards.
Okay.
So we're going to return the um the return we're going to keep them in temporary variables I'm going
to have a temp over here and then uh I'll call this the a previous temp and then t w w temp and then
to be temp
right.
So now when I come over here and then I'll see gradients are dictionary D A plus SDR plus 1
equals D a previous temp.
You can see where this is going.
I'm gonna do the same for DP but just copy and paste.
I start with the W the W four here plus L plus 1.
This is simply D W temp and then DP
This is simply DP temp
right and we're going to return to gradients
okay.
So these are the two new functions we need in order to be able to perform the same computation with
multiple layers rather than just two layers.
So when we go to our our other script we're going to call these two new functions.
Right.
So this is all there is for this this lesson and if you have any questions just send me a message I'm
leaving the questions area.
The next lesson we should develop our new neural network have in more than two layers.
And yeah we shall call these two new functions that we've created.
I'll see you later.
Have a nice day.