Hello.
Welcome back.
In this lesson we shall talk about factorization de Gaulle or factorization is to avoid explicit for
loops.
Let me explain.
We know to derive C. We compute w transpose x plus P. let's say we have stored a.
W values in a one dimensional array or a column vector and also our x values also in a one dimensional
array which is also known as a column vector like we can see over here.
To complete this we will use a for loop to do something like this.
This single block shall be used to compute a single z value.
We are doing this computing element by element.
This that this takes extremely long to compute.
In programming languages such as Python there are libraries that can be used to perform this kind of
computation over 100 times faster than pi library can be used to do this in number pi.
We can compute the entire Z vector without doing it element by element simply by doing an end p dot
w comma x plus B.
Just by using the N P dots function the NUM pi dots function we can be able to do this.
So this essentially is what we call the vector rise implementation using num pi.
Apart from allowing us to compute the entire vector without for loops the NUM pi library also uses the
same D instructions of the process or which makes it optimized for computations such as this one same
day is spelt as I empty and it stands for single instruction multiple data.
This allows data to be processed in in a parallel manner with future process or cycles.
Let's say we have some training examples too.
Let's say we have some training examples over here.
So to compute the prediction of the first train an example we compute for Z.
And then we use z to find a note that over here I am using actual brackets in the superscript rather
than square brackets.
This is to indicate that I'm indicating training example numbers not only numbers.
We can compute the predictions for the second and deferred by doing the same.
Like I've shown over here.
If we have let's say ten thousand training examples we need to do this ten thousand times we can complete
this faster by applying factorization.
Remember each example is a column vector.
In other words a one dimensional array the length of each example is equal to the number of features
which we call n subscript x.
We can put all our training examples together in a single matrix which we can call matrix capital X
like this.
Such that X superscript 1 represent train.
Example One X superscript to represent train an example 2.
I repeat the first column of the matrix represents the first train in example.
The second column represents the second train an example all the way to the last train an example which
we have denoted here as M.
Therefore the size of this matrix is n subscript X by m.
We can also put all the biases B together in a single real vector.
Having done this we can compute Z for all the training examples all at once by performing w transpose
capital X which contains all our training examples plus real vector B which contains the biases for
all d training examples.
If we take the capital Z and um this will give us a real vector a couple Jose as we can see over here
which contains the small as these desired results of all the training examples and if we take this capital
C and apply our activation function we get the capital E which is a real vector containing all these
small e values and the small e values represent the Ukrainian example a value that represents the a
value for each train an example.
So by factorization we can be using these capital letters just by putting all at once and compute and
starting over in a matrix and then compute in such that we can run it faster and earlier we deduce that
to find the loss with respect to Z we compute a minus Y which we call dizzy so the dizzy of training
example one is written here.
Ask these superscript 1 and DC of training example 2 is written here as DC superscript 2 we can put
all that DC together in a row vector which we can call capital Z.
See over here.
Similarly we can have all the A's.
We can have all the A's in a row Victor which we call capital a.
We can also have all the wise in the roof actual fridge free coke up to why we can complete disease
of our training examples by simply brewing down capital C because capital E minus capital Y.
We can compute to be of our training examples by finding B average of DC like this.
And finally we can compute D W of all training examples by finding the average of capital ex DC transpose.
We shall see how to compute all of this in code.
But before we do that let's see an example let's say we have a truly a neural network such as this one
over here.
As you know by now to find a superscript 2 which is equal to y heart we first find Z superscript 1 and
then use that to find a superscript 1 and then use a superscript 1 to find Z superscript 2.
And then finally use z superscript 2 to find a superscript 2.
This is for just one training example to execute this and code for m training examples.
This is how we shall implement it in code which will essentially perform these steps m times.
Just to remind you again x1 x2 x3 over here do not represent train training example.
These are the features of each train an example.
In other words each train and example has three inputs.
So training example one will produce y hearts one which is the same as a superscript square bracket
to bracket one as it is written here.
Describe brackets indicate the.
The layer number and the normal brackets indicate deep training.
Example No.
That is why they all have square brackets to hold the A's have square brackets too.
Over here in the same way you train training example to produce.
Why how to and training example 3 will produce y hot 3.
So to vector right this we take all our training examples and put them in a single matrix called Cup
2 x.
Like this we shall also keep all the ze superscript 1 result in a matrix like this the a superscript
1 resort in a matrix like this as well as well as the Z superscript 2 and a superscript 2 in their own
matrices.
We've not shown those matrices here.
We can then simply take these vector rise matrices and compute a superscript 2 which is the same as
Y huts as we've shown over here in this snippet named vector right.
In summary this is what a complete gradient descent function looks like.
We complete all z values and then 0 8 values.
Then we find DC D W and DP.
Once we have found these derivatives we go ahead to update to a weight and biases over here it is in
a for loop because we want to perform multiple iterations of gradient descent decimals in a thousand
iterations here.
As an example.
Now let's summarize.
Order with land about 4 propagation and back propagation so far in the next lesson.