Hello.
Welcome back.
Before we continue with our cut problem I would like to use this opportunity to introduce a visualization
to known as computation graphs.
Let's say we have a function J of ABC equals three into bracket a plus b c as you can see over here.
Let's say a equals five B equals three and C equals two.
When we insert this into our function the result is that a three like we can see over here.
Now let's break the function down into smaller functions.
We are going to represent the B times C parts of the function with a variable U and then create a new
variable v such that fee equals a plus U.
If we substitute these new variables into our function J becomes equal to 3 v we can represent this
in a computational graph by creating three nodes one for you one for the and one for J.
As you can see over here now then multiply by C gives this variable U which is equal to 6 as we see
over here and variable U Plus variable a gives us variable v V equals a plus U which gives us eleven.
Just like we have in this sub equation over here right.
This the second note and now if we take V and we compute j such that j equals 3 multiplied by V we get
thirty three.
So we've essentially represented our equation in this visualization to known as computation graph over
here we start with you here.
U equals B.C. the result of U is taken to give us V because V equals a new variable a plus U.
And that's eleven and a result of V is taken to give us J because K equals 3 v.
Simple as that.
Now let's see how to compute the derivatives using the same visualization method.
Let's start by computing the derivative of J.
With respect to V.
This is written us D.J. over T V.
This simply means that if we change the value of the.
How much will we change the value of G.
Essentially.
And the answer is 3 we know G is always three times the value of the.
So whatever increment we make to V that increment is going to be multiplied by 3 to get the corresponding
G increment as we can see in this example when we increment V by zero point Treasury a 1 j is incremented
by zero point zero zero three.
By computing the derivative of V we have essentially done one step of back propagation back propagation
involves computing the derivative of the final output variable with respect to the intermediary variables.
Now let's try to compute the derivative of the final variable g with respect to the variable a.
In other words let's see how much the final variable j will increase if we increase the variable a A
is 5.
If we increase a base your point is usually a 1 A becomes 5.0 0 1 V equals a plus u u is 6.
So the new V value becomes six plus five point zero zero one.
This will give us the equals eleven point 0 0 1.
J is three times V and three times eleven point zero zero one is equal to thirty three point zero true
three as we can see we increased a by zero point zero zero one and J.
Got to increase by a point or two or three.
This shows that the derivative of J with respect to e is 3 but semantically what we essentially did
is find the derivative with risk the derivative of V with respect to a and then multiply the result
by the derivative of J with respect to V.
This is because in order to find how much J will increase if we increase a we have to find how much
V will increase if we increase E and then find out how much J will increase if we increase V.
This is because in our computation graph it connects to V and V connects to J.
This also proves the chain rule of derivatives.
We mentioned in the previous lesson detail over the course D J over the DV times DV over d a.
Now let's see how to solve DG over dp 1 and when we increase the value of the variable b how much will
the final variable increase.
To do this we have to first find do you have DP which means when we increase the variable b how much
will the variable you increase.
Remember your equals B multiplied by C we set a verbal c course in the equation you is effectively U
equals to b if not replace C here by its value to then indeed your becomes equal to two multiplied by
B.
This means that whatever increase we make to the variable b it's going to be more supplied by two such
that when we increase the current value of B which is three to three point zero you are one you will
becomes six points to a short term because we will derive u by computing two multiplied by three points
you are short one so then that the U of IDB is equal to 2.
This means when we increase the variable B by a particular amount the variable you is going to increase
by twice that amount.
Now that we know the derivative of you with respect to B is 2.
We simply need to find a derivative of J with respect to you.
But wait a minute we have already computed that we have already computed how much the final variable
j will increase when we increase the variable you.
And the answer is 3 Um by multiplying deejay over to you by.
Do you of IDB we get DG of our DP.
As you can see over here right.
We apply the same method to derive D.J. over D.C..
Let's see how we can apply what we've learned about computation graphs to our logistic regression example.
We said we could see by the transpose x plus P We then passed through our activation function to derive
y hut or a we also defined our lost function to be this equation.
We can represent this in a computation graph like this.
We take W X and B to compute C like this and then we take Z to compute y heart like this and then Y
huts.
I should mention y heart is the same as it over here.
After completing a we apply our lost function.
Remember it is the same as Y hearts.
Like I just said which is a what predicted value.
Y is the expected value.
Right.
So now our aim is to compute the derivatives with respect to the loss.
Now our aim is to compute the derivatives with respect to the loss like I mentioned earlier.
We start by computing the derivative of the loss with respect to the variable a.
When we take the loss equation up here and we compute the derivative this is the answer we get when
we compute the derivative of the loss with respect to Z.
We get this other answer a minus Y and this is the proof of the answer.
You can go through it if you're interested one will compare to fit.
We find that the derivative with respect to w 1 which we have named w 1 is equal to X1 DL.
Similarly D derivative with respect to W2 which we've named D W2 is equal to X2 dizzy.
This essentially gives us how much we need to change w values in order to minimize the loss.
And finally Debbie is equal to DC like this.
This is all there is to this lesson and I shall see you in the next lesson.