Hello.
Welcome back.
In this lesson we are going to see how to perform the complete forward propagation of a work case study.
This one over here.
We're going to do the forward prop show meaning we are going to take our input and then compute it interacted
with a way to get the output pass it through the activation function until we get our y hat.
We're gonna do that here.
So I'm gonna make a copy of the last project
call this forward propagation.
Okay.
So I'm gonna open this
and this is where we left off.
So what we're going to do first is right our activation function.
Let's implement it.
We're gonna write a function to compute the sigmoid
and we said this what a sigmoid looks like if we pass F to the function.
This is the computation we perform to get the sigmoid.
We do 1 over 1 plus e to the poll minus sorry if we pass Z to the function if we proceed to the function
f and we want to find a sigmoid of the number z.
Then with the Form 1 over 1 plus e to the power minus C and we have to implement this in C code.
Let's do that.
Now I'm going to come over here and dysfunction.
I'm gonna call sigmoid.
Why do I get why do I keep starting from here.
Okay.
Sorry bother.
Okay.
This of type w it's returns a double gonna call it sigmoid and we're going to take Z or X. Let's call
it X over here s argument open and close and I'm simply going to come over here see double result equals
1 0 4 1 and then 1 plus we have the exponent function known as E XP minus the argument which is x over
here.
So this over here will get us the sigmoid weather will get us the sigmoid of a single value but we're
dealing with vectors and matrices.
So let's find a way to get a sigmoid of an entire vector.
So I'm gonna come over here and say void
vector sigmoid open close.
The first argument is a pointer to the input vector.
The second argument is a pointer to the output vector.
The last argument is the length of the vector.
So I still have the same length.
We can have just one length parameter instead to underscore t Ellie n open and close and I'll suggest
you post the video and try to implement this on your own.
Since we've already implemented a sigmoid function here right.
Once you are done we can do it together.
Start off by saying for int i equals zero.
And of course i is less than the length I plus plus then open and close and we come over here and see
the output vector index.
I E course sigmoid
input vector index I over here right.
So we can expose the vector sigmoid function we can leave this one here if you want to expose it you
can but we don't need this in our main file we need just this once.
I'm gonna bring this one over here like this
page over here at a semicolon here and now we can go to I mean the c file to perform our forward propagation
and I'm going to clean everything from the internals here I'm going to come over here just clean this
and we have a what data in our buffers and one thing I'm gonna do is send ups one we realize our sit
ups one doesn't need to be a matrix it's a vector cause we saw you had one row three columns so sit
ups one over here rather than it's been a matrix we can simply
we can change it.
Let me see should we change it.
Yeah we can change it to a vector.
No problem
right.
Right.
So actually I'm going to rename these bits over here clean this and I'm simply going to put it together
and say um row X and then vector rise X. We're gonna put the two data types x 1 and x 2 together just
like we have it in our image here.
We're going to we're going to keep all of this in a matrix so that we can compute it all at once.
Like it's often done in real world.
I'm gonna say double row x.
And notice I said row X not row X1 or anything clean this as well.
Row X the size of row X number of features which is the same number of input and number of examples.
And it's a two dimensional array or a matrix.
The first 2 5 1
them show you what I mean by 2 5 1 2 5 1.
So the first rule belongs to the first feature type.
The second rule belongs to the second feature type or input type.
So 2 5 1 here belongs to hours of work out and then
just arrange it in a matrix form.
The second rule here 8 5 8 this is hours of rest right.
And I'm going to have to have another double type here.
Double raw why.
And this just for uniform is uniformity sake I'm going to make it a matrix and it's got one rule so
of past the constant one day and this is number of examples
and we know the y values.
Two hundred ninety and then one hundred and ninety.
Right.
So I think I should put a comment here for you just to allow you to picture what we have
she.
I'm going to say.
Train X
Crane X is this date um it s two five one does the training data for x
8 5 8 and then the dimensions do it cause an X meaning number of features by m mean in number of training
examples and X year number of features how many features do we have we have two features hours of work
out hours of rest by M number of training examples how many training examples do we have we have three
of them.
Column 1 Column 2 Column 3 and we can see this in our architecture here.
Column 1 Column two column three.
So in this row actually in this table what I've written in the code is a transpose of this table here
in what I have in the code the rows have become columns and columns have become rows essentially right.
So I'm gonna put a similar comment here just to give a clearer picture.
Cos my goal in these initial sections is for you to understand if you do not understand and it s works
then my goal hasn't been accomplished so I'm going to heavily comment this and you know sort of drill
down every little thing you might get frustrated you know but you can always skip like I say or have
a go at me in the review like some of you like to do.
Um so this is it.
And the dimension
is time of course one by number of training examples one row by number of training examples and we can
see that over here right you can see the like I said you can and this is a column the lead column has
become a row essentially and this is because I've arranged it like this.
And once you understand what goes on in the background it really doesn't matter what you make room where
you make a column.
Especially since we are starting from scratch like this.
But once you start dealing with libraries this a particular format in which data is arranged and you
have to know that you have to take into account as you work with him later on we shall see how to work
with a carer's and tensor flu and other Python libraries after we've seen a lot of oh after we've seen
some of our C code course in the room where would you be sort of right in the code in Python and then
training with Python and then run in in France only on your microcontroller so you would have to know
a bit of Python 2 but already have lessons on Python for those of you who are not convenient or who
have never studied Python you can go to the last sections of this course there is a section known as
Python primer and you can learn everything you need to know about python there cos I've got other Python
courses on digital image processing and DSP I've just taken the python primer lessons to improve them
here so that you can get acquainted with Python when we start using tensile flew and carers and you
know you don't already know Python you can get a start right.
So have having done this we can go on we said we have row y row X so we're gonna create a buffer to
store a while to store the normalized version is right.
So I'm gonna come over here and see a double Crane X and the size of this is just the size of row y
row X so copy this pasted over here and I'm gonna come over here double Crane y the size of this is
the same as the size of row y.
Okay.
Right.
So I'm going to create some buffers to I'm gonna have see one.
Let's see what we have.
So I'm going to create buffers to hold ze one to hold C A Z and why.
Remember we have ze superscript 2 over here a superscript to C superscript 3 etc. We're going to create
buffers to hold this this slide over here shows the entire forward propagation process this is what
we're going to implement in this lesson.
So having minimized that oh come over here and see
double
Okay let's see.
Let's say we want to start training example by example we want to process training example 1 so I'm
saying train X e.g. 1 and this is equal to number of features
and then
we do a train why each one and then you do a double C one for training example one we take an early
training example one just to make it understandable.
And then ze one of course.
No.
He didn't notice.
That's the size double A1.
The screen should be the same size I see one.
This also of course number of hidden notes.
And then can I see a double
you can have double over here.
See two
of training.
Example one.
And then I see a double y hut which is the same as a two
offering an example one like this right.
So we start off by normalizing our data.
I'm gonna come over here and see normalized data and I'm going to pass a number of features by number
of examples.
And I'm going to start to normalize data the number of features numeric samples and the input the input
matrix is raw x.
The output is train X like this.
We have a typo
okay.
So the reason we have a typo is our normalized function takes two vectors and we're trying to normalize
a matrix.
So we've got to create a new normalize.
Um yeah you've got to create a new normalize.
So I'm gonna come over here and create a new function here.
Call this normalized 2D and I'm gonna create after that yeah after it I'm gonna create another function
for 1 D and 2 this stuff just because of this show that we would have to we have the option.
Anyway I'm gonna come away and quickly do void
normalize on the score data on the score 2D and the first document is going to be the number of rows
and we can make number of columns second.
So come over here you end 32 on the score t row
and then you end 32 on a score t column and then
see double I can simply call this input array
0 input matrix
input matrix I bother and it's rule by column
the next documents can be the output matrix
and this also is rule by column size.
And yeah we can implement it.
I think this should be fine.
And like we did earlier we start off by going through the matrix to find the maximum number.
So this time is not just a one a one D or effect or two matrix.
So we need a nested loop gonna come over here.
I see a double Can I use a different method here.
Double Max because I could not make it an extremely small number.
Okay so let's say that Max and I'm gonna come see a four and I equals zero.
I s less than rows I plus plus pin close and for int j equals zero g s less than columns
J plus plus
we've been close.
And Mark's gonna say if
input matrix high J is created on max then Mark cause the input matrix I and J.
Like this once we've done this we have once we found the marks the maximum number in the matrix
we can come out here and normalize it we say for int i equals zero.
I use less than row I plus plus open and close and this and this that loop us well for int j equals
zero.
J s less than call J plus plus open and close as well.
And then output matrix index I index J E course input matrix I j.
Divided by Max it should normalize for us.
Right.
And what I'm gonna do is let's see our initialization function.
So this weight initialization function is to D.
I'm gonna write a one day version of weight initialization function she gonna come over here and quickly
do weight initialization totally void
where each initialization
one D.
Right.
So we simply going to take input vector output vector or we simply need output vector here we initialize.
So we need no input we just need a place to store the random numbers output for actual and the length
the length of the vector
you enter 30 to 40 alien
you open and close like this.
Right.
And we can run do a here See that's.
And then for each.
J equals zero.
J is less than the length J.
Plus plus open and close and we simply need a local variable here called Double de Ronde.
I'm gonna come down here and see
better ways.
The youngest core runt.
So a D on the score round.
E course
round function modulo 10 and then D and the score run the course divided by 10.
You'd need to generate a random number between between 0 and 10 and see article vector index J equals
the underscore round and our function is complete.
So I'm gonna take the prototypes of these functions to the interface file over here
open and close like this.
And the last one where each initialization control C and come over here.
Paste to here semicolon here.
Now we can go to our main function.
Now let's see now we can simply see normalized data.
We can use the 20 version by simply putting on the score to d over here right.
Of course and the cool on here and I'm going to normalize this as well.
Normalize data score to D and this one by number of examples and then to input this raw why the output
is training y right.
Good.
So once that's done let's fetch training example one from a Y and training set.
I'm gonna simply say training example 1
train training example 1 index 0.
Course we fetched from Crane x
0 0 and then an example 1
x 1
u course create next 1 0
with essentially selected 2 and 8.
So between an example one training example 2 is 5 and 5 train an example 3 is 1 in 8 so that's how come
with navigated this way right I'm gonna fetch the y value as well Crane why each one this a single value
course train example why train why next zero index zero
Okay so once this is done
we can check if we have the right data and can print.
OK.
What.
Let's just quickly print print
create an underscore x.
Each one is
percentage of percentage F
and
one to a screen
score x.
Next zero.
And then green X that you want your next one right.
Print off
you can put another new line here
and we can do the same for y value J print f
print y.
Each one is simply percentage.
This is a trend y.
If you want follow you.
Right.
So I'm going to click here to build and then turn it onto my board and I'm gonna look into a term
to research my board and create an example one.
Is this because it's a normalized value.
This is the x value to raw terms font size has gone back to its default state so that
so the Australian example 1.
And if you normalize these values here
right.
If you divide to show a normalization basically takes the maximum value and then you device each element
by the maximum value.
So if we take two and we divide it by eight we get your point two five.
If we take eight and divide it by eight we get one.
How well trained in x.
How about a y value to y value the y value for our training example one is the highest in the vector
which is two hundred.
So if you take two hundred and divided by two hundred you get one.
Right.
So we're on the right track.
Okay.
So we have the right data.
Now let's initialize and up 0 and snaps 1 wait
for a comment here
I'm going to upload a code afterwards.
That's why I'm commenting it.
Okay.
This proves we have the right data and then we are going to see over here
finish initialized in OPs zero and Snopes 1 quite
I'm gonna say wait.
Random initialization and then this is number of
number of hidden nodes by number of features and we want to store it in the synopsis of Matrix buffer
right
and we can print this out to verify.
Oh leave that up to you.
I'm going to initialize it snaps 1
comment already exist.
Sorry about the just copy and paste this and I'm gonna use our function Initialize one d and initialized
1 D over here
simply takes the article vector which is synopsis 1 and then the length which is number of output nodes
there's the function we can verify it takes output vector in the length and then it generates the random
numbers into that.
Why do we have this way to random initialization on a score one to see
um let's see the function name is different
I'm gonna add random over here for uniformity which random initialization one day I'm gonna fix this
in the c file as well and no this is fixed.
Right.
So we can print this out as well.
So I encourage you to print this out to make sure you farm sort of initialize so accurately and print
this out.
I'm not going to do the want to go ahead and compute the C1 value
and we can take a look at our formula for computer ze 1
we said to ze 1 is simply multiplying the input x by the weight.
Um yeah.
What I see one or Z to the z value is to multiply input by the weight.
So I'm going to do the over here we said as the two.
No problem
I'll see.
I'm gonna use the multiple input multiple output neural network function that we have multiple input
multiple output.
Then the first argument is the input vector which is straight Example 1.
The reason we've taken does a single training example is because we've not written the function to deal
with the entire matrix and you can continue working on that and then the next argument or say a number
of features
and then we store it over here.
This the output vector we call it c 1 e.g. 1.
And then number of it in notes
then the weight stored in synopsis here right.
So once we've done this we can um we can compute a 1 which is the activation
and we simply use our vector or sigmoid function and then
there's the source we want to store the output in a one each one vector the length is number of.
He didn't note
once we've done this we can go on to use the output stored here to compute the next C value.
So come over here see compute
compute C 2 and then we do that by C as each to each one.
This course we use the same function multiple input multiple output neural network multiple input single
output in your network.
This time
and the input is the one each one vector the output should be stored in sign ups 1 vector and Atlantis
number of hidden nodes
commit a typo here and once this is done we can go ahead and compute the Y heart which is the same as
a 2
and that is simply calling our sigmoid function the single sigmoid function.
So this might affect our sigmoid so we have to expose this function we wrote that we thought we might
not need to expose where is that where is it.
Um this one here this is sigmoid of a single value put it over here
right
so to compute a well why heart we simply say y heart each one equals sigmoid.
And then we pause C to each you want over here.
Like this.
Right.
So we are done we can print out the result.
I'm gonna come over here see print f
y hard each one and send percentage Jeff over here.
Right.
And I say let's print the spring term see what US will print f
this is going to print all of them c to the G one
scooch new lines
and then we can print.
Yeah I would advise you print the rest out.
I mean I cannot spend the rest of the time this right in print statement.
So yes um write a number of for loops print them out LSC and you can go through the computation you
can in fact use pen and paper and go through it.
If you find an anomaly you can send me a message but I'm confident it should be perfect.
And when I reset I'm gonna clear First
I'm gonna reset my board
so that's what we have.
We still print in the training example the Y had value.
We came up with this one to a train an example of this.
And um let me comment about the training example
comment this out dude
download onto the port
go to terror to Claire we said to my boat.
So we had ze one of course this and this the Y how to value we got right so um this brings us to the
end of the lesson but like I said um I would advise that you print each level of computation over here
and go through it make sure you understand it and it corresponds with what we said in the theoretical
class.
If you find any anomaly or if there is something you do not understand then just let me know.
And also I should point out this is for just training example one mean and this is four to eight two
hundred.
Right.
So we're not going to go into back propagation until we've learned other things with regards to Chi
Carlos and other things about gradient descent.
We have to pivot upon our knowledge before we can continue.
But ideally after forward propagation we um we compute the difference between a y hut and a y value
and then we land from there and then go on to perform third up you know weight adjustments like we mentioned
earlier learning is simply reducing weight reducing weight or reducing loss as it is often said.
So that's the end of this particular lesson if you have any questions just let me know and I'll see
you later.
Have a nice day.