Hello.
Welcome back.
In this lesson we are going to write a new function known as the logistic regression model and this
function is going to encapsulate the other functions we've written.
We're going to pass our training image our test image the training and test labels as well as things
like a lending rate a number of iterations and this function is going to train the entire model and
then uh give us the m the the the resort for both the train inaccuracy and the test accuracy.
So we're gonna construct such a function in this lesson.
I'm gonna save this.
I call this number seven and then I'll come down here to write a new function def or call this
log write model in the first document is going to be the X train the next document is gonna be the Y
train training labels and then the test data x test then the labels for the test y test and then number
of iterations
we can set a default at 1000 all those that you I want something else and then lending rate
set a default to zero point four.
Yeah I think we have enough.
So we have this new function log rack Model X train y train okay.
So yeah.
Kristin let's see what we have from this end.
Okay
sorry go the.
Okay I'm going to create some space here.
I don't like this
Okay so we start off by initializing a war weight and bias.
So we call our weight and by his function our initialize function.
So when I say WB the return is going to be stored in W and B and I'll call the function which is initialize
zeros
let's see over here
and then what I want to do is say X strain does the dimension extra in does shape index zero.
Right.
Once that is done we're gonna perform the gradient descent by calling we'll learn our land function
and we're going to store the remember our land function returns parameters gradient and cost.
So I'm gonna come over here and see parameters s s s return value 1 or return variable 1 parameters.
We're going to have gradient gonna have costs and the name of the function is learn and then we're going
to pass the weight to bias as argument the training set the training label.
Let's see what's left wage bias training set the training label a number of iterations Len Lenin rate.
Okay.
Number of iterations lending rate like this.
Okay so after this we can we can retrieve the parameters W M B from the dictionary from the remember.
Um this is Paramus is a dictionary we're going to have to be and be stored in it.
Right.
I'm gonna come down here because we need and for further computation.
So I'll see w e course firearms all over here we stored it in a new word court parameters I'll copy
this then I'll pass stop you in here and then B equals parameters I'll pass be here right.
Once that is done we can call our predict function and then I'm gonna see a y y prediction
y prediction for test set equals predict and then this takes the view B and then the data is X test
the test set we're gonna do y prediction for train and then this we see it predict of course stop you.
B X and then crane and then once we've done this we can go and print the resort
we can print the um the test accuracy and the training accuracy during the process.
So I'm gonna come down here.
I simply say print and then I'll say test accuracy
and I'm gonna do this or do this I'm gonna format it's a bit we're going to print the mean so I'll come
here and say that's for months and then one hundred minus and Peter to me and then and Peter it's absolute
value.
We're going to basically subtract the m the y hat which is the predicted value from the expected value
which is y and then yeah and then we multiply by one hundred and all of that subtracted from one hundred.
I'm gonna say y prediction
y prediction underscore
test minus Y test and then I'm gonna come out here and then multiply this by one hundred
mission.
How many brackets do I have.
I have three of them.
Yeah.
Right.
So we're going to do the same for the M.
Let's see what we have.
Yeah we're gonna do the same for the test.
Uh the training accuracy.
This was for test accuracy.
It would have to check for typos.
We're not tested for a while.
Okay so this is train
and this is going to be this word here.
It's going to be train and this as well.
Right after we've done this we can return some information about our model such as the M that cost the
Y prediction the D.
Um yeah the train and test we can basically take some of the the results we get from this computation
start in a dictionary to return so that we can have like the M what's the word we can have we can't
have information about our model essentially.
I don't want to say to them the parameters obviously we're going to sort of pretend the parameters as
well but we can return things like the lending rate the number of iterations that y prediction test
the Y training test et cetera.
So when I have a dictionary here when I say d equals and then I'm going to return cost I'm gonna give
it a label cost and this will return costs then I'm gonna say why predictions.
We can analyze this later this is yeah why prediction test.
And this of course is gonna be why prediction test
and then we're going to have Y prediction Crane this is going to be my prediction train.
We're going to have w.
This would be w as well.
Great to have B this would be B as well.
Going to have a board lending rate
this is going to be the lending rate.
Then we're going to have a number of iterations
it's gonna be a number of iterations.
Right.
Okay.
So once we've done this we can return our dictionary
rich and D.
Like this.
Okay so this is going to be our motto.
This is going to create a model and B.
And once it's done is gonna predict and then print out the test and create accuracies for us and we
would have the characteristics of the model as well as its parameters stored in this dictionary court
D which is going to be written for us so we can test it out by training our model um to try to model
I'm going to come down here too.
Yeah.
I'm gonna come down here.
So this is our pre processing code.
Oh put comments here
pre processing.
And then decide here
training.
Okay.
So I'm gonna say a d e course and then I'm simply going to call a function log rig model.
I suspect we may have some typos because we've not tested for a while so kindly bear with me.
And did um or parameters.
The first one is train set.
We're going to take train set X
so before we can pass our data to our model we have to normalize it first.
Remember we didn't perform normalization yet so to normalize we take the largest value in this set and
then we divide each element by that value.
So since it's an image the the largest value it's an 8 bit image.
So the largest value is gonna be two hundred and fifty five.
So we're going to divide everything by that.
So I'm gonna create a new variable here or put a comment first.
Normalize
normalized data and I'm gonna create a verbal see a train set.
X equals.
I'm gonna take this this entire set and then I'm gonna divide by two forty five and then I'll see test
set I'll do the same for test sets.
We need not normalize the want D binary one in zero to tell us whether it's an E It's a cat or not a
cat.
We need not normalize the y values.
Okay.
So I see test set to set X and this equals test set to flatten divided by 245
okay.
So now over here in our functional model function we going to pass the um the normalized input.
Yeah.
So I'm gonna come here train set x.
Copy this paste it Crane said why we have train set y somewhere.
Copy this pasted over here to set x test.
Copy this paste the Y here.
Test set y.
Paste this over here.
And then a number of iterations
I'm gonna set this to two thousand iterations Len and rate
when I set it to zero point zero zero five.
Yeah.
So this is it.
So let's inspect for typos before we build.
Okay so um over here the first mission I saw is that we are using D.W. here but we've not extracted
it from the dictionary.
We have to remember it t w NDP were returned from the propagation function but they are kept in a dictionary.
We need to pull them out before we can apply them here.
So to do that we've got to sort of pull them out
of CDW because the name of the dictionary is GRUNTS AND THEN WHAT I WANT IS D W and then I'm going to
see a DP across the same dictionary Gratz
DP like this.
So now we've extracted them also in our predictive function here we would have to reshape the w the
wait before we can compute the activation.
So I'll come over here and see W equals w dot to reshape
then I'm gonna say X top shape we're going to do index zero
by one over here like this.
Okay.
Um lost track once more before we um would build.
Um yeah.
This function for instance.
Yeah.
The return here should not be in the for loop.
So we've got to come out of the loop and then let's see.
Okay.
Looks good mm mm mm mm mm mm mm mm mm mm.
This 0 function
right.
The error has to do with this the fourth argument in our function.
Let's see the fourth argument is the whitest but.
But over here I'm still send in train set to this word here should be test.
So we have the crane and X the train and Y.
The test x test y number of iterations we've set it to two thousand and then a lending rate of zero
point zero five three pointers or five.
Okay let's run and see if this works I'm gonna save right.
It's working so.
Okay.
The error reduce and we've run this number of iterations so far.
Right.
And it s completes our test accuracy it says over here
our test accuracy.
Let me.
Let me reshape this a bit for you
our test accuracy is 70 percent and in the train accuracy is 99.
So it is a bit of over fitting going on here.
The um the difference between the two is too large but no problem.
We're going to test with our own image and also we go into um later on we're going to develop a better
neural network.
Now we're using logistic regression is not the best way for doing this type of task later on we're going
to see uh we went to first a four star have to lay a neural network and then we're going to increase
with a deep neural network with multiple layers and then after that we're going to try using a convolution
or neural network to see the results that we get.
So this is the performance of our neural network.
Right now we can increase the arm.
We can learn to lend for a longer time.
I'm going to increase number of iterations to three thousand and let's see what we get an improvement
in the test accuracy.
It's trained in
okay it s it's become worse a problem.
We're going to plot out plot out the graph and see the point at which it started moving from better
to was.
So now it's become 60 80 percent.
Okay so I'm gonna go back to 2k right okay.
Let's plot the um the cost us training and goes on let's see the graph.
Let's see the performance um with this we can try to know at which point our neural network begun to
um like the the part where we saw it move from 70 percent with two thousand iterations to 68 percent
with two thousand words three thousand iterations we can see the graph and see that this number of iteration
and gave us this amount of error and because of that we can decide on how much iteration we should apply
to the model to give it the best that we can.
Um if you don't understand all that I just said let's try it out.
I'm gonna see costs cause I'm gonna squeeze this
and then what I'm looking for is the dictionary we take in the cost from this and then we're going to
plot this I'm gonna say P L T dot plot I want to plot cost I'm gonna add a bit of label to it I'm going
to add a bit of labor to it Oh CPO t that's why label and the Y label is simply going to be cost and
the X label is the iterations TLT dot X label
and this is
iterations
and this is um per 50 50 yeah.
So we said 50 we should print out the iterations we should collect it in 50 years let's see.
We'll see about the
we have to add to the piece of code to record the iteration every fiftieth iteration.
I'm gonna come over here and I'll see if I modulo 50 equals zero
then I want to record I seen costs cost this our list there and we're going to upend the cost.
And this is what we're going to be plotting.
Okay so iterations per 50.
Right.
So once that is done we can simply do peyote dot plot and then plotted or sure we can add a title to
it if we want.
Come over here CPO T dot title
we can order lending rate
now.
We can simply fudge this from our dictionary or dictionary house lending rate.
Yeah.
We have lending rate the C.
Yeah.
This is lending rate so we can pull it from our dictionary so I can just see the string.
What I want is a dictionary
and in rate once that is done we can do POTUS show.
Right.
So let's see.
So this will plot the um the iteration is this who plot the cost pie the cost by iteration graph the
y axis is going to be the cost and the um the x axis is going to be the iteration so we should see the
graph for actually as you see it starts from the top and begin to reduce as the US the model improves.
Let's see what it looks like
when I click here to run
okay to learn in
yeah says Well multiple let us know module called P L T.
Yeah I suspect I know why this is
yeah.
Cos there's a typo here.
I've got oh over here.
Okay let's go
Okay so that's where we go or what.
Okay here's what we got.
And then um I can just show this.
So with essentially plotted this but every fifth yet iteration and I can see we started from the top
here.
This this is the cost from the top at iteration equals zero iteration of course to 250.
This is this the cost that we had.
And then as we continued continued reducing iteration of cost two thousand.
We reduced to this.
And this the best we could get for two thousand iterations.
How about we see the um the three thousand iterations we tried earlier we showed the graph should tell
us where things began to go wrong.
I've changed a number of iterations to three thousand.
Let's see what that looks like
okay.
This is sixty eight percent.
Mm hmm.
Let's try to zoom in and see if there's something we can see.
Okay.
It looks almost like the other one.
Okay.
I'm gonna close this.
I'm gonna try five thousand iterations.
See
this what we have.
What is.
Yeah.
Okay.
5000 iteration gave us 70 percent again.
Right.
Okay.
Enough experimenting with this so you can experiment with this you can change the number of iterations
that lending rate and then see what you can do better than this.
But given the little amount of data and given the type of neural network we using it.
It's a quasi neural network I should say using logistic regression here is not a best tool to use but
the performance is fine it's acceptable here.
Okay so we can go on and on and test.
We can test with which our own image if we want to.