High in this session, we will discuss about neural networks, so let us, first of all, see what is
neural network now, neural network is a technique for building a computer program that learns from
the data.
Well, this is exactly what machine learning is also doing.
So what is different here in this?
It is based on very loosely on how we think the human brain works.
So just how the human brain works in the same way neural networks also work.
In our brain, there are several neurons which are connected with each other and the first message from
one to another, which are decrypted by these neurons, and then we are to think or feel something.
Now, how is it related to neural networks?
That is in neural networks, a collection of software neurons are created and connected together, which
allow them to actually send a message to each other.
Now the network is asked to solve this particular problem.
So we will provide a particular problem to the network and the network will try to solve the problem
and it will try to solve it again and again, again and again.
Then that time it actually will affect the learning process, feel that it is actually able to perform
that particular calculation.
It will keep on doing it again and again.
So every vibration when it is doing this or learning or sending the message again and again, it actually
keeps connecting the dots and actually keeps learning from the data and improves the predictions which
it is making so that it actually needs to success and diminishes the particular patterns which it has
learned to reduce failure cases.
So this is what neural networks are doing.
So how are we actually creating this and what does this actually mean?
So let us have a concentration on this particular image here.
So here we have a lot of circles connected.
Now, these are the inputs, not the ones in the Greens, the blue nodes or other connections which
we know connected to and let us see the red is the decision point.
So here all we want to do is we want to send our input values that is different X values or the features
or attributes from this particular input nodes.
And these green neurons will pass on the information to the blue note.
Now these blue laws will actually try to decrypt these patterns between these green notes and say now
they're listening to the red node, which will finally decide what the output should be, if the value
should be.
Let us see if we are looking out for continuous value, then what value should be predicted, what price
should we predicted?
And in case we are looking for a class, then if it should be yes or no.
So these kind of decisions can be made using this kind of a structure.
Now, it might look tricky on how this will actually be of the information, but this is a very simplistic
neural network example.
Here we have only one little prison and there is another neural network, which is even more simpler
than this, which we are looking at right now.
What is that neural net?
Well, you remember logistic regression.
Our very own logistic regression is the very smallest example of a neural network.
How that is possible, we will have a look at it in a while, but let us try to connect the dots and
create a picture.
So if we look at this particular image, you can see there are several input values.
And these input values will be giving a certain probability to us.
And how are we getting this probability this is a simple logistic regression in logistic regression.
What do we do in logistic regression?
We have several input values and we apply a sigmoid function on top of it that we can avoid this particular
of the input values into our equation.
What will the equation be, the equation will be the one next one, plus we do X to plus B that three
three plus B for export and so on.
Right then we apply the sigmoid function, Lonard or the logit function on it.
What is the logistic function?
The logistic function is one upon one plus E to the power minus the equation which we just formally.
That is logit function, so that is how this.
Swan also will result in the probability to us this is how the logistic regression look.
Now let us have a look at it for the.
Now, the neural network is an algorithm inspired by the neurons in our brain.
It is designed to recognize patterns in complex data and often performs the best when recognizing patterns
in audio images and videos.
Now, neural networks are not something which we will go towards at the very beginning or when we are
striving to solve a particular problem.
Whenever we are starting to solve a particular problem in that situation, we will first of all implement
the Lenie models, then we will implement the decision tree and after that we will decide if we want
to go towards bigging or boosting.
In case we have less amount of data, then these algorithms, which we have learned earlier, like SBM,
Ganon, Nijhuis, a random forest she used, all these algorithms will be more favorable.
But in case they are trying to have more precision or a better model, then in that case we will be
going towards neural networks.
But why are going towards neural networks?
The main challenge we face is the lack of data.
Because neural networks required a lot of data to brain, that is the reason why we cannot implement
neural networks for all types of problems, because we need a lot of data to get a good output from
the neural networks.
Hence, it is used immediately for problems like audio or video data where we can have exhaustive amount
of examples and then we will apply neural networks on them.
Now, neural networks simply consist of neurons, which are also called as nodes, so here these small
blocks, these small soakers, these are called nodes.
Each neuron holds a number and each connection holds big.
So each and every node will hold some information in it.
It will hold some information in there and each connecting line will actually hold a vote on.
So what does this week?
What is the weight, these weights are nothing but the be the values which we have been talking about
in logistic regression.
So in case of logistic regression, we had better values with which we multiply these input values.
Right.
The V created the equation using using the beta values.
So be not.
Plus one x one plus VW x2 plus bigotry to extreme plus veto voting to X four plus veto firing the ex-wife.
So here's the different veto values that we have are the rates only.
So in case of neural networks, we will be calling them.
So this is how it will look like each node will hold certain.
Value on it, certainly dynamic, and each connecting line will actually hold some great value.
Then they have activation functions which are usually non-linear, transforming functions which transform
linear functions into a nonlinear form which is capable of capturing complex patterns now.
Here we have the input data points that does X one, two, three and so on, here we have different
values which are similar to the values which we had in the logistic regression.
Then we have the net input function, which will basically, although some all of these things.
So at this point, what will we have at this point?
The equation will be w more or less because of the assumption W one X one plus W2 x2 plus W three extra
W maksym.
Right.
This is what we will have.
Then we have this activation function, which is nothing but.
The logistic function or the logic, logic function or the sigmoid function, which we have been discussing
about the equation, which was one upon one minus E to the power.
B, where the B is the combination of this particular equation, a combination of these, so that this
activation function now it's strictly says that the activation functions are usually nonlinear, transforming
functions.
So that means that this activation function could be any other function except other than the logic
function also.
But how will we decide what the value of the activation function should be?
So consider this first thing is if we are applying this logistic function or the logic function or the
sigmoid function, which we call it.
So in that case, what does logistic regression give us?
Because we are seeing this neural network, the very basic neural network, resembles the logistic regression.
So and here we can clearly see that the equation which we are forming is similar to the logistic regression.
So what does the output of the logistic regression, the output of the logistic regression is a probability.
Which is usually ranging from zero to one.
This means that in case we want to get probability as an output or we want to solve a classification
problem, in that case, we can use this logic function based on the sigmoid function.
But in case we want to find out the continuous value, then we should not use this function because
this will convert the range of the data.
You remember during discussion of the linear and logistic regression, we said that the equation of
the line, the equation is converted to logistic equation into a global equation because of this activation
function.
Only if you want, you can go back to the theory of logistic regression, have a look at it and come
back and see that.
What I'm saying is exactly Guddi, that we have converted this linear equation into a nonlinear equation.
That is something which is having a range from zero to one using this sigmoid function.
Hence, in case we want to find out of continuous volume, that we want to make a prediction on a continuous
value, that is, if we want to solve the immigration problem, then we don't need to have this sigmoid
function at the end of the neural network.
So this is one guideline that is if we are trying to solve an aggression problem, then by the end of
the neural network, no matter how many layers we have here, no matter what we have on the left in
sight, we will not have a sigmoid function here because the sigmoid function will change the output
from a digression problem to a classification problem, that it will change the function from a linear
function, the logistic function.
So we don't want to do that.
That is why we will.
Remove this, but in case you want to have a class overclassification problem, then we will keep this
logic function.
Another thing to keep in mind, this, in case we are solving a problem where we are deciding if all
animal is a gag or a dog.
OK, this is the problem that we have, if a type of animal is a cat or dog, then how we will be solving
it.
So this one logic will be able to give us an answer at all if the animal is a cat or not, OK?
So for solving if all animal is a cat or dog, then we need to have two lords instead of one.
So in that case, what we will be having is in that case, we will have something like this.
So we will have another function here.
This is one node here and we have another node here.
And these will be connected to this one also.
So one would be connected to this and another would be.
Connected to this one also.
So these will also be connected to this one.
And this will again be a logic function.
So now what will happen?
This one will give answer if it is a cat or not a get.
And here, this one, Valenciennes, if something is up, doc.
Or not all.
OK, so this Wilhelmsen, if this is a or not, again, and this will answer.
This is a guy.
Well, not at all.
So in case the animal is a cat, then the output will be.
One here.
And something near to zero here, and if the animal is a dog, then the output will be zero here.
And one.
OK, so this is what we will be doing.
So always remember that seeing the logic function will give you one set of only X or not the X that
this guy or not look at, if you're trying to find out if there is another class present or not, then
it will have to be dog or not or dog, which will be a part of the class here.
Now, let us have a look at this, so how this logistic regression looks like so this was the equation
of logistic regression, physical to one upon one plus, one minus be done on plus with our next one,
plus that we do.
So this comes out of the blue note, plus the blue one makes one so the blue can be replaced with Vitara.
Rita has can be replaced with the W.
So here we have the blue note plus W one X1 plus WBAY x2.
So this is the equation which we have got from this.
The combination and this combination is when fed into this sigmoid function.
At this point of time.
This is the equation.
And after it is the dysfunction, the equation changes to sigmoid of.
W w an example of this is nothing but distinguished.
So this is actually one on one plus one minus W W one X one does W do it.
So this is how neural network is similar to logistic regression.
But how does it actually help us, because this is a very simple logistic regression and this will be
able to solve only a few problems and it will be solving only immediate problems.
So how can we actually use it for complex problems?
So for using it for complex problems, we will be adding several layers to this structure.
So now, instead of having just this kind of a simple structure, we will be having the framework for
those structures so we can have a more complex structure in comparison to this one.
So the activation function, which is being used here, which is the sigmoid function.
This could be replaced by several other activation functions like that is of this unique step function,
which is used in Perceptron, which looks like this.
Then there is a fine function.
It looks like this.
Then there is a linear function which looks like this office wisely function, which looks like this
logistic function, which looks like this hyperbolic engine which looks like this, then we have rectifier
below, which looks like this, and then we have rectifiers of plus which looks like this.
Now these.
Equations which we have, these equations can be used in different places.
So the most frequently used is the logistics.
And the second most recently used is the venue function, which is used very highly.
And all of these functions, we have a linear function which may or may not be used.
Why do we prefer using linear function is because.
Let's have a look.
Now let's us see.
I am the beauty of linear regression or logistic regression, and instead of using the sigmoid function,
if I apply a linear function, then the output of this will be linear equation.
So after this I will get something like a human plus WOMEX one plus the mutilates do so.
All the same equation will be that if I apply a linear function on the only difference of the population
with this.
Now again, if I might apply something to it and add something to it again, the equation which I will
be getting would be another linear equation.
So if I keep on applying linear equations on this on this collection of the detail on this function,
then the output which I will be getting will be a linear function only, which is not ideal model.
If you want it, you'll have a linear function, then there is nothing better than applying a linear
regression.
We could have simply applied regardless of English and I would have loved the way work now.
So we don't want to do that.
The thing is, we should always use a nonlinear, transforming function so that we can capture different
complex for the patterns.
If we will not have a complex function created, then it will not be able to capture complex patterns
present in the data.
It will only be able to capture linear functions in the data.
That is why we use the nonlinear functions.
So now let us have a look at this particular unilingual.
So in this particular neural network, we have two hidden lives.
Olian what?
Kihei did not have any hidden here.
We did not have any the levees.
We had the input and the output here.
So here we have the booklet and the booklet.
Apart from that, I have done this.
Now, notice that each point of the input layer is connected to both the points and then led one and
both the points in the hidden layer one up and with all the value of nodes and then layer.
Now, for the then, these will be connected with the output again, they will be connected with all
the less so.
So it will look something like this, so all of these will be connected to each of.
So after that, the output will be given.
Now, let's look further now to solve this, what will happen is, first of all, then we will be working
on this.
What will happen is that the input values have we have certain values and those values will be multiplied
with their corresponding weight and added together, I pushed into this sigmoid function.
Similarly, the same thing will be pushed in to the other side of.
So what comes inside this will be somewhat similar like here.
Initially, we could have seen this or we can have random beats also.
So based on the value of the weights, we will have different things coming in to these microchips than
from the sigmoid functions, which will be multiplied from all the lines of juggling connected with
each of these.
I will push into this another sigmoid.
So from this point, the values will go up.
Now let us have a look at this.
So this is the team we would like to be following for the what of the value of the loot?
Because in this entire time, this entire neural network, the more the input values we know of the
output values, we also know what function we will be applying.
But the only thing that we don't know is the value of the beads.
But in these lines.
So we need to find the values of the.
So how we can find out the values of the game is actually designed by Jean.
So let us have a look at this now, this looks like a new rolnick.
So you really would imagine that this is the first move which has value?
It is.
And then there is another input more.
Now, this function, this is one of the sigmoid functions, just imagine this, this is a sigmoid function
and in it we get the biopsy, which is equal to a cross.
Here we have video your which is equal to the last one.
Now, both of these are coming together at the end, which are given by this small, multiplied by the
lower.
So what will happen is that we will be able to calculate these values by we will be able to calculate
the value of C by month by multiplying, and so we can easily find the value of C then the value of
the envy's.
We can buy the plus one, then value E can be easily calculated by C and to be.
So this will be easy for us to get.
But the thing is.
We have to find out the values of the backlights, we want to find out the values of the backbone,
if we want to find out if the only know the value of in will be the only know the value of it.
And then we want to find out the value and then how can we do that?
What then?
We can have to avoid the bad word that so maybe away from it and B the words E that this word for word.
And when we are going from E the West and it is for the back for us.
So what happens in the back for us to find out the value of we can be competitive so that day will feed
with respect to even be equal to one.
Now that they feed, the respect you see will be equal then by then.
See, I'm very critical of it with respect to the will be equal.
Do then evade the.
Now, if we want to find out that initiative of E now, we still don't know the value of so to calculate
the value of the again, we will have to find out the value of the.
So how will we do that?
So they by identity is already there.
Now, before I know the value of Baldie, the info you can give that these people will then be Oborne
baby.
So the value of then E with respect to Dalbavie will be valued by the including the Vivendi.
Similarly, the value of E with respect to E will be.
They eat by the sea into the sea by baby.
Similarly, they eat by the sea in the Bering Sea by then, so it will have a combination of both and
B here.
Because she's a product of both, and so this is what Barillas, which means that if Y is equal to your
faith and ecology of you, so human body is equal to your faith that this is a function of C.
And so if this is a function of Sandy and I want to find out the value or the change in it, then I
can find it out with respect to the change in Sydney FC and they are functional.
And then the change in E can be found both in terms of.
Changing it with respect to C, with respect to a.
That is changing with respect to the with respect to the so this can be done in this particular way
that is dealt with by delegates is equal to the value in value by benitz.
So the intermediate, which is present that become come like this just the way it is something you.
So they need vitality in daily vitality is equal to the Leviathan, so here we have the intermediate
evidence.
They love the.
So this is what Jane Willis.
Now, how we actually use this, we will think of something.
So.
Legacy won back propagation.
So back propagation is calculating the gradient approaching, we always start from the familiar and
propagate backwards of the beats and biases for each year.
So we start from the end note and for the words the.
Input values.
And by going towards the input values, we obey the values of the gates which are in.
And based on the values of faith, we can actually find out the outward values.
So the main thing here is that what happens is that this is the next book, which we have.
And in this network, we have these input values, which we know of.
We know these sigmoid functions and we know about these output values.
But the only things which we don't know about are debates.
Now, in one forward buzz, we can find out dysfunction and find out these values, and then we find
out the.
Predicted output value.
So based on some random values, we can predict the output right now because these output values are
actually not exactly equal to these output values.
That is, the predicted values will be different from the actual value.
So we will have to find out.
We will have the gold in the back for addiction to obey the bits, because only of the weights can actually
improve the output values.
So what we do, we find out what is the change which needs to be done and we go back and based on the
change, we backtrack what the value of these weights and inputs should be and correspondingly, the
input of the value of the wheat.
Now, again, the former vice principal and again, the input values will be multiplied by the updated
values.
And again, the sigmoid function will be calculated.
And based on the updated sigmoid value, the output will be delivered.
Now, again, if the output value is not equal to the actual output value, it will find the error and
back propagation to these input values so that it can actually have these rates.
So this is what will keep on going.
It will all keep on going the forward and backward pass until it actually gets the output values, which
are very close to the actual value.
So this is what neural networks is all about.
It is all about going forward and backward, but until we are able to improve the way it's a high.
So we adjust the leads and biases throughout the network so that we get the desired output in the output.
Now.
This is what it looks like, so we have the input value.
I'm the input value is multiplied by some one.
Then indicate the while we have this activation function, so what goes inside this activation function
in this activation function input into one who's.
So after this one comes out, so we get activation, we get input value into the blue one applied activation
function, and then what value comes out of this?
We multiply, they'll be then again, activation function is applied on top of it.
So we get the union function and then we again apply multiply it with the WP and then again apply the
activation function and we get the answer from this one.
So what does a.
So now if we read this, this entire thing, what is the.
The output is nothing but activation function of light on WTT multiplied by the result of the Hadler
to.
What is the result of his desire is nothing but activation function applied on the menu to include dessert
of the Italian one.
And what is he, the one who delivered one result is nothing like the blue one multiplied with the input.
So you can see the again going forward, very similar to calculating the value and the back prediction
also for calculating the input, values and the values.
So in this particular deal, if we go in this particular process, so the output value will be activation
of light on W3 in the activation of light on W2, the activation of light on the blue one in.
You can read this for a while and actually see the entire thing, me, you can cause the video and have
a look at it right on the understand, because this is what which will be creating the concept.
Now, if people want to find out the change in the output value, what change we want to have in the
output value to improve our prediction, then how would that be calculated?
That will be calculated by then output by Delver, W1 that this change in output, one with respect
to change in W1.
How W1 will change actually impact the change in the output value.
So what this bill is, we will do this will be will do change in output with respect to the delayed
one.
The change in the lead, one with respect to the one in two, changing the earlier one with respect
to the blue one.
We're just changing our vote with respect to the little one.
And the change in the land won with respect to the land, one in the change in the land with respect
to the land won with respect to the blue one.
So it's just going this way.
This is the entire equation now.
We will try to understand this photo.
So.
For us, the output of activation function.
Is they give the the world because we will be subtracting this output with the actual value so that
they give us the added value.
Now we want to minimize this, everybody, so what we will be doing is we will be back tracking this
error value to W1 so that we can gradually update the value of the bits.
So how are we going to do that?
The change in error with respect to W1 is equal to.
The change in attitude with respect to the everyone will be equal to the change in Ed with respect to
the output.
In go change in output with respect to the net include the change in the labor with respect to.
Fewer didn't run into the change and he didn't run with respect to the new one, so this is what we
are doing.
This is.
James and Ed, with respect to the blue one in blue jeans and Ed, with respect to output into output,
with respect to to into change in hereinto with respect to one single change, including one with respect
to the W one.
So let us calculate the error values, we will calculate these.
So in this scenario, let's consider the this as one of the.
So for any length for the first year, this be the legacy, then this is the output layer or any layer
intermediately.
So in this year, there will be an activation function, the.
And there will be an output exergy.
For leered in.
And there will be an input value that is the sigmoid function, the product of every time, everything
as an object and what is in store and sees nothing but multiplication of X1 into W.M. plus X2 into the
video plus extra into the battery from different modes.
So in the case, we want to find out the change in.
And if you want to find out the change in edit or with respect to the change in weight.
That is the change in area with respect to the change in weight.
So how will we do that again with the same function which we learned in the.
So the change in L.A. with respect to we will be equal to the change in era with respect to this sigmoidal
function.
I multiplied by this sigmoid function change in sigmoid function with respect to the change in.
So.
What will this be?
Now, the change in this is with respect to W. is nothing by expanding.
What is this?
This is this is nothing but Xingdou W and X is a constant W is the variable.
So if we take a derivative of of a constant and the variable, then be out with no power on top of it.
So the output will be the constant value.
So we get X minus one, which is from the previous year.
Now we need to find out the change in Ayda with respect to the.
S, which is the sigmoidal the segment of this letter.
So what is this?
This is legacy.
This is day.
This is this Valentine's Day.
Now, how do we find now?
So let us say we want to find out for the final.
So this will be equal to change and e with respect to this, so for the final leg changing, even with
respect to S will be equal to now here, the what will be added value will be the function of X minus
Y value for with.
That is predictive value minus the actual value will split.
Now, this is Gwen, is there.
Now, what is this one?
This X1 is nothing but.
Sigmoid function of light or t, the function of light on is.
Right.
This is the definition of light on is what was.
X is X, Y, plus W, one extra W2 this extra on.
And this is what is going inside this, that the summation value which is going inside, this is the
activation function.
And X is the output value after applying the activation function.
So what do we get?
So it's it's and is nothing but sigmoid function applied on the summation value.
So.
Legacy of this dysfunction, this activation function is damage, then the very of and net is equal
to one minus Tusker.
So we can replace it, we can use this value.
So.
If you are checking for the previous year, this is for the last year, we are checking for the existence
of one one previously, Daniel's enemy of those back to the previous years is we will do they love E
with with respect to S Galavis, with respect to the previous year, X into the of previous years,
with respect to the previous year when they this is the final let the previous year and the year before
that date.
So this is how it will just keep on going globally previously and it will just keep on finding out values
from the one layer behind it.
And based on that, it will keep on finding of different values.
So what is the end goal is in which we will be following this, you know, we don't going to the entire
pocket, only thing that we need to know is this entire thing, which is happening so easily.
You will have something in common values and you will think some random values and build up over us
so that the value, the outward value can be calculated now based on the output value and the actual
value.
Some in an amount that we.
And this where we have to be in.
So if we move this into whatever change will be needed, that will be back propagated so that the value
of the wealth can be a big.
And this will be a big theme, the week of each and every live.
So the values each and every year will be big for the week so that we get better.
I think we'll just keep on doing this entire process again and again until you get the results.
So what do we do?
We to make the leads to a small random number and like all the biases, because, you know, we have
not discussed what biases.
So I have this for now.
So what is bias?
Now, here's what we have, is we have this equation.
What does this equation have this equation have W plus W one X, Y plus W two x2.
So these are the V.
And this one, which is being played here.
This is ordered by.
So this is also one, though, which will be a big.
OK, so this is called by.
So the full report is for next March and the activation function.
Now we will go forward and backward buzz by these forces.
Will it be done or whatever they need that we will not take all the data.
And one thing we will take the view that in batches, because it will take a lot of time every day,
all the already that it once.
So we will do this in many batches.
OK, so we will forward, pass and calculate the activation value.
Then we will back what and calculate the gradient and update the gradient by propagating backwards moving.
And we will update the weights and the biases based on the gradient vector calculated from averaging
over the that.
So we will find out the dates by averaging the gradient vector from the many vectors.
So that is what we will be doing.
Now we will have a look at the court in the next session, so I hope that will make this entire thing
a little more clearer so that you will be able to implement these of your own.
So let's meet at the.