‫In the last lecture we discussed a single cell called perception.
‫Now in this lecture we are going to extend the concepts that we learn in the last one.
‫I told you that a perception takes in binary input that is 1 and 0 and gives out a single binary output.
‫But there is no logical reason to put this limitation.
‫We can easily extend this to any real input values.
‫So instead of having black and white only or zero and one only we can have different shades of grey
‫as well.
‫That is we accept any deal value as input DVDs and threshold still function in the CMB.
‫Next we will take a look at this equation of perception.
‫We will slightly modify it to lead to generally used equation in this equation.
‫We are multiplying without adding these terms and comparing them with the threshold we will make a small
‫change here.
‫Bring this threshold to the left and right.
‫This new term as B.
‫Basically it means that we have B is equal to minus or a threshold.
‫People usually call this constant as the bias doesn't really make any difference.
‫But this is the mathematical representation of perception as you would find in most of the books.
‫Now let's move on and look at the graphical representation of this function
‫if you look at this graph.
‫If the calculated value of this left part that is summation of weight multiplied by features plus the
‫bias if this summation if this left part is less than zero.
‫The output comes out to be zero.
‫So you can see in the graph below zero the output of the function is also zero.
‫When this left part is greater than zero.
‫This function suddenly activates and gives an output of 1
‫this type of function is called a simple step function.
‫This is one type of activation function activation functions are basically those functions which take
‫into account some people.
‫Threshold value here the threshold value is 0.
‫And this function takes a sudden step at this threshold value which is why it is called a step activation
‫function.
‫There are many other types of activation functions.
‫Most popular one is the sigmoid function.
‫It is a pictorial representation of how sigmoid function looks.
‫It is a smooth S shape go.
‫It also has a minimum of zero at minus infinity and maximum of one at plus infinity.
‫But instead of having a step and raising suddenly this function arises gradually and continuously.
‫This function is also called logistic function and is also used in logistic regression which is a very
‫basic classification algorithm
‫not the sigmoid function solves a major problem that we have with this step function.
‫When we are training our perception using historical data to find the value of wheat and threshold this
‫step function is very sensitive to individual observations.
‫For example when we are classifying fashion objects in our fashion M NIST dataset and our algorithm
‫is mis classifying a particular image of boobs as trousers to rectify this a model will need to find
‫new weight and biased values.
‫This is where the problem comes.
‫Small change in the weight and biased values will completely flip the output for a lot of the other
‫observations.
‫This makes the step function very hard to control with sigmoid function.
‫The change is gradual so it is easier to control the behavior.
‫Now when we replace this type function with a sigmoid activation function we call this new cell as a
‫sigmoid neuron or a logistic neuron instead of perception.
‫Mathematically a sigmoid function formula looks like this.
‫It is sigmoid or z is equal to one upon one plus it is to the power of minus.
‫And if you plot this function on the graph that is if you have the Z on x axis and you calculate the
‫value of this function using this formula and plotted on the y axis this is how this formula looks like.
‫Now we will replace the value of Z with this summation plus bias value.
‫So WG a C plus B was the input to r activation function so we input this in place of Z.
‫So this is what the output of r neuron looks like.
‫It is one upon one plus exponential minus summation of words with features minus b..
‫If you calculate this value it will always lay between 0 to 1 and it will have a shape like this.
‫So you can compare it with this that function also in step function we calculated output using this
‫formula with regard 0 if the summation was less than zero and we got one if the submission was greater
‫than equal to zero we have replaced this step with the sigmoid function.
‫This is a continuous function.
‫We do not need two parts to it.
‫So we just input the value of WD X JS and the bias to calculate the output which is a continuous function.
‫Now with this are artificial neural cell is ready which takes in any number of real value inputs and
‫gives an output between 0 and 1.
‫It is time to create an artificial neural network which is basically a network of these individual cells.
‫So just a brief recap of this class.
‫Initially I said that we taken binary input and gave out one single binary output.
‫We replaced the input from binary to any real value and we have replaced the binary output to a value
‫between 0 and 1.
‫So in this generalized form we taken any input which have any real value and we get one output with
‫lies between 0 and 1.