Hello.
Welcome back.
Activation functions introduce non-linear properties to our neural network.
The main purpose is to convert an input signal of a unit or a node in a 1 neural network to an output
signal that outputs signal is then used as an input in the next layer.
We bought a sum of product of input x and the corresponding weight and apply an activation function
to it to get the output of the LIA and feed it as an input to the next layer.
If we do not apply an activation function then the output signal will simply be a simple linear function.
Linear equations are easy to solve but they are limited in the complexity and have less power to learn
complex functional mapping from data and no their work without an activation function would simply be
a linear regression model which has limited power and lacks good performance.
Also without activation functions our neural network would be um would find it difficult or would not
be able to even land and model other complicated types of data such as images videos audio speech etc.
We often refer to activation functions as nonlinear routines nonlinear functions are those which have
degrees more than one and have a curvature when plotted apart from requiring that a war activation function
is non-linear.
We also need it to be easily differential meaning we needed to we need to be able to compute the derivative
of the activation function quickly we need it to be this way so that we can quickly perform something
known as back propagation quickly in a what neural network the three most popular activation functions
are the sigmoid the ton H and directed linear units also known as Red loop for short the sigmoid activation
function has the deform f of c course one over one plus E raised to the power minus C and the curve
has an S shape as we can see over here d ton H is expressed thus FFC equals E race the policy minus
you raise the bar minus the all over you rush to to the policy plus E Race to the power minus the s
we can see over here the range of the sigmoid is between 0 and 1 and the range of the of the 10 H is
minus 1 and 1 as we can see in the graph over here this is zero point over here this is one point over
here for the sigmoid and is the ton H this is minus one here and this is one the realm is expressed
as Max zero Zi meaning f of C E course Max zero C this means if Z is less than zero the result is zero
if C is greater than or equal to zero direct to resort is one the sigmoid function is good for output
Lay is only the ton H function performs best when used in it in layers the realm is versatile or versatile
meaning it's good for all this we also show over here the derivatives of these activation functions
we shall implement both the functions and the derivatives in code.
So this order is for this lesson and I shall see you in the next lesson.