‫Here is a summary table of a classification neural network architecture.
‫In the second table, you can see I have put four columns, first column in both the first table and
‫the second table is for hyper parameters
‫These are the values that we have to set prior to training our model.
‫For example, how many layers will our neural network have is something we have to decide and give beforehand.
‫So the common classification neural network hyper parameters are mentioned in the first row of table
‫one and two.
‫The second, third and fourth column in the second table are for three classification senarios. First is
‫binary classification, which is classifying into two classes like marking a mail as spam or not spam.
‫Second is multilabel binary classification, which means there are multiple binary variables.
‫For example, the first variable is whether a mail is spam or not.
‫And the second variable could be whether a mail is important or not.
‫The third one is multiclass classification, which we discussed in last lecture also
‫if we have four classes, trousers, shirts, socks and ties.
‫This scenario falls under multiclass classification.
‫Now, let's see what values of hyper parameters do we usually use for these three types of classification
‫scenarios.
‫The first parameter is number of input neurons.
‫Number of input neurons are always equal to the number of input features.
‫So if you have 16 input variables, you will have 16 input neurons.
‫So you'll always have one neuron.
‫per input feature.
‫The second hyper parameter is how many hidden layers do we want in our network?
‫Ideally, this depends on the problem.
‫But typically we keep the number of the layers between one to five, keeping more than five hidden layers
‫only increase the computational effort for our system.
‫Third hyper parameter is hidden activation.
‫That is the activation function that we put on the neurons in the hidden layers.
‫This is usually RELU.
‫We discussed rectifier linear unit.
‫It is a very common function which is used for hidden layer activation.
‫I told you earlier, also we used relu because it is very fast to execute in our systems.
‫Other three hyper parameters vary with the type of classification that we are doing.
‫So the number of output neurons in binary classification is one.
‫But in multilabel binary classification, it is one per label.
‫For example, if we are classifying an email as spam or not spam and the other label is important or
‫not important, we will need to neurons in the output layer.
‫One neuron would tell us whether it is spam or not spam.
‫And the other neuron will tell us whether it is important or not important for multiclass classification.
‫We have one output neuron per class.
‫So for example, if we are classifying images into shirts, trousers, socks and ties, we will have
‫four different output neurons for each of these class.
‫And we will put a softmax activation layer on top of it to get the probability of each class happening
‫next, hyper parameter is output layer activation in binary classification, logistic or sigmoid function
‫works very well.
‫You can use step function also.
‫But as we have discussed, logistic function performs much better than a step function.
‫So for binary classification and multilabel binary classification, we use the sigmoid function.
‫But in multiclass classification, after the sigmoid function, we have to put an additional layer of
‫softmax activation. last
‫hyper parameter is lost function.
‫We will be using cross-entropy as the lost function for all types of classifications.
‫So these are the hyper parameters that you have to mention when you are running neural network model
‫in a software.
‫The values that are given here are typical values.
‫That is, these are commonly used values, but it is not a hard and fast rule to use these values
‫only. You can customize your neural network by using any other hyper parameter value.
‫Next is the summary table of regression neural network architecture. Here on the left
‫We have hyper parameters and on the right we have typical values that we use for this hyper parameters.
‫The first one is number of input neurons.
‫Again, it is one per input feature number of hidden layers, that depends on the problem.
‫But usually we keep one to five hidden layers.
‫Then comes number of neurons per 100 layers, again this depends on the problem.
‫But typically we take 10 to 100 neurons in the hidden layer. Then is output neurons
‫If we are predicting only one thing, we need only one output neuron.
‫If we are predicting multiple things, we need one output neuron.
‫per the number of things that we want to predict.
‫For example, if you are predicting house price, that requires only one output neuron.
‫On the other hand, if you are predicting the length and width of a petal of a flower form the image
‫of the flower that requires two output neurons one for the length of the petal and second for the width of the petal
‫Next is hidden activation hyper parameter, which means what will be the activation function in the hidden layers.
‫most commonly used activation function is Relu in the hidden layers relu cannot be used as activation
‫function in output layer.
‫So for regression neural network in the output activation, we do not really need any activation function
‫as such.
‫If you want to play any particular boundary condition on the output, for example, if you want that
‫the output should only be positive.
‫Then you can play a RELU kind of function on top of it.
‫Otherwise, there is no requirement of an activation function on the output layer.
‫Last hyper parameter is lost function for regression neural network.
‫The squared error can also work very well as a lost function.
‫You cannot use cross entropy here.
‫So we often use mean squared error, which is the mean of squared errors that we calculate for individual
‫training examples.
‫So these are all the hyper parameters that you need to specify.
‫While running a regression neural network in the software on the right, you'll see the typical values.
‫These are not fixed.
‫You can still customize your neural network by changing these hyper parameter values.
‫That's all.
‫See you in the practical lectures.