‫In the last couple of videos, we learn how to create neural network with keras sequential api
‫the sequential Api was quite easy to use, but it has some limitation.
‫With sequential api, you can only create neural network with simply sequential architecture.
‫You cannot create complex topologies like this where you have multiple inputs or multiple outputs.
‫For that, you have to use functional api.
‫in functional api.
‫We create each of these layers in the form of functions.
‫Or you can say a building block of your neural network.
‫And you can use this functions to create a complex structure.
‫By joining them according to your structure need.
‫Before going into details, let's first delete the model that we created in our last lecture just
‫just write del
‫And your model name.
‫And then you also need to clear the session of your keras.
‫This will free up the resources for our next model training.
‫So just write keras.backend
‫dot clear session.
‫run this, too, now we are ready to create and train our next model.
‫For functional API, we will be using this example.
‫This kind of neural network is known as wide and deep neural network deep because our input is going
‫through two layer of dense hidden layer and wide because of what input is directly going for
‫output as well.
‫So along with the output of this hidden layers, it also connect all parts of our input directly
‫to the output layer.
‫This linkage, which I have marked as white, is not possible when we are using sequential api.
‫The advantage of this architecture is that it makes possible for a neural network to learn both the
‫deep patterns.
‫by this deep linkage and the simple rules by this wide linkage.
‫In our regular MLP, more than all the data flows through this full stacks of dense layers and thus
‫some simple patterns in the data may end up being distorted by the sequence of this transformation.
‫So now let's create this layers one by one.
‫We will be using the same data set as we use for our last lecture.
‫First, we need to create an input layer.
‫We are calling it input and we are creating it with keras.layers, dot input, and then we have
‫to provide the ship.
‫We can either.
‫write.
‫Just 8 in brackets because we have eight independent variables or we can write it in this way.
‫x train dot shape and then the first and so on attribute.
‫So this is our input layer.
‫We will create this kind of layers.
‫And then we will connect these layers accordingly.
‫So next we create a dense layer with 30 neurons using a RELU activation.
‫And you can also notice that.
‫We are calling this input player like a function.
‫So this input layer is a input for this hidden layer one.
‫And we are calling it like a function.
‫And that's why we call it, functional api.
‫We create this kind of layers and we use this layers as functions for our next players.
‫So we have connected our input layer to our hidden layer 2
‫Now we create our second hidden layer.
‫Again, we are using Keras dot layers dot dense, and we are creating it with 13 neurons and activation
‫is relu.
‫And now for this hidden layer two our input should be hidden layer one.
‫So we are passing our hidden layer one as a function here.
‫Now, the next step is this.
‫Here we want the output of this hidden layer 2.
‫And also we want all our inputs variable here.
‫You can see this linkage.
‫We want these two to go into this concat layer.
‫Concat layer is just merging the output of this hidden layer and all the inputs.
‫So we can write this layer as concat and then we'll use keras dot layer, dot concat.
‫And we are passing the list of ourput of hidden layer two.
‫input layer
‫If we wanted output of hidden layer one as well, we can write hidden layer one here and it will add
‫a linkage like this also, so you can see you can customize all this linkages very easily using a functional
‫api
‫Now next step is to create our output layer, our output should get input from the concat layer.
‫And this should be a single neuron.
‫So we are creating output equal to keras dot layers dot dense.
‫And then a single neuron without an activation function and input as the output of contact layer.
‫So we are passing concrete layer as a function here.
‫Now, we have created all the layers.
‫The next step is to combine all this layers and create a model.
‫So we are creating a model object and then we are calling keras.models.model
‫And here we are mentioning what we want as our input and what we want as our output.
‫So our input is this first input layer and our output
‫Is this last output layer
‫Think sequential api we first create a model and then we create each layer, layer by layer, but
‫in functional api
‫We create this layers and then we join this layers to create this whole network.
‫Just run this now again, just to look at the structure of the model that you have created.
‫You can call dot summary.
‫So our object name is model
‫And we are calling dot summary method.
‫You run this, you will get all the details.
‫First, we have an input layer with eight input variables.
‫Then we have a dense layer with 30 neurons than we hav our second dense
‫layer again with 13 neurons.
‫Then we have a concat layer where we are contacting the input of this second dense layer
‫And our input layer.
‫So you can see we have 30 plus 8, so 38 neurons here.
‫And then we have our output where we have only 1 neuron.
‫Now the next step is to compile.
‫And just as in our previous lecture, we will be using mean squared error as lost function. Sgd
‫learning rate of zero point zero zero one as our optimizer and mae or mean absolute error as our additional
‫metrics to calculate.
‫run this.
‫Now, fitting the model is same. we will just write model.fit provide our training
‫dataset, provide the number of epoch values
‫And then the validation dataset, since last time we ran that regression model for 40 epochs.
‫So changing it for 40 epochs
‫So that's just been more than.
‫Again, you will see the lost functions, validation loss and the value of metrices that you have given
‫here.
‫You can see the lost value.
‫We are getting on our training set is zero point three six.
‫This is the mse value and the validation loss is zero point three six three eight.
‫Let's calculate the value on our test data.
‫You can see the lost value here is zero point three zero six four earlier in our last case
‫I guess we were getting loss as 0.2515
‫So this model is not performing that good, but in some situations, this kind of deep and wide network
‫performs better than a normal MLP
‫mdoels
‫In this case our normal MLP model is platforming better than this wide and deep network.
‫Again, you have all the parameters available to you.
‫So if you just write model underscored history.history, you will get all the lost values.
‫And the mae value that we got during the training, and you can also plot this on a graph, just
‫like we did earlier.
‫So the important thing here is that our validation loss or validation and mae value is still decreasing.
‫So there is a scope of further improvement in the accuracy of our model.
‫So let's just run this.
‫model for 40 more epoch as well.
‫Now, let's calculate the.
‫test value as well.
‫You can see now the loss has decreased to zero point two six.
‫Earlier, we will getting around zero point three.
‫Now we have zero point two six as our loss value.
‫You can also plot this graph again to see whether the model have converge this time or not.
‫You can see this is almost a straight line, so we can see that model has converged.
‫And the test value for our model and the mse value for our model is.
‫Zero one two six one on the test set.
‫This is somewhat similar to the accuracy we bought earlier with our normal MLP model.
‫But.
‫I would say that our normal MLP model was performing better than this deep and wide network.
‫So that's all for this lecture in the next lecture.
‫We will see how to save our model model and how to save check points at the end of each epochs.
‫Thank you.