‫In the last lecture, we saw how to create, how to compile and how to train our classification model
‫In tensorflow keras
‫In this video, we will build and train a regression model using keras
‫For this, we will be using the very popular regression dataset that is California housing data set.
‫This dataset is available in a sklearn database library.
‫The objective here is to predict the prices of house using eight different independent variables.
‫So let's get started.
‫First, we are importing some basic liabilities, such as numpy, pandas and matplotlib.
‫Then we are importing tensorflow and keras
‫And then since this data is available in sklearn dataset, we are also importing California housing
‫from sklearn dataset.
‫And we are saving this database into an another variable called housing.
‫I also want to share one small shortcut to access the help of any function.
‫So if you just click inbetween of any function parenthesis and then hold the shift key and click on tab.
‫So if you hit shift plus tab , it will open the help or documentation of that function.
‫If you hit shift and tab two times, it will expand the documentation.
‫You can see here this function will return.
‫Different parameters such as dot data.
‫It will give us the independent variables.
‫Dot target will give us the dependent variable.
‫And the feature name will give us the details of features.
‫Let's just close this.
‫So if you have any doubt regarding any function, just click between parentheses and hit shift plus tab
‫So I have already listed some of the information about this database.
‫So in this database, there are around 20000 records.
‫There are eight independent variables.
‫We have first variable as med inc
‫This is the medium income in that particular block.
‫Warehouse is located.
‫Then we have a second variable that is house age.
‫This is the median house age in that block.
‫Then we have average rooms, which is the average number of rooms.
‫And then we have average bedrooms for average number of bedrooms.
‫Next, we have population variable.
‫That is the block population.
‫Then we have average occupancy.
‫That is the average house occupancy.
‫And then latitude and longitude of that house block using this eight independent variables.
‫We want to predict the value of the house.
‫The values are in hundred thousand.
‫So suppose if our y variable is 5
‫That means the value of that house is
‫Five hundred thousand dollars.
‫So this is our dataset.
‫We have this eight independent variable and one target variable as price
‫If you want some more detail about this dataset, you can click on this documentation link.
‫This will open the official sklearn documentation of this dataset. Here
‫You will get to know about all the parameters that you can give.
‫And what are included in this database?
‫Let's just go back.
‫So this housing is in the form of a dictionary.
‫We have one key value pair as feature name
‫So just look at the feature names first.
‫You can see these are the eight variable names that we have discussed already.
‫Now to access, the independent data we have to use housing.data and to access the independent data set.
‫We have to use housing dor target.
‫So in this line of code, we are splitting our data first into train full and test dataset.
‫Then we add further dividing this train full dataset inti x train and X validation dataset.
‫We will be using test train split from sklearn model selection, and then we will use test train,
‫the split method to divide our data.
‫We are not giving any additional parameter for test size.
‫That's because by default, the test size is 25 percent of the total data.
‫So 25 percent of the total data that is around 20000 rows will go into test set.
‫And then the remaining 75 percent will go into training set
‫All of that 75 percent.
‫Again, 25 percent will go into validation set and rest of 75 percent will go into our training set.
‫Let's run this.
‫Next step is to process our data and we will be using the standard scalar from sklearn to standardize
‫our data.
‫In standardizing, we subtract the mean of each variable from their individual values.
‫And then we also divide it by the variance because at the end we want all the variables with mean as
‫zero and their variance as one 1
‫This is a standard procedure to create any machine learning model the steps here are very simple.
‫First, we are importing standard scalar from sklearn pre processing.
‫Then we are creating the scalar object using standard scalar method.
‫Then we are training.
‫This scalar Object using our x train data
‫So on our x train data, this scalar will find the values to subtract, as a mean, and to divide
‫as a variance.
‫Then we will use those values.
‫Or this is scalar object to standardize our validation and test sets as well.
‫Just so repeat, we are fitting.
‫This is scalar object on our training data.
‫And we are transforming our validation and test set using.
‫This is scalar object that we have fitted it on our x train data.
‫To fit will be using fit underscore transform method.
‫And we will be using x train as parameter to transform.
‫We will be using dort transform method of this scalar object and we will be using relevant datasets
‫here.
‫So let's create a standardized datasets.
‫We are saving this object in their original name only.
‫So we are replacing the ordinal x train with the standardized version of x train.
‫Origional x validation set into standardized version of X validation set
‫And same for test set as well. Let's just.
‫Run this if you want some more information on this scaling.
‫You can always refer to sklearn documentation
‫Abiut standard scalars.
‫Next step is to set random seeds.
‫This is to generate the same result every time we run this modern.
‫Now, as I said earlier, our initial data set was of around 20000 rows ir records
‫Now let us see the shape of our x train dataset.
‫Here you can see we have eight columns and around eleven thousand six hundred records.
‫In our x train dataset
‫.
‫We should around five thousand records in our X test dataset and around 4000 in valuation set.
‫Now, let's create the structure for our regression neural network.
‫Here we will be first having an input layer.
‫Then we will be having the first dense layer with 30 neurons.
‫Then we want to create second dense layer with another 30 neurons.
‫And then since this is a regression problem, we will be having a single output neuron without any
‫activation function.
‫A single neuron, since we want a continuous value as our output.
‫Again, we will be using the sequential api.
‫We are saving this structure or our model as model.
‫And then for the first layer
‫We are writing.
‫Keras dot layers, dot dense.
‫Here in parenthesis, we have to provide the number of neurons, which is thirty.
‫Then, as discussed in our three lectures, we will be using activation function as RELU.
‫And then since this is our first written layer, we need to provide the input shape since the number
‫of independent variables in our data is eight.
‫We will be using input shape equal to 8
‫You can also use like this input shape.
‫Equal to x train dot shape and then calling the second and so on, elements of over input shape.
‫This way you don't have to worry about changing this number every time you change your data base.
‫You can just write x train dot shape and it will automatically get the number of variables from the
‫shape attribute of our x train object.
‫So this is the structure of our first dense layer will be create
‫Second, dense layer in a similar fashion.
‫We will be using kreas dot layer dot dense.
‫And then the number of neurons in the parenthesis, which is 30.
‫And activation function is relu.
‫Similarly, for the output layer, we will be using keras, dot layers dense.
‫And since this is a regression problem, we will be using a single neuron without any activation function.
‫Just run this.
‫And again, one important thing you can comment using hash symbol inside the cells.
‫So python will execute only this part of the code and will not be executing any code which starts with
‫Hash, hash represent for starting the comment.
‫So now we have created the structure or architecture of our neural network. Now just to conform and view
‫The structure.
‫We can call dot summary method just write model dot
‫Summary, here you will get the information about the structure that we have created.
‫So we have first dense layer with 30 neuron
‫We have second dense, layered with 30 neurons.
‫And lastly, we have a single output layer with one neuron.
‫This is what we wanted.
‫The next step should be to compile this model.
‫Again, the compile method works similar for both classification and regression model.
‫First, we have to mention the loss in classification.
‫We were using Cross entropy
‫But here, since we are running regression, we have to use mean squared error, also known as mse
‫The second parameter is optimizer.
‫Again, here also we are using as sgd, socastic gradient descent.
‫And here we are also providing the learning rate by default, the value of learning rated zero point
‫zero one and to change
‫You can just write.
‫The new value in the parentheses
‫We have already discussed what is learning rate in our theory lecture.
‫So if you have any doubts, just revisit that lecture.
‫And then the next parameter that we are passing is metrics.
‫This is an optional parameter.
‫In classification, we were using accuracy.
‫But in regression, we can use mean absolute error or mae absolute error is the difference between
‫the predicted value and the actual value.
‫Whereas the mean is squared error is the square of that difference.
‫So we are calculating both mean squared error as our function and nae as the metrics we additionally
‫want to calculate.
‫Again, just remember, if you want to look at the documentation or help, just click inside any of
‫the parenthesis and.
‫And press shift plus tab, you will get this kind of documentation.
‫And here you can see that by default the learning rate value is 0.01.
‫So in classification, we did not provided any learning rate.
‫So the learning rate that was used, there was zero point zero one.
‫But you can always change these values according to your need.
‫Let's run this.
‫So we have compiled our model.
‫The next step, the next step is to train our model using training data.
‫The method or the process is same.
‫We are creating an another object model underscoring history for training.
‫Then we are using model dot fit and we are passing our training dataset number of epochs and the validation
‫dataset that we have created.
‫Just run this statement.
‫Again.
‫Just like the classification model, you will get the lost value, which is the mean squared error.
‫You will get the mae value mean absolute terror.
‫And similarly, you will get these two values for your validation set as well
‫.
‫And you can see that the loss on both training set and validation set is decreasing with each epoch.
‫Now we have these values for our training and validation set.
‫We can also evaluate performance of this train model on our test set.
‫And we are going to use the same method as fitted with classification model.
‫We'll call model DOT evaluate.
‫And then we will pass our training test.
‫Run this
‫You can see on our training dataset loss is zero point three.
‫That is mse or meann squared error and mae mean absolute error is zero point four four nine three.
‫Now, just like in classification, we can call model history dot history, that will give us the
‫values of all this metrices in the form of dictionary.
‫Run this, here you will get the loss and mae on training dataset.
‫.
‫And validation loss validation mae.
‫The beauty of this is we can plot this dictionary on a plot just like we did for classification, and
‫that will show us how we're training.
‫Loss and validation loss are changing with each epoch and whether we have achieved the convergence or
‫not.
‫Let's run this.
‫So you can see we have the loss values and the mae values for our training and validation
‫set plotted on this graph.
‫And one thing to notice is this graph is still going down, meaning that if we run some more epochs,
‫this will further decrease the losses and improve the accuracy of our model.
‫So this is the one way to tell whether you have achieved convergence or not or whether you have to increase
‫your epoch value or not.
‫You have to look at this validation, loss and validation mae value.
‫So this the validation loss.
‫And you can clearly see that it is going down.
‫So to improve the accuracy, we can re run this code to
‫Run it for 20 more epochs.
‫So just go there.
‫Now, one important thing about keras is, keras have the weights and Bias's value in the memory.
‫So if you just re run this whole this statement, again, this will not train the data from start.
‫But it will start training the data from this position.
‫So if we run this statement two times that is similar to running a statement with 40 epochs.
‫If we just re run it one more time.
‫You can see earlier the lost values were.
‫Around point seven or point eight for the first epoch
‫And then gradually decreasing.
‫But now we have started from the 21st epoch.
‫Last time, the lost value on our test set was zero point three zero.
‫Let's see whether we have improved this lost value or not.
‫You can see the loss is decrease from zero point three to zero point two five.
‫So our hypothesis was correct that the model was not converged in 20 epoch.
‫There was room of improvement and B, B, then the whole more than 420 more epochs.
‫That is a total of 40 epochs.
‫And we can see this graph.
‫You can just.
‫Focus on this validation loss line.
‫Earlier, it was 0.4 around, re run this.
‫There is a slight decrease in validation loss.
‫Now you can see that the line is flattened out.
‫This means we have achieved the convergence on this model.
‫So not just with regression.
‫If you are running classification model as well.
‫Just look at this graph to identify whether you have achieved convergence or not.
‫Now, to predict the values on the new dataset, you can always use, dot predict method.
‫So your object name and dot predict method.
‫And then the new dataset.
‫I don't have any new dataset.
‫So I'm just creating the sample of first three values of my X dataset.
‫And considering it as my new dataset and then saving the information in Y predicted values and using
‫model dot
‫Predict method to predict the values this are the predicted values using this model.
‫That's all for this lecture. In the next lecture.
‫We will be looking at the functional api of keras.
‫Thank you.