In this video, we will learn how to use the Keras library to build models for
classification problems. Let's say that we would like to build a model that
would inform someone whether purchasing a certain car would be a good choice
based on the price of the car, the cost to maintain it, and whether it can
accommodate two or more people. So, here is a dataset that we are calling "car_data".
I already cleaned the data, as you can see, where I used one-hot encoding to
transform each category of price, maintenance, and how many people the car
can accommodate, into separate columns. So the price of the car can be either
high, medium, or low. Similarly, the cost of maintaining the
car can also be high, medium, or low, and the car can either fit two people or
more. If you take the first car in the dataset, it is considered an expensive
car, has high maintenance cost, and can fit only two people. The decision is 0,
meaning that buying this car would be a bad choice. A decision of 1 means that
buying the car is acceptable, a decision of 2 means that buying the car would
be a good decision, and a decision of 3 means that buying the car would be
a very good decision. Let's use the same neural network as the one we used for
the regression problem that we discussed in the previous video. So a network that
still takes eight inputs or predictors, consists of two hidden layers, each of
five neurons, and an output layer. Next, let's divide our dataset into
predictors and target. However, with Keras, for classification problems, we
can't use the target column as is; we actually need to transform the column
into an array with binary values similar to one-hot encoding like the output
shown here. We easily achieve that using the "to_categorical"
function from the Keras utilities package. In other words, our model instead
of having just one neuron in the output layer, it would have four neurons,
since our target variable consists of four categories. In terms of code, the
structure of our code is pretty similar to the one we use to build the model for
our regression problem. We start by importing the Keras library and the
Sequential model and we use it to construct our model. We also import the
"Dense" layer since we will be using it to build our network. The additional import
statement here is the "to_categorical" function in order to transform our
target column into an array of binary numbers for classification. Then, we
proceed to constructing our layers. We use the add method to create two hidden
layers, each with five neurons and the neurons are activated using the ReLU
activation function. Notice how here we also specify the softmax function as the
activation function for the output layer, so that the sum of the predicted values
from all the neurons in the output layer sum nicely to 1. Then in defining our
compiler, here we will use the categorical cross-entropy as our loss
measure instead of the mean squared error that we use for regression, and we
will specify the evaluation metric to be "accuracy". "accuracy" is a built-in
evaluation metric in Keras but you can actually define your own evaluation
metric and pass it in the metrics parameter. Then we fit the model.
Notice how this time we're specifying the number of epochs for training the
model. Although we didn't specify the number of epochs when we built a
regression model, but we could have done that. Finally, we use the predict method
to make predictions. Now the output of the Keras predict method would be
something like what's shown here. For each data point, the output is the
probability that the decision of purchasing a given car belongs to one of
the four classes. For each data point, the probabilities should sum to 1,
and the higher the probability the more confident is the algorithm that
a datapoint belongs to the respective class. So for the first data point or the
first car in the test set, the decision would be 0 meaning not acceptable,
since the first probability is the highest, with a value of 0.99 or close to
1, in this case. Similarly, for the second datapoint, the decision is also 0 or
not acceptable, since the probability for this class is the highest, again with a
value of 0.99 or almost 1. For the first three datapoints, the model is very
confident that purchasing these cars is not acceptable. As for the last three
datapoints, the decision would be 1 or acceptable, since the probabilities for
the second class are higher than the rest of the classes. But notice how the
probabilities for decision 0 and decision 1 are very close. Therefore,
the model is not very confident but it would lean towards accepting purchasing
these cars. In the lab part, you will get the chance to build your own
regression and classification models using the Keras library, so make sure to
complete this module's lab components.