So in this lecture, we will be looking at the notebook to implement binary classification using TensorFlow.
Note that the main outline of this notebook will be the same as our previous sentiment analysis example.
The only difference is that we'll be using TensorFlow instead of Saikat learning.
So let's begin by downloading our data set, which is a set of tweets about airlines.
The next step is to import everything we need for this script.
Note that we've imported the binary cross and tree loss.
The next step is to set random seeds for Nampai and TensorFlow so that we all get consistent results.
The next step is to read in our data frame using PD to read 6V.
The next step is to call the head method to remind ourselves what this data looks like.
OK, so as you recall, there are many columns in the state of frame, but the ones we are interested
in are the airline sentiment and the text of the tweet.
The next step is to select the two relevant columns and to reassign this 2df.
The next step is to draw histogram of our labels.
So as you recall, this is a free class data set with the classes positive, negative and neutral.
We'll be ignoring the neutral class.
Also recall that the data set is imbalanced, with negative tweets far more common than positive tweets.
The next step is to filter out all of the neutral tweets, since we only want positive and negative
for this analysis.
We'll also want to make a copy of this point since will be assigning new columns to this data frame.
The next step is to create a mapping from the label to an integer representation, as you recall, deep
neural networks work with numbers and therefore this will be needed unlike say I could learn.
There is no functionality to do this kind of conversion internally.
The next step is to call the head method once again to see what our new data frame looks like.
OK, so now our data frame has three columns with sentiment text in the binary target.
The next step is to split our data into train and test.
The next step is to convert our data into RFID, a format.
Now, as you recall, the TF idea of matrices returned by psychic learn are sparse matrices.
Unfortunately, TensorFlow doesn't know how to deal with these.
So we'll call the two array method to convert them to regular Nampai arrays, since we're only keeping
two thousand features.
This is not an issue.
The next step is to assign our targets to convenient variable names, white train and white test.
The next step is to get the number of input dimensions for our model, which is the number of columns
in a train.
The next step is to build our model.
Note that it includes a single dense layer, which represents the expression W Transpose X plus B,
as you recall.
This is a model with the inputs in one output.
Also notice the lack of use of the sigmoid, since this will be accounted for in the binary cross entropy
loss.
The next step is to call a model that summary to get a sense of our model structure.
As you can see, our dense layer has two thousand one parameters as an exercise, please pause this
video and make sure that this makes sense to you.
OK, so hopefully you were able to think about why this makes sense, since we have 2000 inputs.
We should also have 2000 corresponding weights.
On top of that, we have one bias term since there is one output.
The next step is to call the compound method.
As mentioned, the correct loss for binary classification is the binary cross entropy and we set from
logic to true.
Since our model does not apply a sigmoid.
Otherwise, the default for from logic is false, and you would have to apply the sigmoid explicitly.
Again, we'll use the Atom Optimizer with the non default learning rate of zero point zero one.
Since this helps training proceed more quickly.
Finally, note that we also want to track the accuracy metric.
In addition to the loss.
So the next step is to call a FID method.
Most of these arguments you've seen before, except for perhaps validation data.
This is where you pass in your out of sample data so that we can compute out of simple metrics at the
end of each epoch.
So notice that during this process, we get information about the training loss, the training accuracy
and the validation loss and the validation accuracy.
The next step is to plot the loss per epoch.
As you can see, we get a nice decrease on each step, but notice how the validation loss does not go
as low as usual.
Machine learning models tend to perform better on the train set.
Also notice how the validation loss increases a bit at the later epochs.
This could suggest a bit of overfitting in order to rectify this.
We could use a callback to save the best model during training, according to the validation loss.
The next step is to look at the accuracy on each epoch.
So again, we see the same pattern.
There was a nice increase on each step and the validation accuracy is a bit less note again, how the
accuracy for the validation set decreases a bit near the end.
The next step is to get our models predictions now because our model outputs logics and not probabilities.
We won't round the outputs, but rather check whether or not they are bigger than zero.
After doing this, we get an array of cruise and falsies which we would like to convert into numbers,
so we multiply by one.
Finally, we call flatten so that p train and p test become one dimensional arrays.
The next step is to compute the confusion matrix for the train sets.
The next step is to plot the confusion matrix for the train set.
As expected, we see better performance for the negative class, which is the over-represented class.
The next step is to plug the confusion matrix for the test set.
Again, we see better performance for the negative class.
Now, since this is an imbalance dataset, we should compute other metrics like the AUC.
In this case, we get an AUC in the 90s for both training and test, which is pretty good.
The next step is to check the F1 score.
In this case, our train of one is over 90 percent.
But our test F1 is about 79 percent.
The next step is to check the layers attribute of our model.
Again, we have our input layer and the dense layer at position to.
The next step will be to get the weights of the dense layer.
So again, note that this returns a list of two Nampai arrays containing the Matrix and the B vector.
The next step will be to assign the weights of our model to a variable called W.
The next step is to grab the word to index mapping, as you recall.
This is useful because we want to know which words have the most positive and negative weights.
This should help us interpret the model and to check whether or not it's doing something that makes
sense.
The next step is to print the most positive words, according to our wits.
Now you'll notice that this code is a bit strange, but this is due to legacy reasons.
And by that, I mean the previous iteration of this notebook, which was written and so I could learn
instead of printing all the words that have a weight above some threshold, I've simply decided to print
the top 10 words.
What we could have done is just sort it all the words and prints the top 10 and not check the threshold
at all.
In any case, this has no effect on the results.
OK, so the top positive words are thank thanks worries, great, awesome love.
Excellent kudos.
Amazing and best, which all makes sense.
Now, let's go to the negative words, which basically have the same code, but in reverse order.
So the top and negative words are worst paid, not rude, disappointed, nothing web site hung instead
and lists, most of which make a lot of sense.
OK, so that's logistic regression in TensorFlow as a final exercise for this lecture.
Please take this code and adapt it for a spam detection dataset.