So in this lecture, we will continue looking at our notebook for sentiment analysis.
Let us begin by grabbing only the samples, which have been classified as positive and negative.
To do this, we're going to create a list of desired targets which contain the codes for both the positive
and negative class, as you recall.
These are stored in the dictionary called Target Map.
We'll call the result binary target list.
The next step is to filter the f train and D of test where we only want the rows, where the target
is in this list of desired targets that we can accomplish this by using the function is in.
Note that another way to do this would have been to use the not equals operator since we simply don't
want any samples where the target is equal to neutral.
The next step will be to call the head function on one of our new data frames to check that they only
contain positive and negative tweets.
OK, so this appears to be correct.
The next step is to convert our new data set into TF IDF vectors, since we already have a vector riser
object.
We can simply reuse the same one from above as before we call the fit transform function on the train
set and then we call transform on the test set.
The next step is to obtain a wide train and wide test from the target column.
The next step is to train a new model on a binary data set using the same code as before.
Again, we'll print the accuracy to start.
OK, so this time we've done much better, we get about 93 percent on the train set in about ninety
one percent on the test set.
The next step is to compute the AUC.
Note that because we now have binary classes, we only need the complement index one from the probabilities
returned by the predict proper method.
OK, so notice that our EU seas are now much better than before.
The next step in this notebook is to work on interpreting what our model has learned.
We'll begin by demonstrating how to obtain the weights of a trained model.
OK, so notice that we get back a two dimensional array where the first dimension has size one.
Thus, if our model has the inputs, this would be an array of size one by the.
The next step is to plot a histogram of the weights in order to get an idea of the range of values.
So notice that most of the weights are centered around zero.
While there are some outliers around two, three or even four.
Now, as you recall, if we look at our model, only we won't know which words correspond to which inputs.
Thus, it is necessary to obtain a word to index mapping, as you recall.
This is stored in our IDF vector riser objects.
In particular, it's stored inside an attribute called the vocabulary.
OK, so as expected, this returns a dictionary where the key is the word and the value is the index.
So in order to interpret our model, what we're going to do is find the most extreme words for each
class that is, find the words with the largest magnitude of weights, as you recall.
These words are the words that will have the largest effect on the output, provided that the input
is the same since there aren't that many words with the weight larger than two.
We'll set a threshold at two.
Alternatively, you could use a more statistical method like percentiles.
The next step is to live through each word in our word to index mapping.
Note that on each iteration, we get the word and its index at the same time.
Inside this loop will index our model weight vector using the index called index.
And this will give us a single weight.
The next step is to check whether this weight is larger than a threshold.
If this is the case, then we will print the word along with its corresponding weight.
OK, so these results seem to make a lot of sense.
We can see words like, Thanks.
Great.
Best love.
Appreciate.
Awesome.
Kudos.
Amazing.
And so forth.
Interestingly, we also see some terms which correspond to the name of the airline itself.
Perhaps it's simply the case that these are the airlines which are typically associated with positive
sentiment, or that users tend to use these names more in positive tweets.
So the next step is to look at the negative words.
Note that this loop is the same as the previous loop, except that this time we check whether the wait
is less than the negative of a threshold.
That is, we're going to print the words for any weight, which is less than minus two.
OK, so these words also make a lot of sense.
We see words like hours delayed, cancelled.
Nothing worst.
Why?
And luggage, of course.
Terms like delayed or cancelled in luggage are associated with common problems that people have when
traveling by plane.
So it's pretty clear that these words would be associated with negative sentiment.
So now that you understand how to do sentiment analysis, the following exercises are designed to help
you practice further.
The first exercise is to check which tweets are model is getting wrong.
In particular, you're going to print the most wrong tweets for both classes.
In other words, this means find a negative review where the output probability is closest to one.
This means, out of all the negative reviews our model got wrong, this one was the one it was most
confident about.
Print the probability as well.
Similarly, you want to do the same process for the positive reviews.
In other words, find the most wrong a positive review.
The second exercise is very simple.
One easy way to help your model handle imbalanced classes is to simply weight the loss function.
Now, since we've only discussed the intuition behind logistic regression, you may not know exactly
what this means.
But if you decide to study this topic further, this will make more sense.
Luckily, the code for this variation is very simple.
When you instantiate your model, you're going to send an attribute called class way to the string balanced.
This will weight the laws function based on the frequency of each class.
So try this and observe how the confusion matrix changes.
You should find that the model does better on the classes which were underrepresented, meaning the
positive class in the neutral class.
Of course, this improvement doesn't necessarily come for free.
You may find that doing this also leads to the model performing worse on the majority class, which
corresponds to negative sentiment.