So in this lecture, I'm going to give you an official exercise prompt for this section, which is to
implement sentiment analysis, as with the other exercises in this course, please feel free to look
at the official notebook in order to get the data set.
But please do not cheap by looking at the whole solution.
So the exercise can be described.
Quite simply, what you are going to get is a data set of tweets which are labeled as positive, negative
or neutral.
Your job, of course, is to build a classifier and assess its accuracy on both the train and test sets.
Note that because the data set is just a single file, you will need to split the data into train and
test yourself.
Also note that this dataset comes with columns which you do not need.
Therefore, you'll need to look at the data carefully to choose the right columns for the text and the
labels.
So let's go through some additional details that may help you complete the exercise.
Firstly, note that it will be your choice which victimization strategy you want to use, as you recall.
This will include tokenization as well.
You might choose the count victimizer with default settings or even TFI Taf with stop words and limits,
ization and normalization and so forth.
So that is up to you as your classifier, you should use logistic regression, which is available inside.
Get Learn.
You may be interested in trying other classifiers as well.
Finally, you want to check the performance of your model.
Note that by default, when you call the score function inside, you learn this returns the accuracy.
However, recall that this is not an ideal scoring function when the classes are imbalanced.
Thus, you should check whether the classes are imbalanced in order to determine if other scoring functions
are necessary to use some examples of scoring functions that take into account class imbalance or the
AUC and F1 score.
In addition, you'll want to plot the confusion matrix.
Now, one thing to pay attention to is that the U.S. and F1 are not naturally designed for multiclass
problems.
However, circuit learning does offer several alternative options for the multi class case.
So I encourage you to play around with these options and check out the results.
Now, although the original data set has three classes, your next task will be to build another classifier
which learns from among only two classes positive and negative.
That is, you're going to filter out the neutral class.
Note that the code for this will essentially be the same since the interface for Saikat Learn works
for both binary and multi class data sets.
Firstly, what you should notice is that this greatly improves the results.
This makes sense since the neutral class is in between positive and negative in terms of polarity.
That is, we're likely to confuse neutral with the neighbouring classes and vice versa since we expect
positive and negative tweets to be very different from each other.
They should be easier to discriminate between.
Finally, after you've trained your binary model and checked its performance, you should also interpret
the weights in particular.
Try printing out the words corresponding to the most positive and the most negative weights.
I think what you'll find is that these are, for the most part, unsurprising, which is a good thing
since it means our model is working as expected.
So good luck and I'll see you in the next lecture.