So in this lecture, we will be looking at the problem of sentiment analysis, as with our other application
based sections of this course.
This section will be split up into two parts.
The first lecture, which is this lecture, will focus on describing sentiment analysis in why we would
want to do it.
The subsequent lectures will focus on specific solution methods we will use.
So what is sentiment analysis?
Consider the following sentences.
Wow, that was a really great film.
And now consider this sentence.
I can't believe I wasted two hours of my life on that film.
So what is the difference between these two sentences?
Well, it's probably very clear that one of these sentences is a very positive reaction to a film,
while the other is very negative.
This is the concept of sentiment.
Typically, we categorize sentiment as being either positive, negative or somewhere in between.
Note that sentiment analysis is typically thought of as a classification task.
Sometimes we simply call it sentiment classification, which is, in my opinion, a bit more clear now
because sentiment can be thought of as being either positive or negative.
We often build a binary classifiers to discriminate between these two classes.
However, you will find instances where there might be three categories or even five categories.
For example, in some datasets, the categories are positive, negative and neutral.
In other data sets.
Perhaps we'd like to take a more fine grained view.
So we have positive, very positive, negative, very negative and neutral.
So there are some different possibilities when we want to categorize sentiment.
However, note that this does not always have to be the case.
It would be just as natural, for instance, to treat sentiment analysis like a regression task, perhaps.
Zero would represent neutral sentiment, greater than zero would be positive, and less than zero would
be negative.
In fact, one common and related application in machine learning is the recommender system and these
systems.
We typically analyze and try to predict ratings.
One very common rating system, which I'm sure you've seen and which is used for hotels, movies and
online courses, goes from one to five stars, five stars being the best and one star being the worst.
Interestingly, in these scenarios, we normally treated like a regression task, which is interesting
because there's no real difference between five different star ratings and five different sentiment
categories.
In other words, why do we treat this like regression in recommender systems, but we treat it like
classification and sentiment analysis.
In any case, I would personally opt to do a regression in the case where you have many categories since
the categories are not independent but related on a scale.
We know that five is better than four, four is better than three and so forth.
Likewise, we know that very positive is more positive than positive.
Positive is more positive than neutral.
Neutral is more positive than negative and so forth.
This is unlike other kinds of classification tasks, like predicting an object inside an image.
For example, if we're trying to discriminate between cats, dogs and bicycles, there's no ordering
to these categories.
So for sentiment, it makes more sense to do regression, in my opinion.
Thus, if you're the lead data scientist on your project and you have the opportunity to choose the
kind of task you want to do, that might be a good choice.
But if you're given a dataset like we will be in this course, then we'll just use what we have.
OK, so just to be super clear, what is the task of sentiment analysis?
This is a task where we are given a document and we want to predict the sentiment of that document.
As mentioned, the sentiment can be defined as a category like positive or negative or as a real number
like plus one point two either way.
Note that this is another example of supervised learning, since the sentiment will be the target.
Furthermore, this should help give you a sense of how a sentiment analysis dataset would be structured,
as with other supervised learning methods.
We will be given a table of two columns the text and the targets.
The text contains the document we want to make a prediction for, and the target contains the sentiment
value.
So at this point, you should already have a pretty decent idea of how you would go about building a
machine learning model to solve this task.
In fact.
Note that it is no different from spam detection, as you recall.
This is just another instance of my rule.
All data is the same.
So the next topic of this lecture will be to try and understand why sentiment analysis is useful in
the real world.
As always, I like to think of this in terms of money because most students understand money and most
students understand that either as a business or as an individual.
Your goal is to make more money.
So how can you use sentiment analysis to make more money for your business or yourself?
In order to understand this, it's helpful to look at some examples.
One example is reputation management, which involves monitoring of social media.
Basically, the idea is you want to automatically look at tweets and posts on Reddit, Facebook and
so forth that users are making about your company.
You can then pass these through your sentiment analyzer and create reports showing user sentiment statistics.
Note that this can all be done programmatically, so no one has to go in and read tweets themselves,
which saves your company time and money.
You can then take action based on what your customers have said about your company.
Furthermore, note that this is not limited to your own company.
You can do the same with your competitors as well.
So if you're Apple, you might be interested in the sentiment behind the Google Pixel or the Samsung
Galaxy.
Another example is to predict the sentiment of customers who are using your customer support.
Perhaps the action taken by your customer support team is different based on the sentiment of the customer.
Another popular example is to use sentiment analysis on news articles, tweets and other media concerning
companies and industries whose stock prices might be affected by the news.
So if your model predicts that the sentiment of an article is negative, you might want to consider
selling your stock and vice versa.
If the sentiment of an article is positive, by making more intelligent trades, you can profit by buying
and selling stocks at the right times.