In this section of the course we are going to look at how deep learning can be used for recommender
systems.
This lecture will discuss the theory behind how deep learning can be applied.
And following that we will look at the code to start.
I want to talk about recommender systems as a general concept.
Where can we find recommender systems and what is their purpose.
I think you'll be surprised to find out that recommender systems are one of the most applicable machine
learning concepts ever.
It's a type of machine learning that can be used by nearly any consumer facing business and you most
certainly encounter recommender systems multiple times daily.
Let's start with the first Web site.
Everybody goes to when they go online.
Google Google search is in fact a recommendation engine.
Actually when you perform a search multiple recommendation engines are working to build the page that
you see.
Not only do you have the search results which are recommendations based on your query your geographical
location and maybe even your search history.
But don't forget that Google is also an advertising company.
The online advertisements you see when you go to Google dot com are also recommendations based on your
previous browsing history
another great example is Amazon.
When you search for a product on Amazon it's more likely that you'll see advertisements for similar
products in the future.
You might also see recommendations for related products.
For example if you're buying an iPod you might also be in the market for new headphones a case and a
screen protector
another great but seriously unfortunate example is the news news sites employ recommendation systems
and they even do a b testing on their article headlines to improve click rate.
This is very unfortunate because this means that news has become more biased and more polarizing over
time.
People don't care about boring news.
They want to read something exciting and news companies are more than happy to give that to you in place
of real factual and unbiased news
there are really too many examples of recommendation systems to mention.
Here are a few more examples to help you internalize what exactly it means to recommend we have Netflix
which recommends movies and TV shows.
We have YouTube which recommends videos we have read it which recommends a mix of user generated content
and third party content posted by users.
We have Facebook which sorts your news feed based on what will make you stay on it the longest.
Closely related to that we have Instagram which is owned by Facebook and it does largely the same thing
you're shown a feed of Instagram posts that are recommended in order to keep you engaged.
In this lecture we'll be focusing on a specific kind of recommender.
This recommender works on data that comes in the form of triples the three items that form a sample
or the user the item and the rating that the user gave that item.
As an example suppose you're creating a movie recommender.
Here are what a few rows of your data set might look like.
Alice rates.
Avatar 5 Bob rates.
Star Wars of 4.5 and Carol rates the godfather of 4.
So all of our samples will look like this.
The first item is the user.
The second item is the movie.
And the third item is the rating
one important feature of this type of dataset is that it must be incomplete.
Why.
Well think about it.
If all users have already watched and rated all existing movies then what would there be to recommend
if I've already watched and rated every movie in existence.
There is no need to recommend anything to me.
You can't recommend a movie to me because I've already seen it.
Luckily for any real data set.
This won't be the case.
Imagine you have 1 million users and one hundred thousand movies which is not unrealistic.
You can probably count the number of movies you've seen in your life and it's probably a lot less than
one hundred thousand.
The number of movies you've actually bother to rate is probably even smaller.
The question now is once we have such a dataset how does it help us make recommendations.
Let's take a very simple example consider Bob Bob R rated every movie in the Star Wars series of 5 Bob
also rated a few of the Star Trek movies of five.
And Bob also rated Avatar five.
Common sense tells us that Bob is probably a fan of sci fi.
Bob likes movies that have space travel aliens and so forth.
It is probable then that Bob would like movies that have the same features.
A pretty obvious recommendation for Bob would be a star trek movie or a TV show that he hasn't seen
yet.
Let's try to generalize this concept.
As you know we're given a dataset that consists of triples containing user's items and the user's ratings
for those items.
Suppose we can fit a model to this data.
Let's just call it F F takes in two arguments.
A user you and a movie M and it outputs a predicted rating.
There are two things that we would like this function to do.
Number one obviously if the user movie and rating appear in our dataset then we would like the predicted
rating to be close to the real rating and number two.
We would like this function to be able to output a predicted rating even if the rating did not appear
in our training center luckily a neuron that work is a perfect model for this use case.
As you know a neuron that work is a function approximate how it fits to the data set so that when it
makes predictions those predictions are close to the training targets but not only that it can also
make new predictions for data which did not appear in the training set.
So how might we use such a model to make recommendations.
Well here's one strategy.
Once we have a model that can predict ratings this becomes easy for a given user predicts a rating for
every movie that they have not yet seen then sort these movies by predicted rating the movies you want
to recommend are simply the movies with the highest predicted rating for this user.
Pretty simple
The next question is how can we go about building such a model.
We know that we need a neuron network of some sort.
But here's a problem both users and movies are categorical objects neural networks are basically a series
of matrix multiplications.
As you know we can't multiply a categorical object by a number.
What is the Star Wars times five.
This does not make sense
luckily we have encountered this problem before.
Strangely natural language processing is the field we draw inspiration from in order to build deep recommended
systems.
As you know in MLP the main input is words.
Words are categorical.
They cannot be multiplied by numbers.
So what do we do.
We use an embedding Recall that an embedding is a mapping from each category to a feature vector which
is essentially a list of numbers that can be multiplied
here's how we might use a neuron that work for recommender systems.
Let's say we have an input user Bob and an input movie star was the first thing we do is map both the
user and the movie to their respective embedding vectors.
So I have a vector for Bob and I have a vector for Star Wars.
Now usually these embedding vectors are the same size but this need not be the case.
So now both Bob and Star Wars are represented by feature vectors.
The next thing we do is concatenate these two feature vectors into a single feature vector.
Once we have a feature vector the rest is obvious.
We just do the same thing we've always done pass this through a neural network since this is a feature
vector and not some special object like an image or a sequence.
We can use simple antennas rather than CNN or Arnold's for recommender systems.
The final output layer will be a linear regression with no activation function since predicting ratings
is a regression task and not a classification task
as a point of interest.
I said earlier that we can use an techniques to inspire models for recommender system.
In fact the opposite is true as well recommender techniques have been used in an LP.
My favorite example of this is a technique called Matrix Factorization Matrix Factorization which we
haven't talked about is an algorithm that has been known for quite a while in the field of recommender
systems around the time word embedding were becoming popular.
Specifically due to the algorithm known as works of EQ another similar algorithm came out called a glove
Glove essentially solves the same task as where to vac.
It's an algorithm for finding word meetings using these words I'm betting you can do all sorts of fun
analogies such as King minus man equals Queen minus woman.
Well it turns out that glove is nothing more than Matrix Factorization.
Just an old technique from recommender systems so it's always good to study techniques from different
fields because you never know when or what you learn that can be applied to your own work.