So in this lecture, we are going to answer the question, what is a vector?
Now the reason we need to answer this question is because it's going to be very important throughout
the course.
Vectors are what we are going to use as our numerical representation of text.
They are also the foundation for machine learning, which can be seen as a purely geometrical problem.
We'll speak more about that later.
But for now, just know that vector representations are kind of like the raw materials of machine learning
and data analysis.
OK, so now let's answer the question, what is a vector?
We'll start by answering this question generically that is not in a sense that is related to machine
learning or NLP.
So there are two ways to think of vectors.
The first way is probably what you were introduced to first in your high school math or physics course
is a vector is a quantity that has both magnitude and direction.
This is opposed to a scalar which has a magnitude and perhaps also a sign.
But no arbitrary direction.
As an example, velocity is a vector because it has both magnitude and direction.
I can say my velocity is 10 meters per second in the northeast direction at the same time speed as a
scalar.
I can say that my speed is 10 meters per second, but there is no associated direction.
Now, the second way to think of vectors is arguably more useful, especially in the context of machine
learning where we work in very high dimensions, in high dimensions, direction is not intuitive and
the number of angles you need to specify.
The direction will grow with the number of dimensions and thus the second way, which is to represent
a vector as an array of scales is typically more useful.
So, for example, the vector or array three four on the X y plane is an equivalent way of representing
the vector, which has magnitude five and direction is zero point nine three radians from the x axis.
Now, you may recognize that one of these representations uses a Cartesian coordinate system, while
the other uses a polar coordinate system.
In fact, if you think about it, both of these forms are still arrays of scalar components.
It is simply that the components mean something different in each case.
But notice that with either representation, a two dimensional vector still requires two components.
Now, in practice, our vectors could have hundreds or thousands of components.
So it's more useful to think of them as simply points in a Euclidean space.
That is, each component corresponds to an axis and each axis is orthogonal to the other axis.
This is better than thinking of the components as angles.
OK, so now that we know what vectors are, let's think about why they would be useful, especially
in the context of NLP.
In order to do this, it helps to consider what we would do if we did not convert our text into vectors.
Suppose that we only worked with text or sequences of words.
Suppose I said, write a computer program that will take as input the text of an email and output,
whether or not that email is spam.
Consider how you would solve this problem without having to resort to converting the text into a numerical
representation.
In fact, I recommend actually giving some deep thought into how you would write this function.
Once you start to think about this, you should realize that the application of machine learning is
not as trivial as you might assume.
Suppose that this is the 1990s, and machine learning is still not a very popular subject.
Suppose you're a software engineer at AOL, and you just graduated from college with a computer science
degree.
So you know how to write a computer program, but you know absolutely nothing about machine learning.
What would your approach be?
Well, here are some things you could try.
You could try looking for a certain keywords.
For example, if an email contains phrases like Nigerian prince or insurance or credit or make money,
you might want to consider these to be spam emails.
In fact, you could set up email filters right now that do this.
For example, in Gmail or a client such as Thunderbird.
Now, of course, your program doesn't end there.
What we have now is imperfect.
What if, for example, you're interested in data science courses that teach you how to build models
to computer credit scores?
In this case, you would not want to mark the email as spam simply because it contained the word credit.
Or perhaps you're taking a course in NLP and the instructor uses the Nigerian prince example in an announcement
about the course.
Since the course covers spam detection.
In this case, you would not want to mark the email to spam simply because it contain the phrase Nigerian
prince.
So you can see that things are not so clear cut.
Consider what criteria you would use if you want to consider an email to be safe.
Perhaps if the email contains the terms Udemy or lazy programmer, these should be considered safe.
This is because you signed up to receive emails from these entities and thus you actually want to receive
them as a side note.
Recall that we are trying to detect spam based only on the text itself, but we are assuming for now
that we cannot check the sender.
Of course, Udemy is a big company and thus likely to be targeted by spammers.
And so if a spammer decides to write fake emails pretending to be from Udemy, then your rule is broken.
Now what if the previous two rules we just discussed conflict?
How would you write code to deal with this scenario?
Perhaps rule number one should always Trump rule number two or vice versa.
But again, this is clearly not easy to write code for.
And as you add more and more of these rules, you can see how things can become quite complex.
So hopefully this makes you realize the need for numerical representations.
And beyond that, machine learning techniques.
OK, so at this point, you now understand that working with raw text is difficult and working with
numerical representations is potentially more simple.
We also now understand that these numerical representations will be vectors.
So the next question to ask is in what way will vectors be useful?
Now this is more just a preview for the rest of the course, since you'll be exposed to more examples
in more detail as we go along.
But I want to give you at least some idea of the utility of vectors, so let's return to this idea of
spam detection.
Suppose that there is some way to map each email to a vector such that all the emails which are spam,
fall into one cloud of points.
Well, all the emails, which are not spam, fall into a different cloud of points.
At this point, I won't say exactly how we would do this, but suppose that we could.
If we could do this, then it would be very easy to separate spam from non spam emails.
As you can see, we can simply draw a line between the two clouds of points.
Suppose that we have a new email now who's spam status is unknown.
We can ask the question, is this email spam or not?
And intuitively, you know that this boils down to a much simpler question which side of the line is?
The point on that is if we map this new email to a vector, we can determine which side of the line
it falls on.
And if it's on the spam side, we say it's spam.
Otherwise, we say it's not spam.
Now, let's think back to our previous example, where I asked you to think about how to detect spam
based on raw text, as you saw.
This could involve many rules which translate into many if statements.
And ultimately, it ends up being very complicated, since it's not clear what to do when the rules
conflict.
On the other hand, if you know your high school geometry, then figuring out which side of the line
the point falls on should be relatively easy.
Now it does remain to be seen whether or not we can actually convert text into vectors in such a manner.
So for now, you just have to trust that we can.
Here's another thing we can do with vectors.
Suppose that instead of doing spam detection, we simply have a large collection of documents that we
want to organize.
But this collection of documents is way too large to read each one one by one.
So we would like to have some automated way to categorize these documents.
You can imagine that this would be useful for businesses who have to deal with large quantities of documents.
So you might recognize this problem as the problem we call clustering.
Well, suppose that we map our documents to vectors and then we plot each vector on a grid like so again,
we find that certain documents tend to fall into the same clusters as each other while being far away
from other such clusters.
In other words, again, it is the case that turning text into vectors has made our problem very easy.
Now, at this point, we're not going to discuss the specific algorithms we use for clustering.
But rest assured, many effective clustering techniques exist.
But again, we should maintain caution because it still remains to be seen whether or not we can actually
convert text into such nice looking vectors that have the properties I've just described.
Of course, in reality, things are not so nice, but in practice you'll see that the methods we will
use tend to work pretty well anyway.
What we do not want is something like this where all the spam and non spam emails simply overlap.
In this case, we can see that there is no way to draw a line that can easily separate the spam and
non spam emails.
So one might say our objective in converting text into vectors will be to do so intelligently such that
we can avoid the situation.
In other words, we don't just want any set of vectors, but we want a useful mapping from text to vector,
which will make things easier for any subsequent machine learning technique that we want to apply.
So in the coming lectures, we will be exploring methods to convert text into vectors, and you'll see
that we can apply various strategies to give us useful vectors.