So in this lecture, we will be introducing the next section of this course, which is on vector models
in an LP.
So let's think about why this section exists.
Well, this class is all about the study of language and how machine learning can be applied to it.
We know that machine learning is essentially applied math.
We also know that math typically works on numbers.
So this is the main issue.
Language is not represented by numbers.
Language is represented by words and characters, which are discrete, categorical objects.
So this section essentially answers the question How can we convert language into a numerical representation
so that we can apply machine learning to language as a side effect of this?
This section will also cover textbook processing, which is what must be done in the code when you're
working with text to actually get the text into a numerical format.
OK, so now that we know the fundamental purpose of this section, let's go through an outline of what
we will discuss.
We'll start with just some basic definitions in NLP.
Specifically, you'll learn about terms like token, character and vocabulary.
Many people already know what these words mean, but some do not.
So just for completion sake, we will discuss what all of these words mean.
Once we understand our basic definitions, the next step will be to answer the question What is a vector?
Now, I'm sure you all learned about vectors in high school mathematics, so this should just be a review.
We'll also discuss why the concept of vectors is useful in NLP, and I'll give you a little preview
of how they will be used throughout the course.
Once we understand the concept of vectors, the next step will be to discuss various techniques that
will be used during the process of converting text into vectors.
So, for example, tokenization stop words stemming and limitation.
So after we understand some of the basic text processing techniques, well, then look at the simplest
way to convert text into vectors, which is by simple counting.
We'll also look at how this can be applied to do machine learning.
At this point, we won't study any machine learning techniques in depth.
But this is just to give you a quick preview of how effective simple counting can be.
Once we learn how to turn text into victories by counting, well, then look at some of the limitations
of this method.
We'll learn that one way to overcome these limitations is by using a technique called TFI Taf, which
stands for a term frequency inverse document frequency.
We'll also look at how TF IDF can be applied in practice by building a simple recommendation system
as an advanced exercise.
We'll also look at how to implement TFI IDF from scratch.
For those advanced students who really want to understand how things work, this exercise is strongly
recommended.
Otherwise, it's OK to skip.
In the final portion of this section, we will then look at some of the more recent ways that researchers
have been using to create word vectors, such as word and glove.
Now for this part, we will only be looking at basic intuition since the mathematics are quite involved.
In fact, it would take an entire course to cover these subjects in depth, which I've already done.
So there's no need to repeat that, and you could just think of this more like a survey of those more
advanced techniques.