So in this lecture, we are going to briefly discuss a concept known as the bag of words representation.
Well, begin by reiterating that text and language more generally are sequential.
That is, text is made up of sequences of words, and those sequences give our words meaning.
For example, if you take any sentence, but then a randomize.
Other words, you would completely change the meaning of that sentence or more likely, just render
the sentence incomprehensible.
Now, despite this, many NLP approaches simply do not consider the ordering of words in a sentence.
So suppose that we come up with a numerical representation of text that does not consider word ordering.
When we do this, we call this a bag of words representation.
OK, so let's think of an example of where we would lose information by using a bag of words representation.
Consider the phrase dog toy.
The meaning of this is that we have a toy, which is for dogs.
Now consider the phrase toy dog.
The meaning of this is that we have a toy, which looks like a dog.
So clearly these two things are different.
And clearly, any representation that ignored the ordering of these words would not be able to differentiate
between these two entities.
That's why we can say that we are losing information.
Now, despite the fact that the bag of words approach seems to be quite limited, it is in fact widely
used in this course.
What you'll find is that for the most part, vector models and classical machine learning make use of
the big of words approach.
On the other hand, probabilistic models and deep learning do not make use of the bag of words approach.
But keep in mind that this is just a very rough categorization.
We do have probabilistic models that use bag of words, and we do have deep learning models that also
use a bag of words.
So it's not exact, but I hope that as you go through the course, you will look for these patterns
and recognize when bag of words is being used and when it isn't.
And another thing to mention is that although the bag of words approach does seem to be limited, it
is pretty accurate in many cases.