So in this lecture, we'll be starting the next section of this course, which is on lean semantic analysis,
also known as lean semantic indexing.
This is another algorithm with a key role in the field of NLP.
Now, when people introduced this topic, they usually start with two related concepts Tsunami and Palesa
Me.
Tsunami is when you have multiple words, which means similar things.
For instance, a run in sprints Palesa me is when you have one word, which means multiple things.
For instance, consider the word bank.
A bank is a financial institution.
But when we say River Bank, we do not mean a financial institution in a river.
So this instance of bank means something else.
Another example is the word pupil, a pupil is a part of the eye, but a pupil can also refer to a student.
You are a pupil of this course.
So how do these concepts enter in NLP?
Well, they are seen as problematic.
Suppose, for example, we build a recommender system using a document termed Matrix, as we've seen.
Now suppose that the user searches for the word to run.
We know that running is the same as sprinting.
So documents about sprinting should also probably be relevant.
However, since the user didn't enter the word sprints, they will only get back documents containing
the word run.
How about polygamy?
Well, suppose that the user is an ophthalmologist doing research on the pupil of the eye, so they
search for the word pupil.
But then they get back documents about students and how taking handwritten notes is superior to downloading
slides for students.
Of course, those documents are not what the user was searching for.
Since the user was searching for documents about the eye, lean semantic analysis seeks to solve both
of these issues.
Let's go through a quick outline for this section.
We'll begin by explaining the intuition behind this video, which is the machine learning technique
underlying lean semantic analysis.
Once we understand as we do, we can then discuss how it is applied to an LP and why it's useful in
this context.
Once we've done that, we'll look at some code which demonstrates how SVT is useful.
Now achieved is an extremely versatile technique.
In fact, SVOD can be used for many of the applications we've already discussed.
As mentioned, it can help improve recommenders or search engines.
In addition, it can also help to improve classifiers.
So, for example, detecting spam or sentiment as you've seen, it can also be used for text, summarization
and topic modeling.
At the end of this section, applying as ready to these other applications will be mentioned as exercises
for you to complete.
So it's important to note that latent semantic analysis is yet another topic, or it's actually just
a machine learning model being applied to NLP.
In other words, behind the scenes, all we are doing is making use of a well known machine learning
technique, specifically as read and applying that to a term document matrix as we normally do.
So it's actually nothing new if you've already learned about S.V. or PCA in the past.
This is just like naive bays and just like logistic regression.
And just like all of the other techniques we've learned by studying these techniques, you're not only
improving your skills in NLP, but all of machine learning.
Since any of these techniques can be applied in other fields like computer vision or bioinformatics.