So to understand where clustering and elder fit in the grand scheme of things, it's helpful to look
at some pictures.
Let's start with the simplest thing we could do.
The simplest thing is if we had no clusters or, in other words, no topics, and every document comes
from the same distribution.
We can picture this like a single Gaussian, although do note that we don't typically use Gaussians
to model where it counts.
This is for intuition only, and believe it or not, this is in fact unsupervised learning.
So when you're your grade school teacher, makes a bell curve for your exam grades.
They are actually doing a form of unsupervised learning.
Now, one step up from the simplest thing is to have multiple Gaussians.
This is essentially soft clustering, also known as a mixture model.
In this case, each Gaussian represents a separate cluster or a separate topic.
And the job of our algorithm is to identify the locations of these clusters.
This also tells us which documents belong to which cluster.
Now, I'm skipping a few intermediate steps, but if we add even more complexity, we finally end up
with LDA.
Unfortunately, LDA is an easy to visualize with our current machinery.
So you just have to wait until the next lecture to understand this relationship.
What I can say is that for LDA, each document is a mixture of topics.
And furthermore, each topic is a mixture of words that doesn't really differentiate it from a mixture
model.
But again, the precise details will be in the next lecture.
But understanding that documents are mixtures of topics and topics are mixtures of words will help us
with the rest of this lecture.