Now, we've just looked at the API perspective on LDA, so we understand how it works in terms of inputs
and outputs.
We know what inputs to pass in and what outputs we should expect to get back.
Let's also look at the actual API to understand more about how it works and to prepare you for the coding
lecture.
Let's assume we know how to use a count vector riser to build our data matrix x.
We then create an object of type laten directly allocation as we normally do.
We then call the fit method on our object, passing an X as we normally do.
Note that unlike supervised learning, unsupervised models only take in the input data set.
But there are no targets.
So in the supervised learning, you'll recall that this looks like fit x y where Y is an array of targets,
but we don't have targets, so there is no y array.
Now, the next thing we do is call the Transform method, which gives us back a new matrix called Z
Z represents our documents by topics matrix.
Just like with X, Z is a common letter we use in machine learning to denote a specific thing in particular,
whereas X represents data we observed.
Z represents data we did not observes.
If you study machine learning further, you'll see these referred to as latent variables or hidden variables.
Another difference with unsupervised learning is that we use this transform method instead of predict.
This is just a circuit learning convention, but it makes a lot of sense.
With supervised learning, you are trying to predict a target, but with unsupervised learning you don't
have any targets, so you're just transforming the input into a new variable.
In this case, a vector of unknown topics both transform and predict can be generalized under what we
call inference.
So when we transform data to get Z, we call that inference.
But when we predict to get Y hat, we also call that inference.
So inference is a generic term that can mean either of these, depending on the context.
In addition, note that it's possible to combine these two calls into a single call where we just say
Fit, Transform and we pass index, which gives us back Z.
Now at this point, you might be thinking what happened to the other matrix?
As you recall, I said LDA outputs, two matrices, one with documents by topics and the other with
topics by words.
Well, the topics by words matrix is actually stored as an attribute called components.
So to access this matrix, you simply call model dot components with an underscore.
So let's think about why this makes sense.
The key is that this model can be used to transform any document, even one that didn't exist in the
training set.
So suppose that you've already built your model, but your boss comes to you with new documents tomorrow
called X Test.
Luckily, you don't need to retrain your model, since that would take a lot of work.
You can simply call Model Dot Transform passing in X tests, and this will give you Z Test, which represents
the topics for these new documents.
This is unlike the components matrix, which is more a property of the model instead of data to be transformed.
Think about that if it doesn't make sense.
The model stores its representations of each topic.
They are part of the model.
This does not come from the documents you are transforming.