So in this lecture, we will summarize everything we learned in this section.
The section was about the Markov model and how it can be used for NLP.
We started this section by learning about the markoff property, although this assumption seems to be
very restrictive.
We also learned that it has many applications in fields such as finance, reinforcement learning and
speech recognition.
So clearly, although it is very restrictive, it is still very useful.
The next step was to discuss how a market model is represented mathematically, which also helps us
understand how to represent one in Python.
The next step was to look at how to apply the markup model for applications in NLP.
Now, you may not yet realize this, but what you learned in this section is just the start of a potentially
long journey.
In fact, what you have learned does not end here.
We learned in this section that the basic idea is that we want to predict the future from the past.
That is, we want to predict X of T given X of T minus one x of T minus two and so forth.
With Markov models, it turns out that we cut this off that X of T minus one.
However, this need not be the case.
One thing we can do is to simply add more terms when we use one pass value.
It's called First Order Markov.
When we use to pass values, it's called Second Order, Markov and so forth.
Despite the simplicity of this idea, you will find that it appears again and again in machine learning,
even at the most advanced levels.
One example of this is with a time series analysis.
A common way to build a forecasting model is to try and learn a function that maps previous values to
the next value.
This can be a linear function, a neural network, a random forest and so forth.
Another direction we can go is to use more complex architectures.
We'll keep working with text, but we'll use more powerful models that can yield more complex dependencies
between the pass values and the next value.
One example of this is with recurrent neural networks, although that sounds complicated when using
Arden's for NLP, you can do useful things only by training your model to predict the next value from
previous values.
In fact, the story still doesn't end here.
The most powerful NLP models we have today called Transformers are also trained to do this exact same
thing.
Transformers might sound intimidating, but behind the scenes, all they have been trying to do is predict
the next word from previous words.
So again, although this is a very simple idea, it's also very powerful and it's the foundation for
the current state of the art.