OK, so in this lecture, we will be introducing the next section of this course, which is all about
Markov models.
So what are Markov models and why should you learn about these things?
Well, Markov models are essentially everywhere.
Not only are they a staple in NLP, they can also be found in finance as they form the basis for the
famous black holes formula.
Markov models also act as the backbone of the Markov decision process, which is the framework that
we use to do reinforcement learning.
Furthermore, Markov models also act as the backbone of the Hidden Markov model, which has been very
successful in applications like speech recognition and computational biology.
Another major application of the Markov model is Markov Chain Monte Carlo, also known as EMC, which
is a very powerful technique used in Bayesian machine learning, physics and more.
OK, so now that you know why the Markov model is worth learning about.
Let's discuss what we will look at in this section.
We'll start by defining the mark of property.
Note that this involves knowledge of probability.
So if you are not comfortable with this subject, you may want to review it on your own.
The next step will be to look at what a markup model is and how we define one.
We'll look at the relevant mathematical objects that make up a markup model, such as the state transaction
matrix.
We'll also look at how to train a Markov model.
As you recall, we are doing machine learning and thus training or learning is a key part of this process.
Once we understand the theory behind Markov models, we will then look at how to apply Markov models
to do an LP tasks.
These include how to build a tax classifier using Bayes rule and how to generate your own tax to be
more specific.
You'll learn how to generate poetry by creating a Markov model based on an author's existing poems.
The reason that these two applications are interesting is because they are two fundamentally different
ways to apply machine learning.
In one case, we are doing supervised learning, which involves trying to predict a target given a data
set of inputs and labels.
In the other case, we're using a machine learning model to create new things which did not exist before.
This is an unsupervised task because it does not require any labels, only input text.