So in this lecture, we will be introducing the next part of this course, which comprises several sections.
We'll talk about how and why we are transitioning and how what we're about to study differs from what
we have already studied.
So the first major section of this course covered vector models in textbook processing.
This was all about how to represent text numerically so that machine learning models can be applied.
Now, even though we haven't yet covered machine learning in this course, we still saw how vector representations
could be used to.
Document retrieval is one example.
So vector models are useful in their own right, and yet there is a whole subfield of NLP, which does
not make use of these numerical vector representations.
And these are probabilistic models.
So let's discuss the importance of probabilistic models in the context of an LP.
This course will mainly focus on a model called the Markov model.
In fact, the application of Markov models to an LP goes hand in hand with their creation.
So in the early 1800s, Andre Markov, whom the Markov model is named after, applied these models to
analyze the patterns of vowels and consonants.
In 1948, another pioneer of our field, Claude Shannon, used Markov models to generate text, which
he introduced in his famous paper that laid the foundations of information theory and digital communications.
In more recent times, the hidden Markov model has been applied in areas such as speech recognition
and biological sequence analysis.
It's a little known fact that biological sequences are closely related to NLP.
But if you think about it just a little bit, you'll see how it begins to make sense.
DNA is made up of an alphabet of just four letters ATC and G.
These letters can be combined to form words which are then combined to form instructions about what
our body should build with its resources.
In other words, DNA is just another language.
It happens to be a language created by nature instead of by humans.
But it is a language nonetheless.
In fact, what you will find is that developments in NLP tend to be closely followed by developments
in genomics.
Since many of the same techniques, like CNN's ANA and Transformers, can be directly applied to biological
sequences.
The latest example of this is the latest version of Alpha Fold released by DeepMind in 2020, with the
paper published in Nature in 2021.
Later on in your NLP studies, you'll see that even Transformers will implement many of the ideas we
will learn about in the coming sessions.
So one final example is one I really like, which is Google's page rank method, as you recall.
Google is one of the largest and most powerful tech companies in the world.
The reason Google became as large as it is is largely due to the page rank method.
This is back when search engines were not as good as they are today.
And so Google's method was able to do far better than anything that existed.
So what is Page Rank?
Well, suppose that you built a markup model out of all the web pages on the internet.
The page rank score for each web page is the probability that you'd land on that page after randomly
browsing the web for an infinite amount of time.
So let this be a lesson to you if you want to start one of the largest and most powerful Billion-Dollar
businesses in the world.
All you really need is a simple Markov model.
So the next part of this course will be made up of multiple sections in the first section, we'll study
Markov models and what it means to build an Engram language model.
We'll also see a few applications.
One application, which will get its own section is the article spinner.
This is the kind of tool that is used by Black Hat SEO marketers that don't want to write legitimate
content, but instead spend content written by others.
Another section we'll look at an application called Cipher Decryption, also known as code breaking.
You'll see how language models can be applied to decrypt ciphers, which is an important application
in espionage and warfare.
After these sections will then move on to machine learning and deep learning, where you'll get a chance
to see how both vector models and probability models can be applied further.