Everyone and welcome back to this class.
In this lecture, I'm going to answer a question that I get pretty much multiple times per day.
And that is what order should I take your courses in?
This is a great question and it's something that even I have to think about sometimes.
What is the best way to learn all this stuff?
So I think that what you'll find is this ordering makes a lot of sense in that we're always building
on skills we learned before.
So it's just like learning calculus.
First, you learn calculus one, then you learn Calculus two, which is kind of the opposite of Calculus
one.
Then you learn Calculus three, which expands on the ideas from both Calculus one and two.
And then from there, you can go on to even more advanced topics like probability.
So this is a process based on skill building.
Now, it's important to keep in mind that this lecture is not just about my courses in how to navigate
them, although that is the question I'm answering, since that's the question I get multiple times
per day.
But this is also a lecture on how different machine learning topics are related and what depends on
what else.
So even if you end up not taking any of my courses, it doesn't change the fact that these topics are
still related to each other in these ways.
So in some sense, this lecture isn't really about my courses at all.
It's a general lecture about how different types of machine learning models are related and in what
order you might want to learn them from a skill building perspective.
Now, I want to show you this stuff in a visual way.
I've actually set up a webpage where you can go to and look at the chart I made for yourself.
So what I want you to do is go to this website.
Deep learning courses.
Dot com.
Now I want you to click on the catalog link.
And from there, you want to click on the Click Here link.
So this brings us to the court's order webpage.
So at the top, you have sort of this linear chart, right, it's just one course after the other,
after the other.
But this is not really what I want to look at in this lecture.
This linear chart gives you a basic overview, but it's not ideal.
The reason is the relationships between courses are way more complex than this.
Sometimes one course gives you the skills you need for two other courses.
Sometimes there will be a course that's so complex that it depends on multiple different courses.
So this is the perfect opportunity to use a graph.
So if you scroll up at the top, you can use this link to just jump down to the bottom.
And we have this graph where you can see the dependencies between each course.
So what I'm going to do is I'm going to go through each link in this graph and explain to you why it
exists and why it's important.
So let's look at the ask you.
Of course, first since that has no links, so it might show up in the middle so you can just drag it
out to the side.
That's the coolest thing about this graph, because you can just pick stuff up and drag it around.
It's very interactive, so ESCO has no legs.
This isn't a prerequisite to any other course, nor is any other course a prerequisite to it.
It is standalone.
You can take it at any time.
Next, you'll notice that there are courses that have no links going into them.
So, for example, the Nampai course, this is because none of my other courses are a prerequisite to
this course.
This course was just designed to give you a basic understanding of the syntax and tools we use in data
science and machine learning.
The goal is actually to not do any machine learning in this course.
Believe me, it was actually quite difficult to accomplish that because it's so easy to use something
in machine learning as an example.
It was very hard to avoid talking about it.
You'll also notice that there are some courses with no outgoing edges.
You can think of these as the most advanced courses.
They are at the top of the ladder.
So, for example, this one right here, of course, that could change in the future as I create more
courses in which case I will update this lecture.
So let's start with what's coming out of Nampai.
You'll see that there are two links, one goes to linear regression and one goes to Bayesian machine
learning, a b testing and by the way, you can zoom into this graph.
So that's what I'm going to do right now.
So here's linear regression, here's a Bayesian machine learning.
So while Nampai goes to both of these courses, I actually strongly recommend linear regression as the
next step.
Bayesian machine learning doesn't depend on any other courses per se, but it's still more advanced.
It requires a more advanced understanding of probability and of real-world engineering problems like
optimizing the click through rate of a link on your website.
So linear regression would be my number one recommendation for your foray into machine learning.
Now, after linear regression, we have logistic regression.
So why do we go from linear regression to a logistic regression?
Well, the basic idea is both of these are linear models.
In other words, the model is of the form Y equals m x plus B, which is a line.
The difference is that linear regression does regression while logistic regression does classification.
We want to learn classification because that's one of the essential tasks of a neural network, which
is what comes after.
And you might notice that linear regression also has two outgoing links, so there's one two logistic
regression and one to reinforcement learning.
This is because my reinforcement learning class makes use of linear regression, so naturally, if you
don't understand linear regression, you won't understand that part.
But again, this is a much weaker link of and going from linear regression to logistic regression.
This is because reinforcement learning is actually conceptually more difficult.
It doesn't depend on the content of any other courses directly other than what we've shown.
But the ideas behind it are more advanced.
You really want to have some experience with both supervised and unsupervised learning before jumping
into reinforcement learning.
It helps to build the proper perspective on the subject.
We will get back to reinforcement learning later, but for now, let's go along the deep learning path.
So in the deep learning path, we last discussed logistic regression.
This is also known as a neuron.
This is very important because in order to build a neural network, we must first know how to build
a neuron.
A neural network is just a bunch of neurons stuck together.
So this leads us to deep learning in Python part one, which is the bright green logo.
This course teaches you how neural networks work and basic fundamental operations like back propagation.
Once you know the basics of deep learning, you can move on to deep learning part two or modern deep
learning in Python.
This is the same image, but in a darker color to signify that it's a continuation of the previous course,
as the title suggests, this course is all about modern developments in deep learning.
So we go over ways we've improved basic back propagation like momentum and adaptive learning rate techniques
like arms, prop and atom.
We talk about modern regularization techniques like dropout and batch normalization, and most importantly,
we look at modern deep learning libraries like the piano and TensorFlow.
We also take a look at Teraz PyTorch and snare and see A.K.
You'll see that once you know the fundamentals, things don't change that much from one library to the
next, it's all the same concept.
Now, learning about these modern library sets the stage for the next level, which is applying deep
learning to certain special data formats, the two fundamental types of data are text and images.
These are special because they have unique structural properties, and we can make use of our knowledge
of those properties in order to build better neural networks.
For example, we know that text is made up of sentences and sentences are sequences of words, so sequential
modeling is important.
We know that images are two dimensional objects, but importantly, each pixel is very likely to be
the same as nearby pixels.
For example, if you pick a random pixel on an image of a red car, if that pixel is red, then most
likely the pixels around it are also going to be red.
There is also no question about the practical applicability of text and images.
The internet is made up of text, so by doing machine learning on text, you're learning techniques
that essentially allow you to write models of the entire world's knowledge.
Images are also everywhere.
Snapchat, Instagram, Facebook Self-driving cars home security systems Knowing how to deal with images
is super important, and billion dollar companies exist because they are good at it.
So that brings us to the next course in the deep learning series convolutional neural networks or deep
learning in Python Part three.
This is all about how to incorporate convolution into neural networks and what convolution is in the
first place, and how that helps us make use of the structure of images.
Let's scroll up a bit.
So after convolutional neural networks, we take a detour into unsupervised deep learning, unsupervised
deep learning isn't very popular with beginners, but it should be.
If you look at the experts of deep learning like Yann LeCun, Joshua Bengio and Geoffrey Hinton, they
all talk about how important unsupervised deep learning is.
This course introduces us to several new ideas.
First, the idea of latent variables or latent causes.
These are factors that you don't directly observe in your data, but that you can learn about by studying
the structure of your data.
Second, modern techniques for data visualization like tease me this will be very important since,
as I discussed earlier, along with images, text is a really important data format.
In order to visualize text in later courses, we will be making use of Disney.
So it's good to know what it is and why it's useful.
Third, unsupervised pre-training unsupervised pre-training is the foundation of modern ideas like transfer
learning, and we make use of it both in NLP and with convolutional neural networks.
Fourth, the vanishing gradient problem, we demonstrate the vanishing gradient problem directly, so
you understand where it comes from, vanishing gradients are especially important in the context of
recurrent neural networks.
And it just so happens that recurrent neural networks is the next course in the deep learning series
Deep Learning and Python Part five.
So recall that our two fundamentally interesting data types are text and images.
We've looked at images, and in this course, we start to look at texts.
Why might that be?
Well, recurrent neural networks specialize in modeling sequences, and text is just the sequence of
words.
So it's the best type of data to use to demonstrate the principles of our own ends.