Everyone and welcome back to this class.
This lecture is designed to answer some common questions I get about how difficult this course is and
where it fits in terms of academia, the professional world and its practical applicability.
So rather than answering this question individually, over and over again, it seems like a better idea
to make a lecture about it and answer everybody at the same time.
A lot of people will become confused because this is their first machine learning course, and they
just don't know where it fits in the grand scheme of things.
So is this course academic or practical?
And is it for beginners or experts?
Is it fast paced or slow paced?
I'm going to answer the second question first, is this course for beginners or experts?
Interestingly, the problem with this question is a problem for machine learning, too.
Let me explain why this question is itself problematic.
One of the challenges in natural language processing is that of ambiguity when someone says some words.
The problem is those words can mean multiple things, and it just so happens that I have some courses
on an LP.
So this is a problem I think about often.
So how does that apply here?
Well, when we use the words beginner and expert, we have to ask a beginner from whose perspective.
An expert from whose perspective for some of you, you might think that a beginner is a random person.
You take off the street who knows nothing, but even that is ambiguous.
There are people out there who do not even really know how to use a computer beyond turning it on and
checking their email and browsing the web.
There are people out there who are pros at Microsoft Office, so they know how to use a computer and
make a Microsoft Word document or a spreadsheet, but they've never done any programming before.
There are people out there who are very good developers and have lots of programming experience, but
they have no idea what machine learning is.
So all of these people are beginners in some way.
But each of them would require a vastly different amount of work to prepare for this course.
It's the same situation with experts.
Most people consider anyone who knows more than them to be an expert.
But of course, how much you know and how much some other person knows is relative.
If you know how to plug and play into psyche, you learn and nothing else.
You might seem like an expert to someone who only knows excel, but you'll seem like a beginner to someone
who's read bishops.
Pattern recognition and machine learning book.
So just to finalize our answer to this question, is this course for beginners or experts?
This question is misinformed to begin with because beginners and experts are not well defined terms.
Instead, we have to go back to what is the specific skill set you need before starting this course.
And of course, that's defined by the prerequisites which are listed in the course description.
So precision is the key here.
We want to know what is the precise list of skills that I need before taking this course.
This is much better than ambiguous terms like beginner and expert.
Another matter is that people are going to measure their own skills differently.
That's a much tougher issue to deal with.
Sure, I can say you need to know Python coding before starting this course, but you can be sure that
there's a big difference between someone who just learned Helloworld and for loops versus someone working
at a big financial institution.
Now, even in those big financial institutions, there could be big gaps in performance, even within
your own team.
There can be big gaps in performance.
I've seen guys who have been working for 15 years do worse than guys who've been working less than one
year.
So even if I say you need to know python coding, there can be a lot of variation in skill among those
who consider themselves good at it.
A similar situation happens with deep learning.
I can say you need to know how neural networks work, but there's a big difference between someone who
can code them from scratch and has experience using them versus someone who just watched some online
animations for a few hours.
If you had to bet on who knows neural networks better, who would you pick?
So to really answer this question, we have two turns of the course itself.
If I say you need to know Python coding and you claim that you do, but you don't understand some Python
code in this course, then simply put, your Python coding skills are not yet up to par.
So you have to improve your Python coding skills by asking questions on the Q&A and taking the appropriate
steps to cover your gaps.
Quite often, I get people who are just too confident in their own abilities, when unfortunately the
reality is there is a huge discrepancy between what they think they know and what they actually know.
What's funny is these people sometimes say this course requires the IMF.
I assure you, there's absolutely no math in any of my courses that requires a Ph.D. or that only a
Ph.D. can understand.
My courses contain only undergraduate math at most.
Sometimes people even claim to have a Ph.D. and then claim that it's still too hard, which is funny
because that essentially invalidates your entire Ph.D.
The entire point of a Ph.D. is to turn you into an independent researcher.
And so if you're supposed to be at the level of doing independent research, but you can't understand
some undergraduate math, well, that's not very good.
Essentially, what it boils down to is people are really trying to say everything I don't understand
is not important.
Everything I do understand is important and therefore everything you say that I don't understand, I
don't need to know.
You can see why this thinking is dangerous.
If you truly believe that everything you do not know is not important, then you'll never learn anything
new.
Now this brings us to our third question, is this course fast paced or slow paced?
And I think you can see where I'm going with this.
This is yet another example of ambiguity fast pace to whom, whether it's fast pace or not, depends
on which person is taking the course.
So if someone is well prepared, then things will be just right.
If someone already knows most of these topics, then it will probably be too slow.
If someone chose not to prepare for the course or they have an inadequate understanding of the prerequisites,
then it's going to be too fast.
So if I expected you to know something and you didn't, then you have to look it up, which is going
to make the course feel like it's going too fast.
So here is the real test.
If I say something depends on calculus of probability and you claim to know calculus and probability,
but you don't understand the thing that depends on calculus and probability, you have to ask yourself,
honestly, do I really understand calculus and probability?
Or did the instructor invent his own kind of calculus and probability that I don't yet understand?
Well, of course, that's not very likely.
It's most likely that you thought you understood calculus and probability, but you really don't.
Now onto our initial question, what kind, of course, is this?
Is it academic?
Is it for professionals?
Is it practical?
So to understand this, you have to understand where machine learning is usually taught and how it is
usually taught.
Firstly, machine learning is usually taught in upper year computer science and engineering programs
in university and college.
So these are students who already took calculus one to three linear algebra, differential equations,
probability and statistics, discrete math programming and possibly more all in their first two or three
years of college.
So by the time they get to third and fourth year, they know all this stuff and they're ready to apply
it.
So a typical academic course might cover a whole library of machine learning algorithms, so you might
learn closest neighbor logistic regression K means clustering PCA and so on, all in the same course.
Typically, you'd cover one algorithm per lecture, which is about two hours overall.
You might have something like 12 or maybe 20 of those lectures in a term and by cover, I mean, mostly
via math, geometry and derivations.
These courses don't typically have you at your computer doing programming work.
If you do end up doing programming, it's usually part of a lab assignment, which is actually just
a very small part of the course compared to the rest.
So it's mostly theoretical dealing with equations, solving equations, reasoning about equations.
Those equations typically involve probabilities and information theory.
So you have two hours to go through the theory for one algorithm.
And then if there's coding involved, which there usually isn't, you do it on your own time.
The lectures themselves do not involve any coding.
But there's an even bigger problem with these academic courses.
And what is that?
What happens is that the grades turn out to be very low.
Out of 100 percent, the average might be around 25 percent or up to 40 percent.
Now you might think, wow, so everyone fails.
And of course, that's not the case since everyone gets shifted based on the average.
So what is the real problem here?
A lot of people ask me, Well, why don't you cover every machine learning algorithm in one big mass,
of course, and just call it machine learning?
Well, the answer is that if you tried to do too much, you only end up understanding 25 percent of
the material, which I think is clearly demonstrated here.
And as they say, the proof is in the pudding.
If you look at these courses where students are trying to digest 10 different algorithms at the same
time, what is the result?
The result is they don't understand any of them.
If you try to understand too many things, you end up understanding nothing.
I take the perspective that if you're going to take a course, you should aim to understand 100 percent
of the material.
Each course is broken up into appropriate levels based on how difficult they are and by topic.
So for example, you can typically take Calculus one, which focuses on differential calculus calculus
two, which focuses on integral calculus and calculus three, which focuses on multivariable calculus.
You would never learn all of these in just one huge course called math, because that's just too much
stuff to understand all at once.
I want you to understand 100 percent of the material, even 50 percent or 75 percent is not good enough,
in my opinion, if you only understand 75 percent of the material.
And the next course depends on the 25 percent that you didn't understand.
Well, then you're not going to make it to the next level.
Now, let's look at the opposite end of the spectrum.
What most people might consider to be a practical course.
I don't consider these to be practical courses, and I'll explain why later.
But I use the word practical here in quotes, since that's what some of the more beginner students will
refer to the mess.
So how does a practical course typically proceed?
Well, first, you are never really taught how the algorithms work other than at a very high level.
You might learn how they work, but we have analogies and metaphors.
So, for example, suppose you're learning about gradient descent.
Well, the instructor might relate this to a ball rolling down a hill.
Now, some beginner students, they might conclude that I understand a ball rolling down a hill.
Therefore, I understand gradient descent, but they neglect to understand how to take derivatives.
They neglect to understand how to choose a good learning rate.
And most importantly, they neglect how to implement gradient descent in code.
So it's totally wrong to assume that just because you can visualize a ball rolling down a hill that
you understand gradient descent, why?
Well, the problem is, everybody can imagine a ball rolling down a hill, your grandparents who can't
even turn on a computer, can't understand a ball rolling down a hill.
So don't equate understanding an analogy to understanding the real topic.
So what's the next thing these practical courses do, the next thing they usually teach you is how to
plug into an API or, in other words, make use of a library, someone else wrote.
This is also not ideal.
Typically, this involves just two or three lines of code.
You have to wonder if machine learning just boils down to two or three lines of code.
Why is it such a big deal these days?
Why did it take decades for people to realize its usefulness?
And of course, the answer is that there's much more to machine learning than just these two or three
lines of code.
When you use a deep learning library like Keros, you're not doing deep learning.
You were using a deep learning interface.
You're simply programming.
It's more accurate to say I can program with deep learning libraries rather than saying I actually know
deep learning.
But some of you might say, well, that's all I really want to do.
All I need is an API and for the API to give me an answer.
I don't care about how it works.
But then here is where you run into trouble.
Recall that in addition to all data is the same.
We know that all machine learning interfaces are the same.
So the question becomes Which API do you choose if they're all the same?
How can you make an informed choice about which one to use?
Well, the reality is, if you don't know how these things work, then you can't answer this question.
This is why some of these so-called practical courses can cover 20 algorithms in the same course.
It's because they never actually talk about how the algorithms work.
Since the API is the same every time.
It's very easy to just do the same thing 20 times and then not bother with the details.
So this gives people a lot of confidence that they were able to do the same thing 20 times.
But it's dangerous thinking because you don't realize you really only learned one thing.
You just repeated it 20 times.
So how about when it comes to the professional world?
Sometimes people tend to believe that in your profession, you'll be leaning more toward the practical
end of the spectrum, where you don't really need to know how a machine learning model works.
You just have to know its API.
But what it really comes down to is what your job actually is.
If you're a regular everyday programmer and you just want to see how a machine learning model might
perform on your data, then you're going to use a pre-built library.
Most likely.
That's just the two or three lines of code I referenced earlier.
However, that does not mean you don't still have to conform to best practices, make sure you're not
mixing your test set with your train set.
Make sure you're doing proper validation and so on.
And when it comes to tuning your model, then you'll have to be able to dig down into the details and
see which hyper parameters are possible to change and which are likely to have the most impact.
So only knowing how to use an API is only going to get you past the first stage.
You can plug in your data and at least your code is the right syntax and doesn't crash.
But of course, your code not crashing doesn't necessarily mean it's right.
Back on the other end of the spectrum, you have programmers and scientists whose sole job it is to
do machine learning.
These guys are machine learning experts.
They know the math inside and out and have a full computer science education.
And so to even be able to communicate your ideas effectively, you have to be comfortable with the theory
and not just how to plug into an API with two lines of code.
These scientists are building custom models, really digging down into the math to not only use but
invent new things and to be able to invent new things.
You really need to know the math very well.
And so if you're on a team like this, you can't just know how to write two lines of code.
Your skill set has to be far beyond that.
So your theoretical background has to be very solid if you want to be able to communicate effectively
with other scientists.
Now, a lot of people say, well, I don't ever want to be a scientist, so I don't really care how
these algorithms work.
Unfortunately, that's a very bad attitude to have in the workplace.
Suppose, for example, I am running a business now, an employee of mine.
He wants to show off and say he's learning about machine learning.
He says, Oh, I'm taking this machine learning course online.
Therefore, I'll become a machine learning master.
But his approach to machine learning is to learn as little as possible.
He says he doesn't care about the math.
He only wants the practical stuff.
He only wants to know the bare minimum.
He tells me he doesn't care how it works.
He's just going to plug into some API because that's what his buddy told him he could do.
As a business owner, I really don't want this guy working for me.
I run a business that sells something and the life of the company itself and the livelihoods of the
employees all depend on our products being the best they can possibly be.
So if someone comes to me and says I don't care about being the best.
I just care about doing the bare minimum.
My response is I don't want that guy on my team.
In fact, he should be fired.
So be careful if this is your approach to machine learning.
No business owner wants to hear that your approach to the business is to do the bare minimum and to
put off the math because you think it's too hard.
That's a scary prospect for a business owner.
Now, if we go back to our original question, maybe you've realized we have an answer to yet.
What kind, of course, is this?
Well, you can think of it as a marriage between these approaches.
We want to cover all the math and derivations because that's very important.
If you want to be able to communicate effectively with other scientists to be able to debug your code
effectively and to be able to invent new approaches, this is also what we mean by machine learning.
We mean really knowing machine learning and not just programming with an API.
So the theoretical background is very important.
At the same time, we're not going to introduce 20 different algorithms in the same course because unlike
a typical academic course where you can pass just by understanding 25 percent of the material in this
course, it's structured in a way that if you have any missing gaps, it's going to be impossible for
you to move on to the next stage.
But academic courses don't contain much coding, if at all.
So in this course, every algorithm we learn about will be implemented in code.
My motto is if you can't implement it, then you don't understand it.
You can pretend to understand it.
And when you talk about it, you might sound like you understand it.
But implementation is the true test.
It's like a math exam.
Rather than writing an essay when you write an essay, you might be able to fool the teacher into thinking
you understand something.
But in a math exam, either your answers are right or your answers are wrong.
In this context, either your code works or it doesn't.
It's a very clear cut.
When it comes to practicality, we cover that, too.
We demonstrate our algorithms both on real world data like text and images.
And we demonstrate our algorithms on two dimensional data sets so that you can visualize what an algorithm
is doing.
Both of these are important.
Texts and images are quite possibly the most practical data there is.
Some billion dollar companies exist and thrive because of those capabilities.
2D data sets are critical as well because as humans, that's the only thing we can see.
But at the same time, we don't try to pull the wool over your eyes and pretend like you need 20 different
data sets to practice on.
Instead, we learn something much more valuable.
How to use an algorithm on an infinite number of data sets because we are able to abstract the idea
that all data is the same.
Remember that all the computer sees is a list of numbers.
It doesn't know that those numbers represent height measurements or an image pixel, and of course,
by experiencing both ends of the spectrum, both the theory and the implementation.
And then finally, plugging data into your model that's going to make you very well-rounded professionally.
To conclude this lecture, I want to pose a final question, how does all this knowledge help you?
Why does understanding the approach of this course make you better at machine learning in general?
Well, my hope is that this gives you a very high level bird's eye view of the landscape.
You can see the different approaches of people from different backgrounds.
Academic people typically don't see what other non-academic people are doing, and those people who
hate math typically don't see what people who love math are doing.
So with this bird's eye view, you can see everybody.
If you're learning machine learning, it helps you understand who am I competing with when I apply for
a job?
And what kinds of jobs are out there?
And what would be adequate preparation?
If you're hiring for a machine learning team, it helps you understand the different backgrounds of
individuals.
You have to compare to one another to understand the strengths and weaknesses and what they bring to
the table.