Everyone and welcome back to this class
this lecture is designed to answer some common questions I get about how difficult this course is and
where it fits in terms of academia the professional world and its practical applicability.
So rather than answering this question individually over and over again it seems like a better idea
to make a lecture about it and answer everybody at the same time a lot of people become confused because
this is their first machine learning course and they just don't know where it fits in the grand scheme
of things.
So is this course academic or practical and is it for beginners or experts.
Is it fast paced or slow paced.
I'm going to answer the second question first.
Is this course for beginners or experts interestingly.
The problem with this question is a problem for machine learning too.
Let me explain why this question is itself problematic.
One of the challenges in natural language processing is that of ambiguity when someone says some words.
The problem is those words can mean multiple things and it just so happens that I have some courses
on it.
So this is a problem I think about often.
So how does that apply here well when we use the words beginner an expert.
We have to ask a beginner from whose perspective an expert from whose perspective for some of you you
might think that a beginner is a random person you take off the street who knows nothing.
But even that is ambiguous.
There are people out there who do not even really know how to use a computer.
Beyond turning it on and checking their email and browsing the web there are people out there who are
pros at Microsoft Office so they know how to use a computer and make a Microsoft Word document or a
spreadsheet.
But they've never done any programming before.
There are people out there who are very good developers and have lots of programming experience but
they have no idea what machine learning is so all of these people are beginners in some way but each
of them would require a vastly different amount of work to prepare for this course.
It's the same situation with experts.
Most people consider anyone who knows more than them to be an expert but of course how much you know
and how much some other person knows is relative.
If you know how to plug and play into psyche you learn and nothing else.
You might seem like an expert to someone who only knows excel but you'll seem like a beginner to someone
who's read Bishop's pattern recognition and machine learning book
so just to finalize our answer to this question is this course for beginners or experts.
This question is misinformed to begin with because beginners and experts are not well-defined terms.
Instead we have to go back to what is the specific skill set you need before starting this course.
And of course that's defined by the prerequisites which are listed in the course description.
So precision is the key here.
We want to know what is the precise list of skills that I need before taking this course.
This is much better than ambiguous terms like beginner and expert
another matter is that people are going to measure their own skills differently.
That's a much tougher issue to deal with.
Sure I can say you need to know Python coding before starting this course but you can be sure that there's
a big difference between someone who just learned hello world and for loops versus someone working at
a big financial institution
now even in those big financial institutions.
There could be big gaps in performance even within your own team.
There can be big gaps in performance.
I've seen guys who've been working for 15 years do worse than guys who've been working less than one
year.
So even if I say you need to know Python coding there can be a lot of variation in skill among those
who consider themselves good at it.
A similar situation happens with deep learning.
I can say you need to know how neural networks work but there's a big difference between someone who
can code them from scratch and has experience using them versus someone who just watch them online animations
for a few hours.
If you had to bet on who knows neuron that works better.
Who would you pick so it's a really answer this question we have to turns off the course itself.
If I say you need to know Python coding and you claim that you do but you don't understand some Python
code in this course then simply put your python coding skills are not yet up to par.
So you have to improve your python coding skills by asking questions on the Q and A and taking the appropriate
steps to cover your gaps
quite often.
I get people who are just too confident in their own abilities when unfortunately the reality is there
is a huge discrepancy between what they think they know and what they actually know.
What's funny is these people sometimes say this course requires the math I assure you there is absolutely
no math in any of my courses.
That requires a PHC or that only AP HD can understand my courses contain only undergraduate math at
most.
Sometimes people even claim to have a PHC and then claim that it's still too hard.
Which is funny because that essentially invalidates your entire HD the entire point of AP HD is to turn
you into an independent researcher.
And so if you're supposed to be at the level of doing independent research but you can't understand
some undergraduate math well that's not very good
essentially what it boils down to is people are really trying to say everything I don't understand is
not important.
Everything I do understand is important and therefore everything you say that I don't understand I don't
need to know.
You can see why this thinking is dangerous.
If you truly believe that everything you do not know is not important then you'll never learn anything
new.
Now this brings us to our third question is this course fast paced or slow paced.
And I think you can see where I'm going with this.
This is yet another example of ambiguity.
Fast pace to whom whether it's fast paced or not depends on which person is taking the course.
So if someone is well-prepared then things will be just right.
If someone already knows most of these topics then it will probably be too slow.
If someone chose not to prepare for the course or they have an inadequate understanding of the prerequisites
then it's going to be too fast.
So if I expected you to know something and you did it then you have to look it up which is going to
make the course feel like it's going too fast
so here is the real test.
If I say something depends on calculus of probability and you claim to know calculus and probability
but you don't understand the thing that depends on calculus and probability.
You have to ask yourself honestly do I really understand calculus and probability or did the instructor
invent his own kind of calculus and probability that I don't yet understand.
Well of course that's not very likely.
It's most likely that you thought you understood calculus and probability but you really don't
now onto our initial question what kind of course is this.
Is it academic.
Is it for professionals.
Is it practical sort of understand this.
You have to understand where machine learning is usually taught in how it is usually taught firstly
machine learning is usually taught in upper year computer science and engineering programs in university
and college.
So these are students who already took Calculus 1 to 3 linear algebra differential equations probability
and statistics discrete math programming and possibly more all in their first two or three years of
college.
So by the time they get to third and fourth year they know all this stuff and they're ready to apply
it.
So a typical academic course might cover a whole library of machine learning algorithms.
So you might learn K nearest neighbor logistic regression K Means clustering PCH and so on all in the
same course typically you'd cover one algorithm per lecture which is about two hours.
Overall you might have something like 12 or maybe 20 of those lectures in a term.
And by cover I mean mostly via math geometry and derivations.
These courses don't typically have you at your computer doing programming work.
If you do end up doing programming it's usually part of a lab assignment which is actually just a very
small part of the course compared to the rest.
So it's mostly theoretical dealing with equations solving equations reasoning about equations those
equations typically involve probabilities and information theory.
So you have two hours to go through the theory for one algorithm and then if there is coding involved
which there usually isn't you do it on your own time.
The lecturers themselves do not involve any coding
but there is an even bigger problem with these academic courses.
And what is that.
What happens is that the grades turn out to be very low out of 100 percent.
The average might be around 25 percent or up to 40 percent.
And you might think wow.
So everyone fails.
And of course that's not the case since everyone gets shifted based on the average.
So what is the real problem here.
A lot of people ask me Well why don't you cover every machine learning algorithm in one big mass of
course and just call it machine learning.
Well the answer is that if you tried to do too much you only end up understanding 25 percent of the
material which I think is clearly demonstrated here.
And as they say the proof is in the pudding.
If you look at these courses where students are trying to digest 10 different algorithms at the same
time what is the result.
The result is they don't understand any of them.
If you try to understand too many things you end up understanding nothing.
I take the perspective that if you're going to take a course you should aim to understand 100 percent
of the material each course is broken up into appropriate levels based on how difficult they are.
And by topic.
So for example you can typically take Calculus 1 which focuses on differential calculus Calculus 2 which
focuses on integral calculus in calculus 3 which focuses on multi variable calculus.
You would never learn all of these in just one huge course called math because that's just too much
stuff to understand all at once.
I want you to understand 100 percent of the material even 50 percent or 75 percent is not good enough
in my opinion.
If you only understand 75 percent of the material and the next course depends on the 25 percent that
you didn't understand well then you're not going to make it to the next level.
Now let's look at the opposite end of the spectrum.
What most people might consider to be a practical course.
I don't consider these to be practical courses and I'll explain why later but I use the word practical
here in quotes since that's what some of the more beginner students will refer to the maths.
So how does a practical course typically proceed.
Well first you were never really taught how the algorithms work other than at a very high level.
You might learn how they work by way of analogies and metaphors.
So for example suppose you're learning about gradient descent while the instructor might relate this
to a ball rolling down a hill.
Now some beginner students they might conclude that I understand a ball rolling down a hill.
Therefore I understand gradient descent but they neglect to understand how to take derivatives.
They neglect to understand how to choose a good learning rate and most importantly they neglect how
to implement gradient descent in code so it's totally wrong to assume that just because you can visualize
a ball rolling down a hill that you understand gradient descent y well the problem is everybody can
imagine a ball rolling down a hill.
Your grandparents who can't even turn on a computer can understand a ball rolling down a hill.
So don't equate understanding an analogy to understanding the real topic.
So what's the next thing these practical courses do.
The next thing they usually teach you is how to plug into an API or in other words make use of a library.
Someone else wrote.
This is also not ideal.
Typically this involves just two or three lines of code.
You have to wonder if machine learning just boils down to two or three lines of code.
Why is it such a big deal these days.
Why did it take decades for people to realize its usefulness.
And of course the answer is that there's much more to machine learning than just these two or three
lines of code.
When you use a deep learning library like carrots you're not doing deep learning.
You are using a deep learning interface you're simply programming it's more accurate to say I can program
with deep learning libraries rather than saying I actually know deep learning
but some of you might say well that's all I really want to do.
All I need is an API and for the API to give me an answer.
I don't care about how it works but then here is where you run into trouble.
Recall in addition to all data is the same.
We know that all machine learning interfaces are the same.
So the question becomes which API do you choose.
If they're all the same how can you make an informed choice about which ones to use.
Well the reality is if you don't know how these things work then you can't answer this question.
This is why some of these so-called practical courses can cover 20 algorithms in the same course.
It's because they never actually talk about how the algorithms work since the API is the same every
time.
It's very easy to just do the same thing 20 times and then not bother with the details.
So this gives people a lot of confidence that they were able to do the same thing 20 times.
But it's dangerous thinking because you don't realize you really only learned one thing you just repeated
it 20 times.
So how about when it comes to the professional world.
Sometimes people tend to believe that in your profession you'll be leaning more toward the practical
end of the spectrum where you don't really need to know how a machine learning model works.
You just have to know its API but what it really comes down to is what your job actually is.
If you're a regular everyday programmer and you just want to see how a machine learning model might
perform on your data then you're going to use a pre-built library most likely.
That's just the two or three lines of code I referenced earlier.
However that does not mean you don't still have to conform to best practices.
Make sure you're not mixing your test set with your train set make sure you're doing proper validation
and so on and when it comes to tuning your model then you'll have to be able to dig down into the details
and see which hyper parameters are possible to change and which are likely to have the most impact.
So only knowing how to use an API is only going to get you past the first stage.
You can plug in your data and at least your code is the right syntax and doesn't crash.
But of course your code not crashing doesn't necessarily mean it's right
back on the other end of the spectrum you have programmers and scientists whose sole job it is to do
machine learning.
These guys are machine learning experts.
They know the math inside and out and have a full computer science education and so to even be able
to communicate your ideas effectively.
You have to be comfortable with a theory and not just how to plug into an API with two lines of code.
These scientists are building custom models really digging down into the math to not only use but invent
new things and to be able to invent new things you really need to know the math very well.
And so if you're on a team like this you can't just know how to write two lines of code your skill set
has to be far beyond that so your theoretical background has to be very solid.
If you want to be able to communicate effectively with other scientists
now a lot of people say well I don't ever want to be a scientist so I don't really care how these algorithms
work.
Unfortunately that's a very bad attitude to have in the workplace.
Suppose for example I am running a business now an employee of mine.
He wants to show off and say he's learning about machine learning.
He says Oh I'm taking this machine learning course online therefore I'll become a machine learning master.
But his approach to machine learning is to learn as little as possible.
He says he doesn't care about the math.
He only wants the practical stuff.
He only wants to know the bare minimum.
He tells me he doesn't care how it works.
He's just going to plug into some API because that's what his buddy told him he could do as a business
owner.
I really don't want this guy working for me.
I run a business that sells something and the life of the company itself and the livelihoods of the
employees all depend on our products being the best they can possibly be.
So if someone comes to me and says I don't care about being the best I just care about doing the bare
minimum.
My response is I don't want that guy on my team.
In fact he should be fired.
So be careful if this is your approach to machine learning no business owner wants to hear that your
approach to the business is to do the bare minimum and to put off some math because you think it's too
hard.
That's a scary prospect for a business owner.
Now if we go back to our original question maybe you've realized we haven't answered it yet.
What kind of course is this.
Well you can think of it as a marriage between these approaches.
We want to cover all the math and derivations because that's very important.
If you want to be able to communicate effectively with other scientists to be able to debug your code
effectively and to be able to invent new approaches.
This is also what we mean by machine learning.
We mean really knowing machine learning and not just programming with an API.
So the theoretical background is very important.
At the same time we're not going to introduce 20 different algorithms in the same course because unlike
a typical academic course where you can pass just by understanding 25 percent of the material in this
course it's structured in a way that if you have any missing gaps it's going to be impossible for you
to move on to the next stage but academic courses don't contain much coding if at all.
So in this course every algorithm we learn about will be implemented in code.
My motto is If you can't implement it then you don't understand it.
You can pretend to understand it and when you talk about it you might sound like you understand it but
implementation is the true test.
It's like a math exam rather than writing an essay.
When you write an essay you might be able to fool the teacher into thinking you understand something.
But in a math exam either your answers are right or your answers are wrong in this context.
Either your code works or doesn't.
It's very clear cut
when it comes to practicality.
We cover that too.
We demonstrate our algorithms both on real world data like text and images and we demonstrate our algorithms
on two dimensional data sets so that you can visualize what an algorithm is doing.
Both of these are important texts and images are quite possibly the most practical data.
There is some billion dollar companies exist and thrive because of those capabilities to 2D data sets
are critical as well because as humans that's the only thing we can see.
But at the same time we don't try to pull the wool over your eyes and pretend like you need 20 different
data sets to practice on.
Instead we learn something much more valuable how to use an algorithm on an infinite number of datasets
because we are able to abstract the idea that all data is the same.
Remember that all the computer sees is a list of numbers it doesn't know that those numbers represent
height measurements or an image pixel.
And of course by experiencing both ends of the spectrum both the theory and the implementation.
And then finally plugging data into your model that's going to make you very well rounded professionally
to conclude this lecture.
I want to pose a final question.
How does all this knowledge help you.
Why does understanding the approach of this course make you better at machine learning in general.
Well my hope is that this gives you a very high level bird's eye view of the landscape.
You can see the different approaches of people from different backgrounds academic people typically
don't see what other non-academic people are doing.
And those people who hate math typically don't see what people who love math are doing.
So with this bird's eye view you can see everybody if you're learning machine learning it helps you
understand who am I competing with when I apply for a job and what kinds of jobs are out there.
And what would be adequate preparation if you're hiring for a machine learning team.
It helps you understand the different backgrounds of individuals you have to compare to one another
to understand their strengths and weaknesses and what they bring to the table.