To end and summarize this section I want to discuss at a high level what it takes to learn it reinforcement
learning for real.
Personally I think it would be quite difficult to grasp reinforcement learning and understand it on
an abstract level from just a single section of a single course
let's start by recognizing that reinforcement learning is very different from supervised and unsupervised
learning.
Nowadays when you're studying supervised and unsupervised learning many beginner courses take the approach
of avoiding how to implement the models.
This is not a good approach because it doesn't prepare you for understanding real machine learning.
Let's suppose the structure of the courses like this First you discuss the intuition behind how the
model works.
Just some pictures and some descriptive ideas.
Then you learn how to use that model in psyche.
Learn not including loading in the data or looking at the results.
So what is the problem with this approach.
Well it allows you to not only make many mistakes and have serious misunderstandings about the content
but perhaps the biggest problem with this approach is that you will never know what those misunderstandings
are.
In other words you are in a place where you don't know what you don't know.
You don't know your mistakes and hence you can't fix them.
You're confident that you've learned something but this is nothing but blissful ignorance and again
these are just my observations of other students.
So don't take it as a personal judgement of you.
This is not so much a problem.
If that's how far you want to go if you only ever plan on using AP eyes and you'll never want to become
an actual professional that's totally fine and nobody is judging you but it becomes a problem when it's
time to stop using AP eyes.
This turning point appears when you want to learn how to do reinforcement learning.
There is no API for reinforcement learning at least not yet.
So what is one to do.
Well you have to implement those algorithms yourself.
The problem is if you have no experience implementing machine learning algorithms reinforcement learning
is not a good place to start.
I want to also comment about the many blogs tutorials and so forth that exist out there attempting to
teach you about reinforcement learning.
Consider how long it took us in this section to go from nothing to deep Q learning at a pace that I
would consider as fast as possible.
Compare that to the length of a typical blog tutorial.
It should be clear that it would be impossible for a short blog post to include enough detail to get
you to the point where you can actually implement reinforcement learning yourself.
The best advice I can give you if you want to learn reinforcement learning for real is this.
I would recommend taking a full course or multiple courses about reinforcement learning.
Learn about basic tabular reinforcement learning before you move on to approximation methods that will
make understanding the concepts much easier in this section we spent most of our time discussing the
concepts under the assumption that we were using a cue table and not a neural network.
You want to learn about the basic approaches to reinforcement learning which can be categorized into
number one dynamic programming methods.
Number two Monte Carlo methods.
And number three.
Temporal Difference methods.
And make sure to spend some time on implementation
after that.
Move on to approximation methods with linear models.
Once you've done that and you've implemented it reinforcement learning algorithms with linear models
move on to using deep learning for function approximation at this point you should be very comfortable
with the concepts but implementation is still another matter entirely.
Unlike supervised and unsupervised learning reinforcement learning is very hard to get right even if
you are an expert programmer.
It is very easy to introduce subtle bugs that are difficult or nearly impossible to track down
and of course you don't have to take my word for it.
Here is a post by Andre Carr party probably one of the most famous reinforcement learning researchers
talking about just how difficult it can be and this come in another user talks about how they spent
an entire year trying to get Q learning to work with neuron that works for a particular gain.
Many students who take my courses get frustrated when an exercise takes more than a few minutes.
Now imagine working on something for one year Andre response to this comment and says it took me six
weeks to get policy gradients working and keep in mind he's one of the world's leading researchers in
reinforcement learning.
He has all the state of the art technology he needs on demand and he's surrounded by other researchers
who are just as smart as he is so he can ask them for advice or to check his code at any time.
So the key idea I want you to keep in mind is that as you learn reinforcement learning realize that
implementation is very non-trivial even for experts.
You should expect to spend a lot of time and effort trying to get things right.