In this section of the course we are going to discuss the theory behind that reinforcement learning
this lecture.
We'll give you an introduction to reinforcement learning and we will talk about it in general terms
without any math or terminology one thing you have to brace yourself for when you're learning about
reinforcement learning is how different it is to supervised and unsupervised learning.
So if you just took an introduction to machine learning where you learned about models such as naive
bayes and K Means clustering or you come from a statistics background you will be surprised at how different
reinforcement learning is compared to what you are used to.
So try to take some time to soak up these concepts and don't feel intimidated by this new and different
way of thinking.
I want to begin by thinking about supervised learning when we think about say an image classifier.
You can think of it as a static function.
I pass on an image and I get a prediction.
It tells me what kind of object is in the image.
For example there is no notion of time I pass in another image.
I get another prediction.
The image classifier is just a function I given an input and it produces an output
so what do I mean by static.
And what do I mean by time.
You might immediately think of recurrent neural networks which are neural networks that can handle sequences
inputs that vary with time.
However this is not the kind of time I am thinking about if I pass in some stock prices for a given
time period and my model predicts whether the stock will go up or down tomorrow.
That's still a static function.
Here's what I mean by time.
Imagine you are building a self-driving car simulation at each moment in time your neural network can
take a snapshot of the screen and decide what to do next.
Should I still left.
Should I steer right accelerate or brake.
So this is the difference between a supervised and reinforcement learning supervised learning is just
a function you can call this function repeatedly but it's still just a function you pass in an image
and it produces an output reinforcement learning is more like a loop.
It exists to achieve some goal.
For example driving you to your desired destination inside the loop yes it still takes in an image and
produces an output that specifies how to control the car but importantly this reinforcement learning
program has the concept of time.
It doesn't think about just what is this image.
How do I translate this image into an output prediction.
Instead it has the capacity to plan for the future even though at this moment the car may only see where
it is on the road right now.
It knows that there is some sequence of actions it must take in the future that will lead it towards
its goal.
All right so that's the major difference between a supervised learning and reinforcement learning with
supervised learning.
We have no concept of a goal or the future or planning.
We just take an input and produce an output.
It's a static function.
With reinforcement learning we have a plan and that plan can be carried out in the future to reach some
predefined goal
here's another way to think about reinforcement learning.
Think about the data with supervised learning.
Again let's use image classification as our example.
We must have a label for every input in our training set.
So if we have an image of a dog we must have another item specifying the class dog.
If we have an image of a cat we must have another item specifying the class cat.
In other words for every x we must have a Y
it's important to remember that label data sets must be made by humans.
Sometimes students have this really funny idea that we should just automate the creation of labeled
datasets guys.
If we already had computers that could perfectly label data then that would mean we have already solved
machine learning computers that can perfectly label data are actually what we are trying to build.
If such computers already existed then we would not need to build them.
All right so hopefully you are convinced that label data comes from humans and not some super smart
computer.
Why is this important.
Well think about our self-driving car again.
Take this image of the road.
If we were to use supervised learning for this data point we would need to give it a target.
But what's the target.
Should I steer left.
Should I steer right.
Should I accelerate.
Should I brake.
In fact you probably cannot give this image a target
and even if you could how could you possibly label every single frame that the car will encounter along
its journey.
Imagine if you have a standard camera that captures images at 30 frames per second and you have a one
hour drive as your data set.
One hour is thirty six hundred seconds which means you would have to label one hundred eight thousand
images from just one trip
instead.
Reinforcement learning learns using goals rather than targets.
For example suppose you wanted to teach a reinforcement learning algorithm to solve a maze.
In this scenario the goal would be finding the maze exit.
You do not need to tell the algorithm what the correct thing to do is for each position on the Maze.
Since that would be supervised learning instead the only thing the reinforcement learning algorithm
needs to know is what is the goal from there.
Figuring out what to do in each position of the maze can be found by reinforcement learning.
That is the power of this new paradigm.