In this lecture we are going to start looking at machine learning from a new perspective.
While this example might seem overly simplistic you might be surprised to learn that every example that
we do in this course will rely on these basic steps which you are about to learn.
First let's start by describing the basic task.
I'm sure you're all familiar with this example from your grade school of science and math classes.
Let's suppose we've done some kind of scientific experiment.
For example we measured the voltage in a circuit with different resistors or we asked everyone in our
class what their height and weight is.
This gives us a dataset with two columns of numbers one that we can call X and the other that we can
call y.
Now imagine that we plot these points on a grid and we get something that looks pretty close to a line.
We would like to draw a line on this grid that closely passes through these points in your grade school
class.
Maybe you did something like took a ruler and moved it around a little bit until you found the best
possible line.
But now that we're doing machine learning we need more sophisticated methods than just drawing dots
on paper and using a ruler to estimate what the best line should be.
The problem is what happens when we have thousands or millions of dots.
What if our data is more than 2 dimensional then this approach is no longer possible.
So perhaps you might say aha I know what to do.
Let's use our trusty tool socket learn
as you know by now.
This is what a basic regression script might look like if we were to use psychic learning.
First we load in some data call that x and y.
These would just be the two columns of data that we were looking at earlier.
Next we create a model for the coming lectures we'll be interested in linear regression.
So let's create a linear regression model.
Next we call the fit function to fit our model to the data.
Next we can do multiple things with our fitted model such as making predictions with models that predicts
the problem with this is this is just using an API but it doesn't tell us how any of this actually works
what actually happens inside these functions and why is that important to know at this stage.
What I want to highlight on this screen is what we should expect it to be different.
When we look at Pi talk first model creation is going to be different.
As I mentioned earlier pi torture is going to allow us to build all kinds of different models using
compostable building blocks.
So it's not going to be some predefined model as inside you learn next.
I want to emphasize in PI talk there is no such thing as a fit function or a predict function.
This is unlike tensor flowing carrots which do provide you with those functions.
So if you're looking for something simpler than tensor flow or KRS maybe more your pace in PI talk however
that will not be the case.
Therefore we're going to have to find out how to do all these steps in PI talk using our knowledge of
the concepts.
Importantly however this lecture will focus only on the concepts and not on any PI talk in the next
lecture.
We'll look at how to actually implement these concepts in PI towards syntax
so let's first look at what these concepts are.
Before we discuss the details I'm going to go out of order in some sense but you'll see why that has
to be the case.
The three main concepts we want to discuss are number one what is the model architecture and how do
we actually build a model.
If it's not defined for us like it is inside get learn.
Number two how do we make predictions with the model.
In fact this is closely tied to number one and number three.
How do we fit the model that is how do we find the slope and intercept parameters as you'll see.
This is the most complicated step.
It also depends on number one and number two which is why we have to discuss this last
so number one.
What is the model architecture.
Well you already know this.
It's why hat equals M X plus B.
Notice our use of the hat symbol here.
Typically we'll use Y for the true value also known as the target which is given in the dataset and
y hat for the corresponding prediction.
Fortunately for us this actually solves both of the first two problems for us.
This not only defines the model architecture which is a line but it also tells us how to make predictions
given some input x.
I can see exactly how to calculate y hat.
Pretty simple.
So this is our model y hat equals M X plus b
the hard part is number three.
How do we actually find good values of the slope M and the intercept B.
The key is to define what is called a lost function.
I warn you that there are many synonyms for the lost function and you'll see them all.
So don't be afraid if you see these different terms and think one of them might be wrong.
That's not the case.
These words are all used equivalently so you might see this referred to as a cause function or an objective
function or an error function.
These all mean the same thing.
The basic idea is this if we pass in our data set X and Y along with some setting of MLB the lost function
will tell us how good that setting of MLB is.
Let's see how.
First let's assume that we're given a set of pairs of data points X1 y 1 x 2 y 2 all the way up to X
and Y N make a note to yourself now that it will be convention for us and many others to use the capital
letter N to represent the number of samples in your dataset.
This is not always the case but it's at the very least the convention I use and the convention and many
others use.
So if you see something different in the future or you've seen something different in the past don't
freak out believe it or not.
This happens sometimes anyway.
The idea behind linear regression is we will not find a line that perfectly passes through all the points
that's impossible since as you can see there is no such line that can pass to all the points.
All we can do is try to find a line that is a good fit.
So how do we define a good fit.
As I mentioned this is called a lost function.
I'm going to start by telling you what it is and then describing to you why it makes sense for some
of you it'll make sense immediately and for others it might not.
But hopefully by the end of this I will have convinced you that it does make sense so the last function
that we use and regression is called the mean squared error or MSE for short.
Basically you take all the y I's and you find the squared difference between them and the line itself.
The Y had eyes you can think of this as the squared vertical distance from each y eye to the line each
corresponding point on the line is a Y had I a prediction that given x y.
Once we have all the squared differences we take their average you will know how to find averages so
you know that is just the sum of the squared differences divided by the total number of points and
the question is why does this make sense.
I think this is a great exercise to try for yourself if you haven't done so before.
Take a piece of graph paper and draw some dots and a line just like what you see here and calculate
the squared error by hand.
What you should discover is that if the line is not a good fit then the error will be larger.
If the line is a good fit then the arrow will be smaller.
I think that's pretty intuitive.
In addition you should recognize that if the data points perfectly lie on the line then the mean squared
error will be 0.
Next question how do we use this lost function and our model to actually find the values of m and b.
Let's see what happens if we plug in our expression for y hat into the loss.
This is clearly the equation we would get.
Now I have a quiz question for you.
What are the variables in this equation.
I'll give you a second to think about it.
So please take a moment to come up with the answer before moving on to the next slide.
Ok so I hope you thought about the answer to what the variables are in this equation.
If you said x and y.
This is not correct.
Many beginners make this mistake because in a typical math course the variable is X and you're always
trying to do something like solve for x.
However it's important to recognize that in machine learning X is normally not a variable X Y and Y
are just values from my dataset.
You can think of them as numbers from your Excel spreadsheet.
In fact in our case the variables are M and B.
Remember that these are the values that we are currently trying to find.
They are unknown and therefore they are variables
at this point.
We know that our goal is to minimize the loss with respect to M and B this is how we would write that
mathematically.
If you're afraid of math don't worry about this equation too much because it doesn't give us anything
new.
It's just a way of saying what we already said but using math we would like to find the best values
of m and be called M star and B star where these values of m and b are the ones that minimize the loss
l out of all possible values of m and b how do we solve such a problem.
It might help to consider a similar but simply problem say we have a function f of x equals X squared.
A quadratic.
We all know this from high school.
You might say well I can just look at the graph and see that the minimum value is zero when x is zero.
But remember we don't do that in machine learning because in machine learning we work in multiple dimensions.
You can't just look at it at this point we have to introduce our old friend calculus.
If you recall the way that we solve this is you find the derivative D F by the X set it equal to zero
and solve for x.
This is because as you can see the derivative at the minimum is equal to zero so if you draw a tangent
line at the minimum point it should be a horizontal line.
There are of course a lot more details that I'm not including here but you should at least remember
this from your high school math studies.
So how do we find the solution in our case since we have two variables m and b we have to take partial
derivatives but the idea is the same find the partial derivative of L with respect to M and set that
to zero.
Find the derivative of L with respect to B and set that to zero.
This will give us two equations with two unknowns which are MLB at which point we can solve for MLB.
In fact if you've never done this before I would encourage you to do this calculation and solve for
MLB as an exercise.
This can be done for any arbitrary data set X1 y one up to X and Y N So your solution should be in terms
of those values.
There is a problem with this approach however unfortunately finding the derivatives and setting them
to zero only works for linear regression.
It does not work for any other model we will discuss in this course.
In fact the very next model we discuss for linear classification cannot be solved in this way.
Instead we need to use a method called gradient descent.
Very briefly the gradient is the multi-dimensional analogue of the derivative.
So basically when you're working in multiple dimensions you have gradients in place of scalar derivatives
the idea behind gradient descent can be understood with this simple picture.
It's an iterative algorithm.
So we're going to go in a loop on each iteration of the loop.
We're going to take a small step in the direction of the gradient.
It's been mathematically proven that if we keep doing this we will eventually end up at the minimum
point in this lecture.
I don't want to get into detail about gradient descent.
However you are encouraged to check out the lectures in the ends up section of this course.
If you are interested.
Our goal is to get to how we do this in PI to work as quickly as possible.
Gradient descent is a really cool algorithm because it works in pretty much all cases.
That's why libraries that have automatic differentiation such as the ANO tensor flow and Pi torque have
been so popular for machine learning aside from deep learning.
You can use the exact same techniques for things like Hidden Markov models.
K Means clustering Matrix Factorization and more
all right.
So since this has been a pretty long lecture let's recap the steps.
Our goal in this lecture was to take you from the three steps of the cyclone API to what actually happens
conceptually as you recall the three steps are creating the model fitting the model and making predictions
with the model.
After this lecture you now understand at a high level what actually happens during these three function
calls.
However the concepts don't provide the full picture because we still need to implement this in code.
In fact I have a course that goes very in-depth into all these concepts and how you can take them and
put them into a computer program using basic libraries like num pi in the Python programming language.
But remember this course is about pi talk.
So the next question we have to ask is how is this done in PI talk.
Concepts are one thing but syntax is another.
For example the syntax for Pi talk will be different from the syntax for tensor flow.
Even if we are doing the exact same thing.