Throughout this course, we've sort of been following this trend of covariance and then correlation
or the Pearson correlation coefficient.
And now we're going to continue that thread by looking at the coefficient of determination.
So if you remember, covariance tells us about the direction of the relationship between two variables
X and Y, but we don't really use covariance to analyze that relationship because covariance is sensitive
to the scale of the two variables.
So it can take on any value and it's hard to interpret.
What we do instead is use covariance to calculate correlation, which we always indicate with r Pearson's
correlation coefficient.
R or just correlation.
And this value tells us about the direction of the relationship between the variables, but also about
the strength of the relationship.
In other words, it kind of answers the question how close are the data points to the regression line
or the line of least squares of the line of best fit.
And if you remember, we talked about in the last lecture that correlation can take on values between
negative one and one.
So a value of exactly negative one tells us that all of the points in the data set, all the points
in the scatterplot lie exactly on the regression line, and that the regression line has a negative
slope.
So it would tell us that the relationship between X and Y is as strong as it can possibly be in the
negative direction, whereas a correlation of one would tell us that all of the points in the scatterplot
lie exactly on the regression line and that the regression line has some positive slope.
So again, the correlation is very strong and the direction is positive.
So it gives us an idea of how close the data points are to the line of best fit in addition to some
sense of direction.
Now, if we continue this thread one step further, we can think about the coefficient of determination
which really answers the question for us.
How much error is eliminated by using the regression line?
Or we could continue the sentence instead of using the mean of the dependent variable.
So we indicate the coefficient of determination with this lowercase r squared because it is exactly
the square of correlation lowercase r, which means that assuming we have correlation in order to find
the coefficient of determination, all we have to do is square the value of the correlation.
Realize, though, that that only works for simple linear regression.
So we have to be doing linear regression instead of curve fitting the data with a curve of some other
shape.
And we have to have a single independent variable that's affecting a single dependent variable.
In other words, if we're doing multiple regression and or we're doing non linear regression, then
the coefficient of determination is given by capital R squared.
And we need to say that capital R squared is not simply the square of lowercase r, the correlation
or Pearson's correlation coefficient.
So we find this in a different way, but for now we're sticking with simple linear regression, in which
case we can find the coefficient of determination by squaring correlation r Which means that the value
of the coefficient of determination is going to fall between zero and one, and we usually give our
squared in terms of a percentage.
This makes sense because if we think about correlation as taking on values between negative one and
positive one, when we square any value in this interval, we'll end up with values only in the interval
0 to 1.
So that's why this interval of values becomes this interval of values when we square our to get r squared,
which means when we say how much error is eliminated.
What we're really saying here is what percentage of the error is eliminated when we use the regression
line instead of just the mean of the dependent variable.
One last thing we want to say about capital R squared here the coefficient of determination for multiple
regression or for non linear regression.
This value capital r squared can take on values in the interval negative infinity to one, not just
the interval 0 to 1.
But in either case, whether we're looking at lowercase r squared or capital R squared, the closer
the coefficient of determination is to a value of one or 100%, the more we can say that the regression
line is a better estimate of the data than just using the mean instead.
In other words, when we have a value for R squared that's closer to one, it means that the independent
variable has more explanatory power about the dependent variable, or that there's a higher percentage
of the variation in Y that's explained by x.
So R squared really just tells us the goodness of fit of our regression line or the strength of our
linear regression model.
So with some of that background out of the way, let's actually look at a scatterplot so that we can
get a visual understanding of what we're trying to say here.
So over the last couple of lectures we've been working with this data set here, we've looked at it
a few times.
We have the values of the independent.
Variable X and the values of the dependent variable Y.
And so each of these pairs gives us a coordinate point in our scatterplot.
So if we look at the point here, 00.8, we find it right here in our scatterplot and these are identical
scatter plots.
They just have a different line through them, which we'll talk about in a second.
But the points are all the same.
So we have this data set.
Each of these points is plotted in the scatterplot and in this left hand scatterplot.
The yellow line represents the mean of the dependent variable y.
So we calculated this before, but the mean of all of these Y values is 0.8.
So if we just sketch in the line Y equals 0.8.
We have this line here.
If you think about it, this is the most crude basic way of coming up with some kind of a regression
line for the data in a scatterplot.
All we're doing here is adding up all of the Y values in our scatterplot and dividing by the number
of points that we have.
And then we get that average or that mean and we just sketch the line in at that mean.
And in theory we could call that a line that plots some kind of a path through the scatterplot.
It doesn't do a very elegant job, but it does try somewhat to balance the error that we find in our
data set.
In fact, we can see here that if we look at the error between each data point and the line representing
the mean, so we look at each of those distances, the distance here, the distance here, the distance
from each point to the line.
So all these.
Different.
Distances and we add them all up.
The sum of those distances is zero.
We see that in this column right here, the Y minus Y bar column, and that's because we treat the error
for data points below the mean as negative and the error for data points above the mean as positive.
And so the line representing the mean here just balances out those positive and negative values by finding
the value at which those positive and negative values would sum to zero.
In fact, that's the definition of the mean.
But remember that when we're talking about error, we always talk about squared error.
And so if we square each of these distances, if we create an actual square from each of these, so
something that looks like this for each.
One of these, and then we have this big square up top.
Here, etc. We have a certain amount of squared error when we use the mean as our theoretical line of
best fit.
In this last column of this chart, we're finding the area of each of these squares and if we add up
all of that squared area, we get a total sum of 2.24.
That's the total area of all of these squares.
When we use the mean for this line here.
Now, if instead we use the regression line, the line of best fit, and we found the equation of this
line earlier, what we expect is that this regression line should do a better job of fitting the data
than just using the mean.
The value of r squared.
The coefficient of determination tells us exactly how much of a better job this line will do.
So what we can do is a really similar calculation to the one that we did in this table.
Again, we have the same data set with these raw values of X and Y, and we can use the value of X to
find all the values of Y hat all of the predicted values, the values predicted by the regression line,
by plugging each of these values of X into this equation for the regression line.
So we get the value y hat that corresponds to each of these values of X, and then we take y minus y
hat, which remember is the residual.
This is the residual E, the error that is the distance from the actual value y in the coordinate point
to Y's predicted value along the regression line.
So if y minus y bar is the distance of each point to the line representing the mean, then y minus y
hat is the distance from each point to the regression line, the line of best fit.
So these values from this column are again all of these distances, which because this line is a little
different than the mean line, these distances will be slightly different.
Then the ones that we see over here.
And then when we square those values, we create a square from each of those distances.
So we create a square for each of these and when we add up.
The area of all of these squares, we get the sum of the squared area.
And for us, that total sum here, when we're using the regression line is this value here 2.217 approximately.
So what we can see is that in this particular example, this value, the amount of area we have over
here is just barely smaller than the amount of area we had over here, which means that the regression
line in this case for this particular dataset does a very, very, very slightly better job at fitting
the data than the mean line itself.
And we can use this formula for the coefficient of determination to figure out exactly what that percentage
is.
So if we do our calculation here, we get one minus.
This is the sum of the squared residuals.
It's the 2.217 value we found.
So this is approximately 2.217.
And then this sum in the denominator is the 2.24 that we calculated.
And so if we take this decimal number divided by this decimal number and then we subtract that result
from one, we get approximately 0.0102.
Or if we change that to a percentage, we get approximately 1.02%, which means that using this regression
line compared to using the mean line eliminates about 1.02% of the error, which we could confidently
say is really not very good.
Remember, ideally, if the regression line is a great fit for the data, we would have an r squared
value much closer to.
100% then to 1%.
And we can actually see this 1% value reflected in these scatter plots.
With the mean line in the regression line here, you can tell just by looking at these that the regression
line is barely any different at all then the mean line, the square areas look almost identical and
we can see that the data points in the scatterplot are spread out far away from both the mean line and
the regression line.
So neither of these lines does a great job fitting the data.
The regression line does about a 1% better job, but still overall a pretty poor job.
But in general here, just remember that we're saying that the better that the linear regression line
fits the data in comparison to just the mean line, the closer the value of the coefficient of determination
will be to 100% or to one.
If we're using a decimal number here, we ended up with 0.01.
We want that closer to one if our linear regression line is eliminating a lot of error.
So here we calculated the coefficient of determination from scratch.
But in the last video we did look at correlation and we found that correlation for this same dataset
was approximately 0.101.
And if we take the square of this value, we do get close to this 0.010 to value.
This original correlation value is rounded, this value is a little bit approximate, but we could have
just taken our correlation and squared it and come up with the coefficient of determination.
Or we can use this formula to calculate it and work with the data this way.
Now the last thing that we want to talk about here is what's called root mean square error or or you'll
also see it referred to as root mean square deviation.
Msdh.
They mean the same thing here.
The formulas we use to calculate root mean square error.
This is just two different ways of writing the same formula, but we can think about this value as the
standard deviation of the residuals.
Essentially what we're talking about here is really similar to the standard deviation we've looked at
before.
Remember that standard deviation is all about the spread of the data around the mean.
And up to now we've been thinking about it in terms of a data distribution.
Well, a scatterplot is like a data distribution.
It shows us visually how our data is distributed.
And in a plot like this one, we can see how it's distributed around the regression line.
Why hat and so root mean square error or root mean squared deviation is going to give us standard deviations
around this regression line.
And if you remember that 68% of the data lies within one standard deviation, that means that 60% of
our data points are going to lie within this same kind of standard deviation, or that 95% of our data
points are going to fall within two standard deviations of the regression line.
So notice here that to calculate root mean squared error or root mean square deviation, we just take
the y minus y hat, quantity squared values.
We already have those in this last column here.
We add them all up.
Notice this formula tells us to sum those so we add them all up.
The sum is about 2.217, and then we divide that sum by the number of data points that we have in this
case for this data set and equals seven because we have one, two, three, four, five, six, seven
data points.
So we divide this 2.217 by seven data points, and then we take the square root of that value.
And in this case we get an approximate RC of 0.5628.
So what that tells us is that the distance between each of these.
Why Intercept's here?
Each of these.
Distances is 0.56.
Two eight.
We can see here that the intercept of the regression line is 0.7143.
If we add 0.71432.5628, we get about 1.28 or so.
And we see that the y intercept of this blue line here is about at that value.
This is the upper edge of one standard deviation around this regression line.
This is the lower edge of one standard deviation around the regression line.
So we know that in this interval here we should expect to find 68% of our data points.
And then we haven't sketched the lower edge of two standard deviations around the mean.
But we would have another red line down here.
And between this lower red line and this upper red line, that would be two standard deviations around
the regression line.
And so we would expect to find 95% of all of our data points within that interval.
Remember, those values are coming from the empirical rule, which tells us that we find 68% of our
data within one standard deviation, 95% of our data within two standard deviations, and about 99.7%
of our data within three standard deviations.
So just think about this as the standard deviation of the residuals, which means that the larger the
value is of our MSE.
The further apart these lines will be, which means the more scattered our data points are and the weaker
the correlation is in the data.
If we find a smaller rmafc value, a smaller standard deviation of the residuals, that means these
lines are going to be closer together, which means the data points are more tightly clustered around
the regression line.
And that means that we have a stronger correlation within our data set.