OK, so in this video, we are going to discuss linear regression.
Now, I'm sure most of you already know at least a little bit about linear regression.
So I'll try to keep this short and only focus on new and noteworthy concepts.
OK, so let's start with the simplest form of linear regression, which is just the line of best fit
on a 2D plane.
As usual, we have a bunch of data points called X, Y and a Y one extra Y two and so forth.
As an example, X might represent years of experience and Y might represent salary.
So hopefully it makes sense how you might collect such data points in the real world.
Perhaps you work for the human resources department and you keep this kind of data inside an Excel spreadsheet
and in fact are real.
World use of this model might be to figure out what salary to offer to new hires based on their past
years of experience.
Such a model would allow you to pay your employees in a fair and unbiased way.
Our job, of course, is to find a line that fits nicely through these data points.
In this scenario, we have two parameters the slope and the intercept.
Thus our job boils down to finding out what the slope and intercept should be.
Note one limitation of this model, which is that it can only fit lines the equation Y equals M, X
plus B must be aligned.
So if you're trying to fit a data set that looks like it curves or it turns at any point, then linear
regression is not a good fit.
Later in this course, we'll learn about machine learning models which are non-linear.
Another way that linear regression becomes more complicated is when you have more than one input.
Perhaps you'd like to predict a student's exam grade using the number of hours they studied and the
number of hours they slept the previous night.
Let's just call these X one and X two.
In this case, we typically call the weights W one and two, and we still have an intercept called B.
Note that in this scenario, the objects we are trying to fit is no longer a line, but a plain.
In fact, it's possible to fit a linear regression model with any number of input dimensions.
Now, of course, if we have a million dimensions, we don't want to write one X1 to X2 and so on a
million times.
Instead, it's more convenient to express all the WS in a single vector called W and all the Xs in a
single vector called X.
As you recall from your high school math studies.
This element Y's product in summation is also known as a DOT product.
Therefore, our model can be written as W, transpose X plus B.
Now, this is such an important expression, in fact, you shouldn't even think of this as MAFF, but
rather you should automatically recognize it as merely a pattern.
You'll see the same pattern when you study logistic regression, when you study neural networks, when
you study support vector machines and so forth.
So basically, do not run away when you see this because it looks like scary math.
Nearly everything important in machine learning is based on this.
So if you want to do machine learning, this is fundamental.
And again, it's not math, it's just a pattern.
When we talk about neurons, you'll understand this more in depth.
So just keep this in mind until then.
So the noteworthy part of this lecture is this hopefully you've realize now that the auto regressive
model is nothing but linear regression.
If we replace the X Factor with the lags of a time series and we replace the output with the current
value of the Time series, we have an auto regression.
In fact, we can immediately extend what we learned and make it more powerful, as you recall, our
models can only predict one step into the future.
But with linear regression in general, there's nothing stopping us from simply adding more outputs.
We can have a vector output along with a vector input.
For example, we can use one way to and y three to predict y foreign y five.
In this case, our weight vector becomes a weight matrix and our byas term becomes a base vector.
Basically, if you have D inputs and outputs, then W will be of shape D by K and B will be A vector
of size K.
Alternatively, you can think of this like having parallel linear regressions.
So if you have K parallel models and each of them has a weight vector of size D, then clearly by putting
them together you'll have something of size D by K or covid, but we usually used by K as convention.