In the next couple of lectures we are going to see how easy it is to apply what we just learned to a
real world data set.
In fact this real world data set is one of the most significant datasets of all time.
This lecture is about Moore's Law.
As you recall Moore's Law is the observation that the number of transistors per square inch on integrated
circuits doubles approximately every two years.
In other words roughly speaking computing power grows exponentially.
Some futurists such as Ray Kurzweil have observed that this pattern applies to technology as a whole
so that even when Moore's law runs out for transistors perhaps some new invention will appear to take
its place.
Maybe that will be quantum computing A.I. or something we can't yet imagine.
Now you might be thinking wait a minute this isn't linear at all.
If something grows by a constant factor over a given period of time that's actually exponential growth.
For example it will grow from 1 to 2 4 8 16 32 64 and so on
we can write the generic equation for exponential growth as C equals c not.
Times are to the power t here.
C is the output variable and that's the thing we're counting C not as the initial value.
When C equals zero.
T is the input variable which in our case represents time and finally R is the rate of growth.
You can see that if t increases by 1 then C increases by a factor of R.
This holds true for all time periods.
So if I met T equals 1 going to t equals 2 I can find the next value simply by multiplying by R.
And similarly for going from T equals to the T equals 3
luckily by taking the log of the count variable we can turn this into a linear equation.
Basically we take the log of both sides and we show that the log of C is a linear equation with respect
to the time t.
This is exactly what we want.
So if we use a linear regression on the log of transistor counts with respect to time we should find
that the rate at which transistor counts double is approximately two years
if you can't see how this works right away.
It might help to substitute the variables in the previous equation with some more familiar variables
so let's replace log C with y.
This will be the output variable.
Let's replace log R with a slope.
Let's replace T with X.
This will be the input variable.
And finally let's replace log of C not with B the intercept.
Now we just have Y equals X plus B which is our linear equation from earlier.
Note that I'm using the letter A instead of AMA for the slope which is perfectly okay
at this point.
The problem becomes exactly the same as our previous problem.
As long as we can transform our data center into a quote unquote Excel spreadsheet which has the log
of transistor counts in the y column and the year in the X column then our code for building training
and predicting what the PI torch model should be exactly the same as it was before.
This is what we mean by all data is the same as you will see.
There is no difference in the model between this example with transistor counts and the last example
with synthetic data and you might wonder why do I say this so often.
And really I just go by how often I get asked.
So it's really you.
The students who guide what I say.
If I find that a lot of students have a problem with something then I will emphasize that point more
often.
So despite the fact that I repeat this constantly students will still sometimes ask me can you do an
example on such and such dataset.
And my response is as always the code is exactly the same.
No changes are needed although because this message is pretty effective I think it definitely has helped
a lot of students understand the idea a lot better.
There is one caveat to this which you will see in the following lecture.
In order to use linear regression or any other kind of model in this course effectively you should make
sure your data is either normalized or in a small range of values and centered around zero.
Or at least near zero in order to do this.
We're going to subtract the mean and divide by the standard deviation.
This makes it so that the sample has means zero and variance one please note that in this course we
will use the terms normalize and standardize pretty much interchangeably standardized is a term that
is used more in statistics and it refers specifically to the case where you take a sample and transform
it so that it has mean zero and a variance one.
This is opposed to the term normalized which could refer to standardization but as a more general term
overall for example it could mean transforming your data so that it stays in the range 0 to 1.
This is not the same as having means zero and variance one on the other hand.
In cases like batch normalization which you'll learn about later in this course it does refer to the
standardization operation.
The real question is Why do I mentioned this as a caveat.
Here's the problem we're doing two transformations on our data.
This is going to make it more difficult to prove that transistor counts double every two years.
So here's a quick question for you given that the equation for exponential growth is C equals c not.
Times are to the party.
What does r have to be in order for the transistor count to double every two years.
If T is in units of years I would strongly recommend you try to figure this out for yourself before
we move on to the next lecture.
The next question you want to consider is if we take the log of both sides of this equation.
How does that value of r relate to the slope of the line.
And finally if we normalize the data in both the x and y axes how does that affect the slope and intercept
of the line.
We'll go through these calculations in the next lecture in detail but the main point is we are not going
to get the answer directly just by fitting a line to the data.
For example just because the transistor count doubles every two years this does not mean the slope of
the line is going to be 2.
It's going to be related to the number two somehow but to find out exactly how you have to do the math.