So in this lecture, we'll be looking at a notebook where we use TensorFlow to find a line of best fit.
Remember that the goal of this notebook is to learn the basics of TensorFlow syntax.
These include how to build a model, how to pass in data, to fit the model, and how to use the model
to make predictions.
So let's begin by importing everything we need.
You can see that I've imported a few layers the model class and the Adam Optimizer from TensorFlow that
carries some students aren't aware of this, but Google, who invented TensorFlow actually recommends
using the Keras CBI unless it is not possible.
So some students always ask Why are we using Keros instead of TensorFlow?
And this is because Google has stated that this is the correct way to use TensorFlow.
So basically, you listen to the people who invented this.
The next step is to create a linear regression dataset.
We'll begin by setting end the number of data points to 100.
The next step is to create our inputs, which we'll call X to be uniform random data points between
them minus three and plus three.
The next step is to set the targets y to be a linear function of X, along with some Gaussian distributed
noise.
As you recall, the linear regression model actually assumes that the noise comes from a Gaussian distribution
for reasons outside the scope of this course.
This is connected to the mean squared error, which you'll see shortly.
You'll want to make note of both the slope and intercept of this line.
As you recall, the equation for a line is Y equals m x plus b.
Here we can see the slope is zero point five, while the intercept is minus one.
One of our goals in this script will be to see if we can recover these values.
The next step is to draw a scatterplot of our data to see what it looks like.
So it's a line with some added noise as expected.
So the next step is to build our model.
As you can see, this consists of three parts the input, the dense layer and the model class.
For the input, we've specified that its shape is a tuple containing a single value of one.
This is because, as you recall, our data is one dimensional.
If we add dimensions, then we would set this value to D later.
In this course, you'll see how this can even be multidimensional.
For example, number of time steps by number of input features.
The next step is to create a dense layer.
Basically, a dense layer implements and a fine IT transformation, also somewhat incorrectly known
as a linear transformation.
Basically, this is doing the M X Plus B computation I told you about earlier.
So it takes the input, multiplies it by some internal parameter called M, and adds another internal
parameter called B.
This all happens inside this layer, so you don't have to worry about the details.
Note that in general, this can multiply in input vector by a matrix and add another vector.
It just so happens that in this case, the matrix is one by one, and the bias vector is just a vector
of size of one.
This is because, as you can see, our input has dimension one and the output size, which is the argument
into the dense object constructor is also one.
Another detail to notice is that this uses the Keros Functional API.
In simple terms, this means that although the dense object is a Python object, it can also be treated
like a function.
So when we say dense one, this creates an object, but we can have additional parentheses after that.
To call this object since, it behaves just like a function.
Note that this returns an output which we simply call X.
Now, please note that some of the more naive beginners will ask Why do we use all these single that
are variable names?
This is simply because this is the convention.
When using TensorFlow and Keras, I've written an article about this.
So if you have this question yourself, please contact me and I will send you the article, which contains
a more in-depth discussion about this topic.
The final step here is to create the model objects.
You can see that the constructor takes in two arguments.
First, the input which we defined above in the output also defined above.
The next step is to print out a model summary, which we accomplish by calling modeled that summary.
As you can see, this prints out some information about the model in the data that passes through it.
In the first column, we can see each layer along with its type.
So we have an input layer followed by it.
That's in the second column.
We can see the shape of the data after going through each layer.
As you can see, this is none by one in both cases.
So why is it none?
As you recall, when we do machine learning, we can have any number of samples for this notebook.
We've created 100 samples, but obviously we want our model to work with other numbers as well.
We might want to pass in just one sample or one million samples.
So the nun is like a wild card that says this dimension does not have to take on any specific number.
Finally, in the third column, we can see the number of parameters in each layer, the input has zero
parameters because it's just an input and the dense layer has two parameters as promised.
As you recall, these are the slope and intercept of our line.
The next step is to call the compile function, which allows us to specify some important information
about how our model will be trained.
As you recall, we've only just defined our model, so it has a slope and intercept, but we don't know
what these values should actually be.
Our model has not yet seen our data.
The first argument is the loss where we pass in MSI, which stands for a mean squared error as promised.
If you don't know what this is or you need to review, please contact me or use the Q&A.
The next step is to specify the optimizer.
Basically, the method of training a neural network involves a process called gradient descent, which
you can visualize like a ball rolling down a hill.
This will be discussed a bit more in depth later in the section.
But for now, all you need to know is that there are different flavors of gradient descent.
The most popular is called Adam, and these days it's often the default choice.
You'll notice that I have two atoms here with one commented out, the first atom is a string.
This is convenient because it takes less typing and you can use this if you're OK with the default values
for this notebook will be creating an atom object explicitly, which allows us to set the learning rate
to zero point one, which is a better value for this data set than the default.
We'll discuss more about how to choose these values later in the course.
As a side note, recognize that the same principle applies to the loss as well, although we've passed
in a string.
We could also pass in a Keros object or even write a lost function ourselves in the case where we don't
want to use the default values.
Finally, we have the metrics argument, which allows us to specify more metrics other than the loss,
which will be computed and displayed on each step.
For this example, is kind of pointless because for regression, we typically care about the MSE, which
also happens to be the loss.
You can see that I've specified the M-80, which stands for mean absolute error, which is another possible
metric.
This will be more relevant in the classification example, where the loss is not the same as the metric
we care about.
The next step is to call a fit method.
So this is where we actually perform the gradient descent process to find the slope and intercept the
previous step was only to set up various training parameters.
But now we will actually do this training on our data dataset.
Note that the first two arguments are the inputs in the targets.
You'll notice that have reshaped the targets to be minus one by one.
As before, minus one is a wildcard.
So this value should just become whatever is left after sending the second dimension to be of size one.
The reason we need to do this is because, as you recall in machine learning, our data sets in general
are of size n by D, where N is the number of samples, and D is the number of features.
In this case, the is one.
But what we currently have is only a one dimensional array of size n in order to convert this into the
end by DX format.
We need to make it a two dimensional array of size n by one.
So that's effectively what this does.
The next argument is epochs, which I've said to 200.
This sets the number of steps of grading at the scene through the data set that you want to do.
Again, this will be discussed in more detail later in the course.
The final argument is batch size, which allows us to specify how many data points of our model will
be looked at on each gradient descent step.
The reason we need this is sometimes our dataset will simply be too large to fit into memory, so it's
not possible to do the same on the whole dataset at once.
Generally speaking, we tried to make this value as large as possible without degrading performance.
Finally, note that the fit method returns, something which I've simply called are in this notebook.
So let's run this block.
OK, so as you can see, when we call the fit method, this will print out some information after each
epoch, like the amount of time it took, the value of the loss and the value of any metrics you want
it to compute.
In this case, that's the main.
OK, so once our training is complete, there are some things we should always do since, as mentioned
in deep learning, we cannot trust that the process simply worked as expected.
As you recall, we got back this variable called R from the previous step.
This is an object which contains a history of the training process.
You can see here that the history attribute contains a key called loss, and we can plot these values
to see the loss per epoch.
OK, so you can see that we get a chart where the loss starts out very high and gets smaller overall
on each step.
This is what we like to see.
But again, this will be discussed in more detail later in the course.
Note that any metrics you pass in can also be plotted.
So in this step, we'll also plot the Emmy.
OK, so as you can see, the MP for EPOC looks pretty much the same as the MSI, which was the loss.
The next step in this notebook will be to see how to make predictions.
To do this will create a new set of inputs called ex tests, which are just 20 evenly spaced points
between the minus three and plus three.
As you recall, we should reshape this to be of size and by one, the next step is to call Model Duck.
Predict passing your next test to get the predictions, which we'll call P test.
The next step will be to plot the original training points, along with our test, predictions for the
training points will make these a scatterplot.
But for the predictions, we'll want to have a line chart.
As you recall, our model is Y equals m x plus b, which is a line.
OK, so as you can see, we have, in fact found the line of best fit, which passes nicely through
the training points.
The next step in this notebook will be to check the slope and intercept that her model found to do this.
Well, first note that we can access the layers of our model by calling model layers.
As you can see, we get a list of all the layers we created above.
The next step will be to demonstrate how to get the parameters that were learned for our dense layer,
which is the layer at position one.
Note that this is done by calling the method get weights.
So as you can see, this returns a list of two an umpire raise notice that the first umpire, Ray,
is a two dimensional one by one array as promised.
As you recall, this is because in general, this parameter can be a matrix with multiple inputs and
multiple outputs.
In any case, you can see that the value here is close to zero point five, which corresponds to the
slope we defined above.
Now, notice that the second umpire, Ray, is a one dimensional array, which in general can be a bias
vector in the case where you have multiple outputs.
Again, we see that the value is close to minus one, which corresponds to the intercepts we defined
above.
In other words, we have recovered the hidden parameters of our data sets.