At this point, we've done most of the legwork to describe what we are about to implement in code.
Now we just have to discuss a little more about what the code is going to look like and then we can
look at the actual code.
So let's again recap all the usual steps in the context of our data and our new model.
So step number one is to load in the data for the coming example, will be looking at the classic amnesty
data set, which is a data set containing handwritten digits.
Our job is to accept as input an image of the digit and then classify what the digit is.
Since there are ten possible digits, zero up to nine.
This is a multiclass classification problem.
Also note that even though these data are technically images, they don't come from actual image files
like JPEG or pages, instead the amnesty to say it is actually already included in the Torture Vision
Library, which makes things a little bit more convenient for us.
Also, they are greyscale images, so we don't have to worry about color right now.
Later in the course, we'll look at how to load an actual image files.
Step number two is to build our model as per this section of the course, the model is a feat of forward
neural network.
Basically, it's going to be multiclass logistic regression with a few more layers before it all change
together sequentially.
Step number three is to train the model.
Now, this is the beauty of deep learning and deep learning frameworks.
There is zero additional work to do here.
All we have to do is pretty much the same training loop we have already seen, although there is one
small modification we are going to make in order to improve the training process called batch gradient
descent.
We'll see what that is later on in this lecture.
Step number four is to evaluate the model.
Again, this is exactly the same, you could say that these steps are model agnostic.
They don't care what the actual form of the model is.
Step number five is to use the model to make predictions.
Well, do something interesting in this section, which will be to look at which predictions the neural
network is getting wrong.
This might give us some insight on why the neural network is doing what it's doing.
So let's start by breaking down step number one, as mentioned, this data is going to come from the
torture library itself, which will keep things simple for us, actually, to be more specific.
It comes from a sister library called Torture Vision, which you normally have to install separately.
But since everything is installed for you in Google CoLab, we won't need to worry about that.
Torture Vision specializes in providing functionality for, you guessed it, machine vision.
This includes both helper functions and data sets of which will see multiple in this course.
Of course, one of these data sets is the famous amnesty data set, which is a standard machine learning
benchmark of handwritten digits.
This is an image classification problem.
So the data are images and the targets are what the image is of.
Each image is the same size 28 by 28 pixels, which is equal to seven hundred eighty four pixels in
total.
The amnesty to say it contains only grayscale images.
So the images are just of size 28 by 28 and not twenty eight by twenty eight by three, which is what
they would be if they had color.
A grayscale image is pretty basic.
Just imagine it like a matrix of numbers, the number in each element of the matrix tells us how dark
or how light that pixel should be.
So if you have a very small number close to zero, you'll get a very dark color like black.
If you have a very large number, close to 255, you'll get a very light color close to white.
When we call the function torch vision data sets that Imust and passenger train equal to true, we get
back a training data set object, which is similar to what we saw in Sijia Learn.
Specifically, if we call the data attribution, we get the input data, which are the images and if
we call the target's attribute, we get the targets, which are the labels telling us what digit the
corresponding image is of.
We can think of train data set data as XStream and train data set targets as we train.
So X train is of shape and by twenty eight by twenty eight, whereas Y train is a one B vector of length
N in our case and is equal to sixty thousand.
Correspondingly, if we call the same function, Tajh, vision data sets not missed, but we pass in
a train equal to false, then we'll get back the test data set as before.
We can call test data, set data to get X test and we can call test data, set targets to get Y tests.
X test is of size and test by 28 by 28 and Y test is a one Devecser of length and test and in our case
and test is 10000.
As a reminder, the image pixels are stored as integers from zero to 255, so we'll have to scale them
to go from zero to one.
This is because we like the input data into a neural network to be in a small range, as you recall.
In addition, the neural networks and other machine learning models that we have seen so far expect
their inputs to be of shape and body, which is a two dimensional array and is the number of samples
and the number of features.
But as you just learned, our data sets come in the shape and by twenty eight by twenty eight, which
is a three dimensional array.
Therefore we'll have to use the special view function in PI Torch to flatten the data into a two dimensional
array and treat each of the pixels as if they were an input feature.
This is thanks to my rule.
All data is the same.
We don't care if this input represents a breast cancer feature vector or image pixels.
Neuron that works will work the same way regardless.
Step number two is to instantiate the model.
So first, let's start with the equation that represents how we get from the input to the output in
our neural network.
If we write it out all in one line, you can see that it follows the pattern inside to outside.
That's just your basic order of operations for arithmetic.
But we can split it up into intermediate variables so that we can see each step one at a time.
So A1 is equal to one at times X plus one.
Zwaan is equal to Sigma A1.
A2 is equal to two times one plus two and then technically Y is equal to softmax of a two.
But remember that we omit this step in PI talk because it's already included in the lost function.
One important thing to note is that even when we use an activation function such as the rescue, we
often still write it as sigma when we write down the math.
So that's just something to keep in mind for the future.
So you don't get confused when you see it.
Basically, the important thing to remember is that each of these math equations is just an object in
pie talk.
So we have N.N. Linear 784 128.
And then you add a final and linear one.
Twenty, eight, ten.
As mentioned previously, Ruyu is a finite default choice.
But you might be wondering why did I choose the value one twenty eight.
And of course, there is no special reason.
This is just a matter of hyper parameter selection for which you should use trial and error as well
as past experience to guide your decisions.
You might also notice that oddly, I had to specify the number one eight twice.
This makes sense because if the output of the first layer is one twenty eight and that's the input to
the next layer, then the next layer must also have input.
Size 128.
But it also does not make sense in the sense that this is a totally redundant, for example, in intensive
Lotu or CARUS, you only have to specify this number once, and it also leads to more opportunities
for error.
If you specify different numbers, your program simply won't run.
You'll notice that the same theme will apply to later sections of the course as well.
And this gets especially complicated for convolutional neural networks.
Finally, there are 10 output's because there are ten classes which correspond to the 10 digits, zero
up to nine.
Step number three is training the model, this will almost be the same loop as we had previously, but
with one additional detail.
Previously on Each Epoch, we took the entire data set, made a prediction on it, and then calculated
the loss and did gradient descent.
But ask yourself, what if your data set is actually too large to fit into memory?
Previously we just had a few hundred samples.
Now we have 60000.
Today's data sets, such as the famous image that data set have millions of images, image that is approximately
one hundred fifty gigabytes, which can't even fit on some laptop drives, never mind fit into memory.
So clearly we need a better solution.
The solution is batch gradient descent, and the idea is this let's suppose we split our data up into
batches, then instead of having just one loop over each of the epochs, we'll have a nested loop over
each batch.
Then we'll do all our usual steps on that small batch rather than over the entire data set.
That includes zeroing the gradient, calculating the outputs for that batch, calculating the loss and
doing one step of gradient descent.
In fact, batch gradient descent is such a fundamental part of deep learning that the Pitchfork's library
comes with a data loaded class which comes with a batch size argument and acts as a generator over the
data set.
So although in my in-depth courses I will teach you how to write a batch gradient descent loop by yourself
in PI talk, we can just loop over a data load or object and it automatically yields batches of inputs
and targets.
The next question to consider is, why does this work, why is it OK to do any of a sent over batches
instead of the entire data set?
Here's some intuition.
Suppose we want to know the average height of all human beings on Earth.
Think about why that would be an infeasible task, just like how storing all of imaging and memory is
infeasible.
Well, there are seven billion people currently on Earth.
You can't possibly ask them all how tall they are.
Therefore, we have to do what all scientific experiments do.
Take a random sample from the population.
This is what scientists do when they want to test drugs or when they want to do a psychology experiment.
They can't possibly ask everyone in the world to participate in their experiment.
So they have to choose just a small sample of people and generate statistics based on this sample.
Only at the same time do we not expect that the sample represents the true population.
The answer is usually yes.
Do we expect that our measurements on the sample will be close to the measurements we could theoretically
take on the entire population?
The answer is yes.
The same thing happens when we do batch gradient descent, even though we are only looking at a small
sample of the data at any given time.
We do expect that the loss and its gradient will be representative of the entire dataset, which itself
is what we expect to be representative of the true distribution of where the data set comes from.
Furthermore, on each epoch we still see every data point, so we don't miss anything unlike real world
population samples.
Finally, step one, step five are to evaluate the model and make predictions using the model, as we
just discussed, with large data sets, it may not be feasible to load in the entire data set into memory.
Therefore, we can't just do model of inputs and pass in the entire data set all at once.
And so this is yet another place where we have to make use of our data loader objects due to the fact
that our predictions come in batches, we won't just have a single array of predictions to compare to
the targets like we did before.
One way we can get around this is to just apply the definition of accuracy directly and calculate it
ourselves.
For each batch we get the predictions and the targets.
We don't want the accuracy on this batch only, but what we can do is count up the number of predictions
we got.
Correct.
If we tally up how many we got correct and how many samples we saw in total, then we can just divide
the number correct by the number total once the loop is over.
And that's our accuracy.
Another complication is how we actually make predictions in the first place.
Recall that we are no longer doing binary classification.
So it's not a matter of just checking whether the output is greater than zero or less than zero instead
of the output is of size.
Work is the number of classes.
As you know, these K outputs will theoretically be passed through a softmax function to give us the
probability that the input belongs to each of these classes.
But do we need these probabilities?
Well, just like with binary classification, the answer is no.
Why is the softmax function called softmax function in the first place?
It's because it's kind of like the max function, except that instead of just telling us which element
is the biggest, it just weights each of the elements relative to each other.
In other words, a softer version of the max.
In other words, the maximum value before we do the softmax will still be the maximum value after we
do the softmax.
It's just that the softmax happens to map those values to probabilities.
Therefore, we just need to pick which of the key outputs has the maximum value and not worry about
them being probabilities or not.
In Pittsburgh, this is accomplished with torchlight, Max, which returns both the maximum value and
its index.
And because we're doing classification, we're actually interested in the index and not the value itself.