So in this lecture, we'll be doing code preparation for eons, which will introduce the syntax will
be using in the next lecture to recap the previous lectures, you recall that a neural network is simply
a composite function where we just repeatedly apply w transpose input plus B with some activation function
F on top.
As you recall, we already know how to implement this function in TensorFlow, which is just a dense
layer.
And as such, building an a& model is very easy.
It's just like the previous section, except that we have more dense layers.
You'll notice that in this case, we also specify the activation function, which is most commonly the
Rel U.
Furthermore, you'll notice that for each of these dense layers, we need to specify their output size.
This is what we've been calling the number of hidden units, and this is called a hyper parameter,
meaning that you don't compute this value.
You just choose this value to suit your needs.
For example, by picking the value that leads to the best out of sample performance, well, discuss
how to choose hyper parameters elsewhere in this course.
Finally, note that because we'll be doing multiclass classification, the final dense layer will have
key outputs where case the number of classes along with a soft max activation.
The next step is to discuss the compile method.
In this case, we will still pretty much always use the Atom Optimizer, and we will still always want
to know the accuracy metric.
What changes, as you recall, is the loss, which is now the categorical cross entropy or the sparse,
categorical cross entropy.
The difference between these is in how your targets are represented.
If you want to use the regular categorical cross entropy, then your targets will be a matrix of shape
and by K where and is the number of samples and case of the number of classes each row will contain
a single one.
And the rest all zeros denoting the target class.
So, for example, if y three five is equal to one.
This means that the third sample belongs to Class five.
This is assuming that indexing starts from one, which is not the case in Python, but is the case in
conventional math.
On the other hand, if you use the sparse, categorical cross entropy, then your targets will be a
one dimensional array of length, then containing the integer representation of each class.
So for example, if y of three is equal to five.
This means that the third sample belongs to Class five.
Again, this assumes that indexing starts from one.
Now, clearly, the sparse, categorical cross entropy is more efficient.
So unless you have some reason not to.
The sparse version should be used.
The reason we like to think of the non-sports version is that it's more intuitive in how it works.
Note that when we use this, the target matrix has the same shape as the neural network output, which
again is end by K.
As a side note recognized at the end by a target matrix is called a one hot encoding of the end length
target vector.
This idea of one heart encoding will be helpful when we discuss embeddings as well.
Now, as with the binary cross entropy, it's possible to combine the final soft max activation with
the loss so that we only need to compute the logics.
Again, we like to do this because it's more numerically stable.
The soft max, as you've seen, contains exponentially.
And the cross entropy contains logs, both of which can make things either explode or go to zero.
But if we combine logs with exponential those, then they cancel each other out.
In this form, our final dense layer has no activation, but still has output size K.
Our laws function now cannot be specified as a string, but instead will create an object passing in
the argument from logic equals true.
Finally, note that just as with our previous examples, the rest of the code will not change.
So calling fit is the same plotting the last per epoch is the same, and making predictions is still
the same.