So in this lecture, we'll be continuing our discussion about TensorFlow syntax specifically.
We'll be looking at classification with real data instead of regression with synthetic data.
Now, just in case you're curious, students taking a more naive approach sometimes ask me, why do
we use synthetic data?
The answer is that this is how we check and verify that our model is working as intended.
As you saw, we were able to check whether or not our model came up with the right answer because we
knew the right answer.
This is unlike real world data where you do not know the answer, and therefore you cannot check whether
or not you are correct.
In any case, the next notebook will look at binary classification.
Some examples of which are spam detection and sentiment analysis.
As you recall, our data will be in the form of an input x matrix of shape and by D and a target y vector
of link.
Then note that because we'll be discussing binary classification.
This target matrix will contain only zeros and ones.
The X Matrix will be a TFI Taf matrix for now, but you'll see how we can get rid of this later in the
courts.
So let's discuss the differences and similarities between this model and the previous model we looked
at.
Firstly, notice how they both start with an input layer because their inputs have dimensionality.
D We pass in a tuple with one element containing D into the argument for shape.
The next layer is again a dense layer, which, as you recall, represents in a fine transformation.
Simply put, it represents W Transpose X plus b here.
W is a weight matrix and B as a bias vector.
But again, note that this is the generic case because the output of this dense layer only has one element.
W can effectively be thought of as a D by one vector, and B can effectively be thought of as a scalar,
even though in practice it will be a vector with one element.
Now, as you recall for logistic regression, we also apply a sigmoid function after this transformation,
which maps the output to a probability between zero and one in TensorFlow.
You don't have to write this function yourself, but instead you simply pass in the string a sigmoid
ends of the argument for activation, which automatically applies the sigmoid after doing W transpose
x plus b.
Note that in deep learning, we call these functions activation functions, which explains the name
of the argument.
Finally, note that our instantiation of the model object is the same as before we pass in the input
as the first argument and the output as the second argument.
The next step is to consider the compile function.
The major difference here is that for binary classification, we use a loss function called the binary
cross entropy.
In this course, we won't discuss the details for why this is the case.
But note that if you'd like to learn more, you can check out the resources in extra reading data to
see.
I've also taught this in my in-depth series of courses where such detail would be more appropriate for
the optimizer.
Note that we again choose Adam as a default, which seems to work just fine as with the loss.
There is more in-depth knowledge about this that have taught in the past, so you can check that out
if you're curious.
But it would be inappropriate at this point.
The final argument is called metrics, as you recall when we're doing classification.
What we're often interested in is the accuracy.
We want to know how many predictions did I get right out of all the predictions I made?
This is unlike the binary cross entropy, which is kind of a weird value, in contrast.
So when you pass an accuracy as a metric, this means that later on, when you call the fit method,
both the cross entropy and the accuracy will be computed.
As a side note, for those who are curious, I'll show you the expression for the binary cross entropy
in this case, why is the target and why?
That is the model output, which, as you recall, is a probability between zero and one.
You should convince yourself that when Y is equal to y had this loss, function equals zero and when
y and y hat are opposite.
These laws function approaches infinity.
What I mean by opposite is that Y is one and y had approaches zero or y a zero in a y had approaches
one.
So plug in some numbers and verify that this is the case.
Now, because of numerical stability, we will actually not use the syntax I just showed you that was
for intuition only.
In practice, we're going to do something a bit more complex.
Intuitively, we know that functions like the exponential in the log are unstable because they make
things explode or shrink to zero.
It turns out that the sigmoid involves exponentially, while the binary cross entropy involves logs,
as you just saw.
Now, as you may recall, the exponential is the inverse function of the log, so they effectively cancel
each other out.
In effect, there is a way to combine the sigmoid and the subsequent laws function that makes the computations
more numerically stable.
Now, luckily, you don't have to understand this.
You just have to understand the syntax.
So here's how the syntax changes in this form when we create the model, it will look exactly like linear
regression without any sigmoid.
In other words, the model is now just W transpose X plus b.
We call these logics, which is connected with the fact that the sigmoid is sometimes called the logistic
function.
The next change comes from how we call compile, as you recall, when we specify the laws function,
we can use a string, but only when we're OK with the default values.
Now we are not OK with the default values because we have to tell TensorFlow that the laws function
should be combined with the sigmoid.
In order to do this, we create an object of type binary cross entropy and we pass in.
The argument from logic equals true.
So when we use this method, one minor difference is that when we call models that predict we will get
these logics, which can be any value instead of probabilities, which can only be between a zero and
one.
Finally, it's worth mentioning that all the subsequent steps, such as calling the fit method, plotting
the metrics over each epoch and making predictions all remain the same as before.
So these steps require no changes.