OK, so in this video, we're going to discuss how to actually build a neural network using tensor flow
at this point, you know, the theory behind neural networks, but you do not know how to practically
implement them.
So this lecture will go over those details.
Now, obviously, you're going to see this in the lectures also, but by seeing everything together
now, it'll give you two exposure's, which hopefully improves your ability to retain this information.
So let's start with the fact that for Tenzer flow to Google recommends using the API, I sometimes encounter
students who wonder why we are using Keris when we are supposed to be using sensor flow to.
But this question is misinformed.
Tenzer flow is designed such that the proper thing to do is to use the API.
The only time you should not be using the carers API is if you cannot use the carrots API to do what
you want to do.
And remember, this is advice from the developers at Google who invented flow flow.
If anyone knows how to use tensor flow, it would be them.
OK, so with that said, at a high level, the Keris API is very similar to learn the basic steps are
as follows.
The first step is to create the model.
We'll go through the details of how this is done soon.
The second step is to fit the model to a data set.
The third step is to make predictions using the model.
OK, so this is just like cycle learning that you call the fit function when you want to train the model
and you call the predictive function when you want to make predictions.
So it's almost exactly the same as before, except with some minor differences.
OK, so for reasons which may not be clear yet, we are going to save how to create the model for last,
and this is despite the fact that it will be the first step in the code.
What we will discuss first is how to train the model.
So imagine that you've just created your model instance and this is saved to a variable called model.
The next step is to call a function called compile.
This does a few things.
Firstly, it tells your model which laws function to optimize.
As you know, the standard for regression is the squared error or equivalently the mean square error.
Basically, the act of learning in the neural network is really just doing calculus.
Given some lost function, we want to find its minimum.
Now, it's not my goal in this lecture to discuss this in any detail, but you might want to check the
lecture.
How does a neural network learn if you want to learn more?
That being said, the method of minimizing this lost function must also be specified.
The typical default in deep learning nowadays is called Atem.
We want to explore the different kinds of optimization methods in this course, but you're encouraged
to check out extra reading text.
If you want to learn more specifically, look under the heading.
How does back propagation work?
Another option we can specify in this function is a list of metrics.
Basically, tensor flow will track these metrics during training so that you can plot them at each step
of the training loop.
For example, if you're doing classification, you might want to track the accuracy rate over time.
If you want to see an exhaustive list of possible metrics, I'd encourage you to check out the documentation.
OK, so that's the compile function.
Now, the reason this function is called compile is really historical.
But again, I'd encourage you to check out extra reading dot text if you're curious about learning more.
OK, so what happens after you call compile, the next step is to call fit.
So normally with psych you learn, you pass in your training data and that's it.
But for Tenzer Flow, you have a few more options.
As mentioned, training neural networks as an iterative process.
That means something is going to happen inside a loop, specifically back propagation.
Because of this, we would like to specify how many times that loop iterates.
We call this the number of epochs.
So by specifying this number, you can control how many iterations you train for.
Now, typically, you can guess this value beforehand.
You simply look at the loss per iteration after you are done training to check if training has converged.
If not, you go back and tweak your settings until you get a nicer looking curve.
Alternatively, if you want to be more fancy, you can follow the laws per iteration during training.
Another thing you can pass in is your validation data.
In other words, out of simple data.
So when you plot your metrics after training, it will show both the train metrics and the test metrics
over time.
This will let you decide whether or not your model is overfitting.
Now, please note that this list of arguments is not exhaustive, but these are the basics, if there
are any more that we use in the code, you can simply learn about them on the fly.
OK, so after training, it's common to plot your loss per epoch, as mentioned, this is so that you
can confirm everything worked as expected, and if it didn't, you can go back in the and things further.
Basically, your fit function is going to return an object.
This object will have an attribute called history.
The history attribute is basically a dictionary that stores your metrics at each iteration of training.
So, for example, I can access the train lost per iteration by indexing the dictionary.
With the string loss, I can access the validation loss per iteration by indexing the dictionary with
the string value loss.
And also, if I passed in inaccuracy as a metric to the compiler function, I can do a similar thing
with that metric or any other metric as well.
OK, and so after training your model, you can call the predict function, the predictive function
is unremarkable because it works in exactly the same way as I could learn.
So now that we understand the whole workflow of building a model, compiling it, training it and making
predictions, we can begin to look at how models are actually built.
In fact, this part is what differentiates each architecture from the others.
So when we discuss CNN, Zain Arnon's, all the steps we previously discussed still apply.
Those won't change.
The only thing that will change is how the model is built.
In fact, in this section alone, we will discuss several different Annand models.
So the first model we will discuss is the basic feed for day and then which works for tabular data and
univariate time series.
This is thanks to our rule.
All data is the same, which you already observed in the previous section.
So basically, this is just four lines of code.
In the first line, we create our input layer.
This isn't really a layer per say, but it defines your input shape.
As you can see, we pass in one argument called shape, and the value is a tuple that specifies the
shape of each dimension of your input.
Note that this must be considerable, like a tuple or a list.
So even if your input only has a single dimension, you cannot pass that in by itself.
The second step is to create a dense layer, which is basically the linear expression we keep seeing
here.
We specify the number of hidden units and the activation function.
Note that at this point you can chain more dense layers if you like to make a deeper network.
Again, I'll remind you that choosing the number of hidden units, a number of hidden layers, is a
matter of trial and error.
There is no magic formula.
Notice this convention that we simply use the variable X to represent the output of each layer.
There's no need to give them special names unless you have some reason to.
The next step is to define our final layer, where we specify the number of outputs, what you have
called K. on the slide.
So, for example, you'll have K equals one for a one step forecast.
OK, equals three for a three step forecast.
And note that for all three cases, regression problems and binary classification problems and multi
class problems, they will all have this structure.
You may have expected to see the sigmoid or softmax activation, but we'll see a better way of dealing
with that soon.
The final step is to construct our model using the layers we just created, the interface to the model
constructor is pretty simple.
The first argument is the input and the second argument is the output.
And as a side note, notice that if you wanted to build just a regular linear model, all you would
have to do is simply remove the first dense layer, although that is not of interest at this time.
Another thing I want to make a note of here is that this is called the Kharas Functional API.
The reason it's called this is notice the syntax is a bit strange.
I'm creating objects like input dence and so forth, but I'm also able to use these objects as if they
were functions.
So you can see here that after creating a dense object, I can follow it with round brackets and pass
in its input as if it were a function.
So that's what we mean by functional API.
This method is very useful, especially when you want to make a neural network with branches, for example,
with multiple inputs or multiple outputs.
OK, so at this point, you must be wondering, how can it be that we do not need the sigmoid or a softmax
activation?
The reasoning for this is far beyond the scope of this course.
But again, you're encouraged to check out extra reading text if you want to learn more.
Basically, there is a mathematical reason that we do not want to apply the sigmoid or softmax directly.
As you recall, taking exponents is numerically unstable because the values will get very large.
So we want to avoid that as much as possible.
What you effectively end up doing is combining the final activation with the loss function.
So for regression, nothing needs to change since it's just linear regression.
You have your linear equation and then you apply the mean squared error for binary classification.
You'll have one output and you'll use the binary cross entropy.
You then pass in from logits equals true so that the model knows you have not yet applied the sigmoid.
Basically logit is a fancy word for the value you have before you apply the logistic function for multiclass
problems.
You'll have key outputs and you'll use the sparse, categorical cross entropy again.
You'll pass in from Logits equals true for the same reason as before.
OK, so now you know how to build a feel for neural network for three different tasks.
One important lesson to remember is that the main steps do not change, even though the model is different.
So later on, when you learn about different models, we still have the same compile fit and predict
steps.
This is helpful to know so that writing the code is not a matter of learning everything from scratch
each time.
Only the few lines in the middle will change after you declare your input and before you finish instantiating
the model objects.