Hello, everyone, and welcome to this amazing session, which we are going to see how to integrate
tensor bot callbacks with TensorFlow.
And this session we are going to look at how to log in information from our training process or from
all different experiments into tensor bar, how to view model graphs, how to do hyper parameter tuning
with tensor bar, how to view distributions, histograms time series, how to log image data like convention
matrices, RC plots, and finally, how to do profiling with tensor Bot.
In one of our previous sessions, we saw the importance of working with callbacks as the parameters
to modify certain key information during the training process and also store certain information and
do certain modifications during the training.
That said, we have this tensor board callback right here, which we spoke of last time but didn't really
get into.
And so in this session we are going to go in depth and see how to make use of tensor board to visualize
on a web interface vital training information.
So notice also that this tensor board callback is going to be used in a similar way to the way we had
done with the previous callbacks.
So yeah, we define the callback and then just here you see in this callbacks argument we pass in the
tensor board callback right here.
So let's go ahead and copy this out.
We have this we could just copy from your we've copied this to clipboard and then now we go ahead and
see how to pass this in our training process.
So we see exactly how it works.
So yeah, we open up this callbacks, you see the different callbacks we have created previously.
And then now we're going to include our tensor board callback.
So just, yeah, let's add this text and adequate.
So we paste this out.
Then let's take all this off and simply go, Let's run.
The cell looks fine, and now let's move on to our training process here.
Are we going to have the callback?
So we have the callback equal tensor board callback, and then we're ready to train our model.
So let's check on this.
We had gotten an error, unexpected keyword argument callback.
Oh, okay.
You should have callbacks.
They'll run that again.
Now, while our model is training, you'll notice that there is this locked folder which has been created.
And the reason why the logs folder has been created was because we actually specified that in this tense
about callback call.
So here we have this log dire log directory argument and we're going to store in some information or
some training information.
And as this training information that turns about, we'll use to display very important training information
on the web interface.
Now, you could click open here and you'll see that you have these two folders in this train folder.
You see you have this information concerning the training data which has been stored here and information
for the validation which has been stored right here.
The next thing will do is copy out this here, this command, and then run it just below.
So let's get to where we have visualizations.
Like we reduce this one and we have the visualizations here.
Now we're going to have this.
We're going to see how we're going to replace this visualizations with our tensor board visualizations.
So it paces our we have ten.
So BART log the pattern logs.
Then we created this log dye variable here which takes in our path.
So let's get back.
And then instead of putting our path directly, we just have actually just log there.
So we have the log DA.
Now, before running this, we are going to add this code cell and then load this tensor board notebook
extension.
So here we have load extension ten saw board, we run this and then you see already loaded.
We are loaded this already.
Anyway, we have that and then tensor board now.
So we run this and then we should expect to have some interface which contains all our log data.
No dashboard active for the current dataset.
Check out on scalar is what we have.
Anyway.
Let's, let's do this.
Let's check on this.
Let's have logs logs.
Run this again.
Okay.
And there we go.
We see now that we have this interface which pops up or what do we see here?
We have this logs.
We have both train and validation.
You could pick this out last.
You could take only the training data.
You can see only train data.
And then here you have this killers.
These killers basically is.
All this information which we pass in here, like the loss and the metrics that we had defined these
metrics here.
So we we are going to get all these metrics information.
So unlike previously where we had to manually do this step and do this step for the loss and the accuracy,
now this is done automatically, so let's run this and then we will compare what we get here with what
we get from ten.
So bored.
You can see here we have this loss and then we get back to ten.
So bar we have this level of skills.
Let's let's reduce this accuracy, Let's view the loss first.
So let's reduce this.
Okay, We check out we have this epoch loss now this is what we get for the loss.
Let's include the validation.
So you see, you have this plot right here clicking on this.
You could also expand.
So you see, this is the plot we get.
Now what do you notice?
You notice that it's exactly the same as what we had here, but this time around, we didn't have to
write any code.
All the information was automatically locked in this file here and tense about took care of the rest.
So that's how this works is really very interesting.
And it's a kind of tool you want to master how to use.
Because when working on different machine learning experiments, you wouldn't want to always have to
log all these values by hand or manually.
You want to have this done automatically.
Now, one of the interesting point is you have all the metrics here.
You just have to select any one.
So let's look at the accuracy which we we've seen already.
We can check out this.
Let's click here.
Okay.
So we look at we see this accuracy and then let's scroll down.
You see, you could compare it with what we have here.
See, we have this zip around.
This should be the ninth epoch or let's say it epoch.
Anyway, let's come back and check here.
Let's scroll up from here.
It's actually an interface in this other interface.
So we have this, you see this nine epoch.
What do you see here?
You see notice on this here you will have this name train, smooth add value.
What do we have?
Step nine, Step the time and that's it.
Okay, so there we go.
We see how we could plot all this automatically, and then you could get at any point so you could go
through each and every point and then get all the exact values.
So that's how we look at this.
Let's reduce this.
We could take now, let's say precision so you could see, monitor the precision.
You could also check out a number of false negatives.
You see how this as you keep on training, a number of false negatives keep reducing.
And then the false positives.
What do we have here?
False positives.
That's loading.
While that's loading, let's scroll down and we have the loss.
We've seen this loss already.
We have let's look at true negatives.
That's loading true positives.
There is a plow we get for the true positives and the true negative.
And then the section where we have this evaluation, evaluation evaluations, practically the validation
metric and loss that we plotting against the number of iterations.
So that's why we have here is just like the validation accuracy versus the iterations.
If you take off the train, you see nothing really changes here.
But when you do this, you see all that goes.
So that's our validation and we could monitor this and observe that the highest accuracy we have is
94.09% or now it's 94.21%.
Scroll down.
We have precision.
I was a precision.
Here is 93.6 know the value is 94.39%.
As of now, we've been able to log this information just from compiling our model.
So because we pass our loss and the different metrics, we're able to log the information and visualize
it on TensorFlow board.
But there are other possibilities that is, it's also possible for us to log information manually instead
of just logging only this information.
So what we could do is log, for example, image data.
We could log even this different learning rates here.
So we are going to log or we can log the learning rate values for each and every epoch.
And this gives us that freedom to log in just any kind of scalar or quantity we want.
So here we have this metric.
We're going to create a metric DA metric directory.
This is going to be logs.
And then here we have metrics.
Then what we're going to do now is create this train writer because now we are doing this like manually.
So we create our train writer.
We have TFW that summary that create file writer.
And then here we.
Have our metric directory.
So that's good.
We have our trained right to create it based on this metric directory, which is going to be in the
logs.
So in those logs we're going to create a metrics directory.
So that's it.
We have our train rider, we could run the cell.
Then just in here we have width.
So once we're done with this with with train writer as default, we train writers default.
We want to have this locked.
So we have to have the summary that's killer.
And then we specify that we're dealing with the learning rate.
So our learning rate and then we're going to pass in the learning rate actually.
So your personal learning rate and then we pass in the epoch.
So this is like as you could see here, you have data look at look at this pop up.
You see here you have data and then here you have steps, all right, a step.
So you have the name of this killer, you have the data and you have the step.
You also have the description.
So yeah, we have the name which is learning rates.
We have the data, which is this learning rate.
And then we have the step, which is the epoch.
Now we have to do this like with this, we have to put it in each and every one of these since either
we get into this or get into that.
But to avoid writing this twice, we just have to set we could set a learning rate.
So in your we define this learning rate learning rates and then year two, we have our learning rates.
There we go, we have the learning rate.
And then out of this we return our learning rate.
Okay, so that's it.
Now we return the learning rate and then we are also logging this data.
So yeah, we should change this and have learning rates.
So that's it.
Now we have the set and then we could go on and train before training.
Recall each time you want to log in this kind of data, all this kind of custom data, first thing you
do, you create your writer as you create this file writer.
After creating the file writer based on a given directory that you set, you now go ahead and then put
this in this train writer scope right here.
So with this, we can run this.
Now.
We've run this already.
We could run this now and then since these are scheduled, our callback will have to add this in our
fit.
So let's go ahead and add it in us.
Our feet matter here.
Let's take now just five epochs.
So yeah, we have ten.
So board callback and then we have scheduler callback.
Okay, so we run this now and see what we get.
That's training.
Our training is now complete.
Let's go ahead and check this here.
We have our logs.
You see we have this matrix and we have this locked next step we go to visualize.
So here we have the stance of board and then here let's do anyway, let's let's run this first so you'll
see what we get.
As you could see right here, we now have this learning rate which has been logged and which we can
visualize.
So you see as we go from this epoch to this epoch to this this, if we set this at zero smoothening,
we have this Now, the reason why we after this we don't get any value is because of what we have here.
So we send the learning rate into this tensor and what we should be doing here is getting that non pi.
So we should have this learning rate, right?
Your learning rate equal to learning rate because this is going to be converted into a tensor.
So we have learned to read the non pi and that should be cool.
Okay, we run this again and that should work.
And then to, to view or to have image response, let's set this to just one.
So if the number of epochs is greater than greater than one, then we would have the learning rate being
modified.
So let's get back again to training.
This time around, let's say we just have three epochs, okay?
This time around we have this actual value locked and so we get back.
Let's reduce this custom training loop, we get back to our visualizations.
We will run this again.
Getting back here, we see we start with this loss and then this drops after this second epoch right
here.
As you may have noticed, each and every time we run new training process, the previous values are
the previous locked input data is being deleted.
So what we could do now is we'll modify this file name.
That's this log file we're using your where this folder name now depends.
On the current time.
So yeah, we're going to have date time.
Date time.
That date time now and then we get a string string from the time and then format this output.
So here we're going to have.
Percentage the day, percentage the month, the year, and then we specify the exact time.
So you're going to have the are the minute and the second.
Now let's go ahead and import the time up here.
We have date time.
So yeah, we import the time.
Simply that should be imported.
Okay.
So we run that and that should be fine.
Okay.
We've imported that time.
Let's get back and then let's print out this log dye right here.
Let's have lock dye, print it out, see what we get.
You see, we have this logs, and this is actually our new folder.
Now, if you print if you print out this again, you see, we're going to have a different folder.
And this is important because each and every time we do not need to cancel our previous runs from here,
let's take all let's take this off.
And then it set this as our current time.
So current time current time is equal to this.
Take this off, here's our current time, and then we have this plus current time.
Okay?
And then, yeah, we do the same.
We have this plus current time.
Let's take this off and then have the current time year.
Current time.
If we get back to this training, we can run this again.
Training now complete as we could see here in this locks, we have this new folder created which is
depending on the time in which we decided to do the training.
And then in this matrix we also have this.
But what we want to have is actually just this one photo which contains the train validation and the
matrix.
So we should modify this right up here.
So instead of having this matrix before we should take this off, we should take this off and then added
it later on.
We have plus and then we add this.
Okay, so we have this slash and then slash, then we should take this now.
So that's it.
We have recreated this and then we will rerun again to avoid this kind of error.
So let's go ahead and retrain our model.
Let's let's have it to be here.
That's fine.
Let's say two epochs and then I'll run that again.
Okay.
The training is complete.
Now you'll see that if you open this up, you have trained validation.
This is what we had previously, but now we have matrix train validation.
This is actually what we want.
We want to be able to log all this into this one directory and you now notice how we do not have to
erase previous logs.
So let's go down to run this again.
We run this.
Take this off.
Okay, here we go.
We have this information now locked.
You see, we could take all the previous locks out and focus on just this lock here.
So let's have this one.
This should be zero five.
The same entry, seven one.
So we want to focus on this one, which ends with seven one.
And here we go.
So we have this matrix, this strain and this validation for this.
We are not interested in log in this.
So that's it.
Let's take this off now and then get back.
So you see here we have the learning rate, we have the epoch accuracy, we have this true negatives,
we have the recall and that's it.
So yeah, we've seen now how to create this directories which are dependent on the current date and
time.
Now the next step we'll be doing is how to actually do this logs without or when we're doing a custom
training.
So let's get back to where we did this custom training loop.
We had this custom training loop and then we have this feed method which comes directly with TensorFlow.
So what if we now try to use our what if we actually use the custom training loop and we do not have
the possibility of just simply saying, okay, callbacks tensor by callback and then the job is done?
And what if we just have this custom training loop?
In this case we are going to use exactly the same process we've just followed here.
So we're just going to create this file writer.
So let's copy all this.
And then just as we did here, actually, we're just going to put right in this scalar values and then
like create this killer, put in the data and then specify the step.
So that's basically how we're going to function.
Now, let's get back to this custom training loop.
We're going to add this code cell.
And then in here we have current time, as usual, we have now let's call this custom directory.
We have logs current time, and then we have custom.
Let's call this custom trained writer.
Let's call this custom train writer.
Custom train writer.
And you could also define a custom validation writer.
We could have that too.
So let's have your custom.
Custom direct and in your we you specify also train.
So notice that since you are using this feat what we got was automatically we automatically got this
train and validation.
So what happens in the background is this two firefighters are created that is the train and the validation.
And we're just going to do exactly that.
Your we have custom and then let's say we have custom train directory, custom train and then custom
validation.
So we have that.
And then here we have custom validation.
Now we specify our writer, we have custom train and custom validation.
Then here we have custom train.
Custom train, custom validation.
Okay, so I think this is okay.
We could now run this and then let's copy out this code we had put out your in the section and the scheduler,
so let's simply copy this out.
You see how easy it becomes when you have already done this.
So you're you, you now have to say instead of just only printing this out, you let's have this.
So with our let's cop let's get a name from here with our custom train writer.
And then here we have with our custom train writer.
Custom train writer.
We have the loss.
So we have the loss.
Let's call it train loss.
We have the training loss.
We have the data.
The data is now this loss here.
It's this loss.
So we have the the data which is passed, which is now the loss.
And then the step is the epoch we have here.
So that's it.
Okay, we've logged this.
Let's now go ahead and lock for the accuracy.
So we paste this out and then we have training, accuracy, accuracy and then the accuracy.
So we kind of paste out, we will see metric the results metric that results, okay, Metric the results.
And then we have the step specified.
So this is for the training process.
We could separate this block and that's it.
Okay.
So we've done this.
We now simply copy this out and then do the same for the validation.
So in here, instead of writing this out like this, we could simply put out your custom.
Val We have custom.
Val, take this off here.
We have validation, validation, loss.
Here we have the loss.
But this is loss of our loss.
Wow, that's fine.
We have metric.
Wow, metric.
Wow, that's fine.
We have validation, accuracy, validation, validation, accuracy.
Now you have this to take this off, and then we have the Val.
Okay, so that sounds fine.
Everything looks okay.
We could run this here.
So let's run this, We run this and then we run, learn, and then we start with a training.
Training now complete.
Let's go ahead and see what we have.
You could check out this log.
You see, we have our values now locked in your custom train and validation.
So we have this locked.
We now go ahead and rerun the stencil board.
As you could see, you have all this values here.
You could, as usual, let's take this off, toggle around runs, and then let's pick this very last
one.
So we picked this last one and also pick out this one because this is a train and this is the validation.
Then we come right here and check out the training accuracy train loss.
There we go.
We have train accuracy.
We have train loss.
If we if we do this, you see, we have okay, we have the train accuracy, we have the train loss,
we have the validation accuracy and we have the validation loss.
Now, if we want to take off all the information stored in the locks, you could have this command.
So we remove all this information and then we specify the locks.
So we run this and then open it up this.
You see, you don't have that locked folder anymore at this point.
We'll go ahead and see how to display image data with Tensor Bar.
So unlike previously where we've been displaying information like the loss, the different metrics,
now we'll see how to implement or rather we're going to display image data like the confusion matrix
we had seen previously.
Let's get back here.
We have this confusion matrix right here.
And what we'll do now is after each epoch we are going to display this conversion matrix with tensor
board.
That said, we're going to copy out all this code we used in displaying this conversion matrix right
here.
So we have this code and then we have this log images callback right here with this on Epoch end method.
And then in this method we are going to paste out this code we used in visualizing the conversion matrix
previously.
So here we have this label input right up to this we have the conversion matrix based on the stretch
threshold, and then we're going to visualize this conversion matrix.
But now since we're working with a callback, what were we doing is at the end of each and every epoch
we are going to display this with Tensor Bar.
Now that this is set, we are going to for now, we've actually just been able to visualize this.
But how do we put this?
How do we make this work with Tensor Board?
What we're going to have here is we create a buffer.
We have this buffer IO dot bytes IO here we have bytes IO and then so that's how our buffer we are going
to save this image, the compression matrix image in this buffer.
So we have this plt that save Feig and then we save it in that buffer and that will specify that the
format should be P and G.
So we have the PNG format and that's okay.
So now we have this buffer.
We've saved that into our buffer.
The next step we will take is create an image of this buffer.
So from here we have this image.
Let's take this up, we have this image, we use the TensorFlow image, decode PNG method, which takes
in this buffer.
So we have our buffer get value number of channels equals three.
That's it.
We have this image and then once we get this image, we then write this in tensor bar.
So we have this image writer which we've created right here, similar to what we've done already.
We create, we use this create file writer, let's modify this and have your image directory.
So we have the image directory and then we create this file writer, or rather we create this image
writer.
So from with this image writer's default, what we're going to do now is instead of having this summary
dot scaler as we used to have here, now we're going to use summary dot image.
So you could see here that tensor BART permits us not only input or write skills but also images, and
that's basically all we needed to do here.
So let's have this and then run this cell so we make sure we have this run, we run this, we run this,
and then we can we have this log images call back here.
Copy that.
Now that's copied, we have reduced.
This one.
And then right here we run this matrix and then compile and run this.
We get this arrow where we said, Oh, well, we have this valley error, no step set, so let's get
back to this callback and then we specify the step.
So right here we have the step, step equal to epoch.
So we run that again and this should be fine.
Trend is going on.
And then the image data has been locked into ten.
So bored at the end of each epoch training now done.
We could go ahead and then run this on ten.
Subodh Once training done, we could now visualize this conversion matrices on tensor board.
So we run this two cells and here is where we get you'd see your step zero, step one, and then step
two.
This is because we actually run this for three epochs so we could notice how let's come back to the
top.
We notice how here we have 52 three and then as we keep training, this drops to 35.
And then finally here, let's scroll down a little.
Finally, here we have 240.
So this tells us that the last epoch wasn't helpful in improving the number of false negatives you could
see also, even with the validation that, yeah, we had 16 false negatives, eight false negatives,
and then we this value rows up to 100 and 111 false negatives.
So it's kind of similar to what we have with the test data, which is exactly why we're logging in the
tensor bar.
And now that you know how to lock in image data with tensor Bot from the example on the conversion matrix,
what you could do is lock in directly this rosy plots.
You could also lock in data like this one right here where on the test data you're going to put out
the actual value and what the model predicts.
And so that's it for this section and log in an image there.
Now let's move on to visualize and model graphs with Tensor Bar to visualize a graph.
We're going to rerun this command to delete all the logs we've stored so far, and then we run this
tense about callback once more.
So we have that.
And then let's get back to metrics.
We run this and that's fine.
Now we have the training done.
Let's go ahead and rerun this again.
So we run this to cells again and as expected, years where we get.