Hello, everyone, and welcome to this new and exciting session in which we are going to log our data
to one DB.
In previous sessions we already saw how to do one DB logs.
That is, logging our metrics like this.
Also, log in this confession matrices.
As we can see here.
We also looked at hyper parameter tuning dataset, versioning and model versioning.
In this session, we'll not just only log metrics but also log tables like this, just as we had done
before.
We are going to start with this weights and biases installation.
So we we have this PIP installed one DB that's fine.
And then we import one to B and then from one DB cross we import the one to be called back.
So we run this and then we go ahead and log in.
Now we've gone through this already in some previous sessions and here you just have to click on this
authorize.
So you get your, you should have this link or this code to copy from your.
Now if you do not have this this key, if you don't see this key to copy, then you would have to sign
up on the weights and biases platform.
So let's get back here.
We now paste this and then we press enter.
So that should be it.
Now with with that, we initialize this project and we call the project emotion detection entity neural
learning.
So we have that and that should be fine.
So you could change this to whatever name you want and then whatever entity you corresponds to to yours.
So that's it.
Currently logged in as neural and use this to force re log in.
Okay, So we have that and then we have this config here.
So we run that train and validation directory.
That's fine.
Now we go ahead and train our model and we include this one DB callback right here.
So all you need to do here is include this one to be callback and training can start now.
We just run this for say three epochs, just do that for three epochs and then let's just take a small
part of our dataset.
So let's take the small part.
So we have that and we run this Now, as we train this model, you'll notice that we are just a very
small part of our data set.
The hugging face model performs quite well.
As you see, we already have 77% of validation accuracy with just this small part.
Now the other point is we get back to one TB So here we could check out on our profile.
There we go.
You see projects, you select emotion detection, and then while this loads, okay, so we have this
and we click on this hopeful plant right here.
This is what we currently have and these are current run.
So you see, it shows us already this chart.
Here we have our topic accuracy, validation, accuracy, validation laws and all of that just by specifying
that one to be called.
So this is already information which has already been locked in.
And as we know, you could always get access to this any time since this information is locked on weights
and biases servers.
From here we see that the validation accuracy goes right up to about 79.63%.
So that's about 80% validation accuracy while the accuracy itself goes to 82.6%.
And you could check this here you see 82.6 and then you're seven and 9.63.
So that's it.
We're able to log this data very easily.
The next callback will be added here will be that of including coefficient matrices.
So yeah, we'll put log confusion metrics.
Okay, So we have this log conversion matrix and now we'll go ahead and put out a code for this.
Note that for the one DB callback we didn't necessarily need to pass in any information here, but with
this log of metrics, this is our custom callback.
We are going to write code such that each and every time we done with the training, we are going to
automatically generate this coefficient matrix based off what the model predicts.
Now recall that we've already treated conversion matrices, so you could click on this and then we'll
copy out this code, copy out this part which permits us, get those predictions and the labels, and
then we will not make use of this any longer as one DB as a method which takes in our conversion matrix
method which takes in those predictions and this levels and automatically produces the.
Coefficient metrics.
So let's copy this here and then let's get back to our callback.
There we go.
So at the end of each epoch here we get the epoch logs and then we, we, we go to the validation data
set and then we have our image and the image is partial model to get the predictions.
And then we have the levels obviously coming from this level right here.
Then from here we produce a simple list of the predictions and the labels.
So now once we have this list, once we're able to get this list, which we've treated already in some
previous sessions, you now just have this part so this you could check on the documentation for this
plots.
Let's get to the documentation, the want to be documentation.
You could check this out here as type in confusion matrix.
There we go.
We have confusion matrix want to be of that.
Let's check out this log plots right here.
Okay.
So yeah we yeah we in this experiment tracking and log editor we want to be log log major and objects
then log plots.
Okay so we have this we the RC curves or our curves and our coefficient matrices.
So we saw this already in some previous sessions so you could get back to looking at how to make use
of this one.
DB So that's it?
You see, we copy out this piece of code here where we have this coefficient matrix, we have all the
ground truth that's a levels and then we have what the model predicts.
Then we obviously have the class names, so that's it.
And then you see just want to be log and then you specify this string right here and then you pass in
the coefficient matrix.
CM Which is this, and that's basically what we are doing here.
So we just copy down that code.
So with this year we have our Y true spreads class names we should specified already and then we just
run this.
So we run this and that's fine.
Now the next thing we'll do is we are going to create tables and now we're going to see all how important
working with tables is as compared to just simply log in values Tables will permit us visualize our
data in a more interactive manner.
And you're you see all of this data visualization.
You have those log tables and we'll see how to create a simple table and log this.
So here we have the data.
You notice that we have different rows this row 100012 and three, and that you have the image and you
have the predicted and the truth levels.
So basically what we're doing here is we're taking this image.
Then we log.
So here's our image, then we log as predicted value.
Let's say, for example, class eight, let's say predict class eight, and then the true level is class
zero.
Then we take some other image that's a predictor, this one and yours one.
So basically, this is what we're doing.
We created now this table where we have different roles and different columns.
Okay, so that's it.
We see how to create this and we also see how to log this tables into one tab runs.
So getting back to the code here, you see we have this different columns, image predicted and labeled,
and then we're going to call this table of validation table.
Or we have those columns which takes in our columns here.
Obviously, you could add this, you could increase this and put whatever values you want there and
then fill that.
So we have this validation table, which we've just created.
And then now we're going to fill this table color row by row.
So for our first row, you see we have the predicted values, our model text in the image, that's it.
And we have the labels.
And then here you see for a given row we have one DB image, we pass in our image.
That's it.
We pass in the predicted scores, then we pass in the labels course to the corresponding to these three
columns right here.
And then for this validator table that we created here, we're going to add this data.
So we're going to do this for each and every element of our validation data set.
Now, here would decided to take 100 and then we'll get him back up.
What we'll do is we are going to somehow batch this, so we're going to change this batch size to one
because we want to be able to play around with a single validation element and not, say, for example,
32 of them.
So because of that, we'll just changes to one and we run this again.
This should be fine.
Okay, So, so we have this validation data.
That's fine.
And then you're, uh, training.
Okay, we run this again, and that's it.
We have our validation there.
Now, let's get back here.
As we said, we're taking just 100 of this.
So we take just 100.
Let's let's even reduce this.
Let's say 25.
Okay.
So you could take all the elements.
Actually, we just want to make this a bit faster so we have that.
And once now you feel this, you see this follow permits you to fill each and every row.
So you just keep filling rows in your table and that's it.
You could get this here, See this?
We you see you have this table, the columns and you have the data right here.
And then you just log this information.
So it's from this documentation, you could be able to understand exactly how we build this table,
although it's quite straightforward.
So from here we do want to be log and then we have our model results and we pass in the wild table.
So let's run this, let's get back here.
Let's run this to cells.
We run this to cells now and then we start with the training.
So we have the training now over and we could check out this custom chat, which is that of the coefficient
matrix you see here.
We could get the number of times, the model predictor angry when we're supposed to be angry.
Here is the prediction.
Happy when it's supposed to be happy, then your is prediction sat when it's supposed to be sad.
So we see this is 357 times for happy 876 for sad 682.
And then here we have the times when the model predicts sad when it's supposed to be angry.
So this is our coefficient metrics which we generate automatically using one DB conversion matrix method
and then we move on to this tables.
So here you'll notice that this tables contains a hundred different elements.
So you could keep going on to the next.
So we want to the next ten and so on and so forth.
So let's get back or let's, let's just work with this now here you'll see that with this table you
can actually like your you see, if you scroll, if you get get to this, hover on this and you have
this three dots, you click on that and you could group.
So check on this you see you group by image for this one.
What we have here is this predictions you see happy the level happy, sad, sad, angry, angry and
so on and so forth.
So this model predictions here are quite great.
Now, if we increase this move, you see we have this example here where the model or rather the true
labels is happy and the model predicts angry.
But when you look at this photo is it shows that this person is not necessarily happy.
So we understand why the model may be thinking that this person is angry.
And so with this one, they be tool.
You see that it becomes very easy to detect mislabeled images like this one.
Now, another way to get this more systematically is by getting here.
But since we do not have the group, we just start by grouping right here.
So let's get back here.
Click on this.
We group now with one group already and then we get to this level and then we group by this different
levels.
You see that here we have happy, sad, angry.
Now this is what the model predicts.
The model you see, the other model predicts mostly happy because obviously the level is happy group.
By happy level you're sad, you see mostly sad, angry and mostly angry.
Now, what is interesting to note here is for this angry.
Well, the level is angry.
The model never predicts happy.
Anyways, let's get back here.
You should be able to scroll to this.
You see, You should be able to scroll through all the different happy images and you should be able.
Now, let's look at this.
You should be able to detect cases where the model or rather the data set has been mislabeled.
Like you're this person isn't necessarily happy.
So already you could detect these kinds of miss classified or mislabeled data points.
Then we also have this image which is almost completely blurred out.
You could always check on those kinds of images and take them off, actually, because if your model
isn't going to be seeing this in real life, then it's preferable to take this off.
Nonetheless, if in the real world the model is going to be meeting this kind of images, then you could
allow this.
Um, let's keep going this way.
You could also check out for those sad images and be sure that this I actually saw people.
You're again.
You see this person actually smiling and is labeled as sad then for the angry you see, this little
boy was visibly happy but instead labeled angry.
So the want to be tables permits us check these kinds of errors in a very interactive manner.
Now we will again on group so we can group this and then we could take this out for the pretreated group
by pretty levels.
So this is what the model sees.
So the model predicts that this is happy, this is side and this image is angry.
You see, what you notice here is because this image recall, this image which we just saw, because
this image has been had been labeled as angry.
The model now predicts that this image is an angry image, which is not true.
And so when you deploy this now into production, you will find that this kind of images or this image
will be predicted as angry, whereas it's meant to be happy.
And the worst part of it is that even your evaluation wouldn't find this as an error, because already
the level of the class for this was set to be angry.
Then another thing you could do here, if you get your let's have this back.
Another thing you could do is you get to this column settings here.
Click on this.
You could get the column settings just by clicking on this column settings, this column settings,
click on that.
And then you're you select the labels.
So here you, you're picking all the labels where the predicted value is different from what the label
or rather what the model predicts.
So here we could see row predicted.
So when row level is different from a row predicted, we and that's it.
So we have that.
So press enter.
And then what we notice here is we have this true, this false and and that.
So basically the reason why you have this true and this false is the true is for when let's get back
to this year.
The true is for when the road level is driven from the road predicted.
So this is this are the wrong predictions.
So what we could do here is we could modify this to equal so that equal, that's fine.
They should be reset.
So you see.
Okay.
So you see now it's true, meaning that this false is when the prediction is not the same as the level.
So that's it.
We could increase the size of this table.
Let's get to this.
Increase the size and basically that's it.
So you see the greens are for when the coincide and the reds are for when.
What the model predicts is different from what the model ought to have predicted.
Now we could get again to this top and then check this column settings and then take this off.
So you see, you could simply take that off.
Okay, So we have that reset again and you now have your levels as you had originally.
You could do the same for the predicted.
Now you could also get your see this, click on this and then you you check the row row.
We're trying to get all the rows where the row label is different C different from the row C type row
predicted.
So that's it.
Because if you say row label from a row label, obviously you have no row.
So let's test that and you could see you could apply, you should have no element in your C, There's
no element that's not actually possible.
Now let's take this off and now we take row predicted.
So row predicted and then you apply or you apply both know, um, some connectivity issues.
So yeah, we apply we check out where the, the the predicted event from the label.
So yeah we have this your C happy and the predictors either side or angry.
So you are the wrong predictions.
So with this you could check out and see the kinds of images which the model finds difficult or find
difficulties in correctly predicting.
So like with this one you see happy this the label here is happy, but clearly this man isn't happy.
This one doesn't look too happy, doesn't necessarily even look happy.
This one is happy.
And you could look at that.
So generally for this problem, we see clearly that many, many problems come or many of these classifications
come already from our data set.
So this means that the dataset we have to look at the dataset and ensure that it is cleaned before making
use of it.
So you see this happy, this person isn't happy, this, this happy, this happy.
We could get too sad.
You see, this person is happy, but this labeled as sad.
So with these kinds of interactive visualizations, we could be able to save.
For example, we want to see where the model was was pretty a certain value and it didn't produce that
value.
Now, we could also check out where this is equal, so you could check equals and say, okay, where
the the row label is equal to predicted, we apply and then we check out the kinds of images we have.
You see from here that most times the labeling will be correct.
You see the label happy label sad, the label angry.
We are we still have this image here, the shows that we have even gone as far as teaching the model
to look at this kind of image as an angry image.