Hi and welcome back.
In this section, we'll take a look at implementing video classification, which we've done in the previous
lesson, but now be using Transformers to do that.
So let's get started.
So open notebook 67.
And again, this notebook comes from the official Keras tutorial repository, and it's produced by St.
Paul.
So let's get started.
So as just so much as I just mentioned, we are basically duped into the same thing we did before.
But instead of using the CNN Arnon architecture, we're going to use it to transform Mobius architecture.
And Transformers should be naturally applicable to video data because it streams in a sequential order.
So and that's what makes Transformers so powerful to get all of the correlations between each of those
inputs.
So now let's get started.
So firstly, we need to install TensorFlow docs.
So it's too that it takes about 23 seconds, then we need to download the dataset.
And the dataset again is the the S.F. one to one day to set its benchmark data set used for action or
video recognition.
So you can see that it has many different scenes like this of different activities baseball, pitch
boxing, diving, cricket shot a lot of different things here of cutting and coaching as well, so that
has a quite a variety of stuff.
So we'll be using a subset of that with just a few different classes as well.
So we download a subset here doesn't take that long, about 15 seconds.
Then we load on libraries to find some type of parameters that we'll be using for this.
Next, we prepare the dataset, so we just get the training levels, training data as well.
And so our.
Then we just have some crop center functions.
The loading video function here that uses open copy to load the video, as you can see frame by frame.
Then we have the build feature extractor part of it.
So in this case, we're using dense net 121, which is a fairly good network, actually.
So we just build a feature extractor here and then we create that feature extractor right there.
Then we use some labeled preprocessing here with Keros.
And then we also prepare all the videos here in this function.
So this loops of everything in the video parts and returns to frame features, which is the feature
extractor that we got, as well as levels.
So quite a lot going on there.
So we do that and actually it takes about 10 seconds to run and these are the classes here.
So we have cricket shot, playing cello, punch, having beer and tennis swing.
So we just grab the subset of that.
You have to see one on one data.
Next, we need to download the prepared data here.
This is what we produced in the previous lesson that took us about 20 minutes to download.
So we just download the pickled visions here.
An umpire is.
And we just get load them here.
Well, actually, it's the typical it's it might maybe a pickle file for, I know.
And next we build the transform the model here.
So we have to encode the positional embeddings because remember, transformers don't like by default,
don't have that positional embedding.
So we have to added back to it.
And that's easy because the video are in the sequence so we can use that.
But then we have to create our transformer cool here with the multi attention head to remember this
important little note there.
Then we just have our training function, so we get to compile to model here, to the model, and then
we just run the experiment through here.
So it's all relatively simple so far.
Now we just start training our model and this actually prints quite quickly, just about 15 seconds,
and we're getting pretty good accuracy here in our small dataset.
That's because it's a small dataset and pretty much a easy dataset for it to learn.
And now we can just prepare a single video here and predict its action out of it.
So I it creates a GIF in the end and we go in the end.
This is a GIF.
I guess the guy playing cello and or classifier predict playing cello very well, 100 percent probability.
So that's it for this lesson on using video transfer of Transformers for video classification will be
enjoyed a lesson and we'll stop there and I'll see you in the next lesson.
Thank you.