Hi and welcome back.
In this section, we'll take a look at Vision Transformers, which are, pardon the pun, transforming
the Peter Vision world right now, and they're almost almost replacing scenes in many of the state of
the art computer vision, deep learning tests.
So let's get started.
So what exactly are vision transformers?
Well, what they're not.
They're not those transformers from the movie, and they're also not electrical transformers that do
step up or step down voltage changes.
What they are, it's actually something.
It's a method or algorithm.
You can see what type of neural network design that came from the A.P. world in a famous people in 2017
that was called attention is all you need.
And this was a ground breaking people in the A.P. world because it allowed it allowed them to encode
way more information into their networks and beating less DMs and earnings.
At the time that so much A.P. tests and pretty much, no, it's all they only do is transformers.
And if you want to do a state of the art stuff, at least not simple classifiers, but steady state
of the art A.P. is almost all transformers, and that is almost changing in the computer vision world
right now.
And they use something called self attention, which allows us to take sequential inputs like words
and find the correlations between different features and towards that information as part of the networks
knowledge.
And from 2020, it started to gain a lot of traction in the computer vision world.
And you can see why you can see in this task here or in this model, this T2D vision transformer is
actually beating resonance and previous votes, as well as well and in accuracy and model size and max
as well.
And you can see model sizes actually getting better results from a similar model size to mobile that
vision, too.
And this is just one transformer network that the researchers are presenting here, comparing it to
a few of the popular networks.
It doesn't mean this is the best right now, but it's pretty good.
So how is it used for images?
Well, firstly, we've got to split an image into patches and then flattened as patches and then create
a basically all of those patches, a lower dimensional linear embedding was formed from those flattened
patches.
And then next, we add a positional embedding alongside of those the slower dimensional effector that
we've created.
And because usually this is a trainable position embedding, it's typically used.
And then we feed that sequence as an input into a standard transformer encoder.
And next, we pre-trained the model with image labels from a very large dataset, something like image
net.
And then when you when you want to train it with your own dataset, you fine tune that network on the
downstream dataset for your image classification tests.
Now, let's take a look at this very cool animation produced by Google that explains vision of Transformers.
So you take an image here, you split it up into patches, as you can see here, then have a linear
projection of that in patches.
You add the positional embedding still, and then you pass it through transforming CUDA it to A.P. head.
And then we predict the classes out of it.
So let's watch this one more time again.
You can see the positional embedding here after we have a projection of flattened patches.
This goes into a transform encoder, then to an LP head and then classify results.
Go here.
So let's talk a bit more about Vision Transformers, so the image patches analogous equivalent to sequence
tokens, which is words and LP.
That's how we treat images as inputs into a transformer network, which relies on a sequential input.
And if you wanted to get a deeper vision transformer, you can just increase the number of blocks inside
of the Transformer encoder.
And these are what the blocks look like, says in multi attention distance, that's the most important
parts or the most important parts of the transformer encoder.
And it's important to know that Vati is to require a lot of data to be trained to even be compared to
state of the art scenes.
So that means basically means creating from scratch when you have smaller data set is not really an
option with Vision Transformers, so you may as well stick to CNN something like a small C and then
like a resonant 18, would probably work well for most general image specification tests.
However, what we can do is take a pre-trained vigilance transformer and then fine tune it on your smaller
data set and change it and they'll be updates to your class size.
So if it's like five classes or two classes, change it and that's it.
So we'll stop this now because this is just a brief overview.
An introduction to fishing and transformers.
What we'll do next, we'll take a look at some implementations of fish and Transformers in both PyTorch
and Keros.
So stay tuned for that and I'll see you in this lessons.
Thank you for watching.