Welcome back. Transfer learning
is one of the most important techniques of
deep learning and TensorFlow lets you do
that with just a handful of lines of codes.
Rather than needing to train
a neural network from scratch we
can need a lot of data and take a long time to train,
you can instead download
maybe an open-source model
that someone else has already trained on
a huge dataset maybe for weeks and
use those parameters as a starting point to
then train your model just a little bit more on
perhaps a smaller dataset that you have for a given task,
so it is called transfer learning.
Transfer learning, and I find it's really cool because
we've been looking at convolutional neural
networks up to this point,
and when we're training with
a convolutional neural network we're
extracting features from an image
and learning those features,
learning the ones that match images.
But when we download a large models
something like an inception that has
all of these features that have already been extracted
off lots more images than we can use,
I find that really exciting because as you
mentioned researchers who built
these wonderful models of lots of images
have been able to share them and today
we've been looking at convolutions and
convolutional neural networks where
from scratch we've been
training something off of our data and off of
our images to be able to
extract features from these and learn them.
But now these models that have been trained off with
lots and lots of images over
a lot of time might have
spotted other features that weren't in
our dataset and will help
us to build better convolutional networks.
Yes. There's so much stuff
for neural networks to learn about the world.
There are edges, corners,
round shapes, curvy shapes,
blobs and then there are things like eyes,
circles, squares, wheels.
There's so many things in the world that
convolutional neural networks can
pick up on but if you have only
a 1,000 images or even 25,000
images may not be
enough data for content to learn all those things.
So by taking an inception network
or some other network that someone else's has trained you
can basically download all this knowledge into
your neural network to give it a
huge and much faster start.
It's really exciting to be a part of
that community where these things are being shared so
that people who don't have access to
machinery to be able to build
massive models like this one,
are able to derive
the features from the people who have done so.
I personally find that really inspirational.
Yes. Many days I wake up really
grateful to the whole AI community for the openness,
so many breakthrough AI ideas are
shared in papers on the Internet,
on the open source code shared on the Internet,
and I think this has been
a large reason for the rapid advance of AI.
I think to all the learners if you end
up doing something consider sharing
it on the Internet as well freely to contribute back to
this open community that's making
all of us go much faster.
Definitely.
Transfer learning is a technical embodiment that
lets us use these ideas to accelerate the whole field.
One of the cool things about transfer learning is
that it's so simple to implement,
in TensorFlow you download a model and then you
say set these models as
trainable and freeze or lock
those other layers and then you just run.
Yes. In the course what
the students are going to be looking at
is the inception model
which is a beautiful image classification
and classifies a 1,000 images it has been trained on.
I think it's like over a million
images and what you're going
to do is you're going to take a look
at one of the lower levels in that.
You're going to lock all the way down to that.
You're going to add a DNN
a deep neural network underneath that,
and then you're just going to retrain
for those lower levels,
and as a result using all of these,
I like to call it standing on the shoulders of giants,
you're going to be doing all that to be able
to make the classifiers that you've been
building in the course to date
much more efficient and maybe even
quicker to be able to reach higher levels of
accuracy than if you're training it from scratch.
So this week, you'll learn how to
implement transfer learning and use it to get
your models to not only train faster but
also get higher accuracy. Please dive in.