Welcome back. In this week,
you'll learn how to use the Keras layer in
TensorFlow to implement word embeddings,
which is one of the most important ideas
in Natural Language Processing.
If you have a vocabulary size of say 10,000 words,
rather than using the numbers
from 1-10,000 to represent these words,
can you have a better way to represent those numbers?
So this week you'll learn about
word embeddings and also get to
train your own neural network for text classification.
It's really interesting that I really liked
how embeddings work for
representing the semantics of a word.
So now instead of the word just being a number,
it's like a vector in n-dimensional space.
We're using 16 and 32 dimensions in different examples.
So for example the word dog,
might be a vector pointing in
a particular direction and then the word K9,
could be learned as a vector pointing
in a very similar direction,
and we know they have very similar
semantic meaning off of that.
All of this is actually done for us in embeddings.
So as we train like our datasets,
we'll train for example with IMDB which is movie reviews,
and there's a set of positive reviews
and a set of negative reviews,
and what will actually happen is
the embeddings for the words in these reviews,
and we can actually plot them on a chart.
So we'll see something that looks like
a globe and one of the poles on
the globe or all the words that are
clustered around for the positive review,
and all the words on the other pole
of the globe are clustered around the negative review,
and it really helps us to start to see
the semantics behind these words.
I think one of the coolest things
about word embedding is you can
download the pretrained word
embedding that maybe someone else has trained.
This gives your learning algorithm
a hint into the meaning of these words.
So when you see another vocabulary word,
maybe you see K9 for the first time,
it's as if you can let the algorithm know
that K9 means something a little bit
like dog related to dog even if you've
never seen the word K9 in your specific training.
That's amazing.
Today in the world of applied NLP,
word embeddings has proved to be one of
the most powerful and useful ideas to
help teams get good performance.
So in this week,
you'll learn how to use word embeddings in TensorFlow.
Let's go on to the next video.