We've seen classification of
text over the last few lessons.
But what about if we want to generate new text.
Now this might sound like new unbroken ground,
but when you think about it,
you've actually covered everything that
you need to do this already.
Instead of generating new text,
how about thinking about it as a prediction problem.
Remember when for example you had
a bunch of pixels for a picture,
and you trained a neural network to
classify what those pixels were,
and it would predict the contents of the image,
like maybe a fashion item,
or a piece of handwriting.
Well, text prediction is very similar.
We can get a body of texts,
extract the full vocabulary from it,
and then create datasets from that,
where we make it phrase the Xs and
the next word in that phrase to be the Ys.
For example, consider the phrase,
Twinkle, Twinkle, Little, Star.
What if we were to create
training data where the Xs are Twinkle,
Twinkle, Little, and the Y is star.
Then, whenever neural network sees the words Twinkle,
Twinkle, Little, the predicted next word would be star.
Thus given enough words in a corpus with
a neural network trained on
each of the phrases in that corpus,
and the predicted next word,
we can come up with some
pretty sophisticated text generation
and this week, you'll look at coding that.