So now, let's take a look at how to get a prediction for
a word and how to generate
new text based on those predictions.
So let's start with a single sentence.
For example, 'Lawrence went to Dublin.'
I'm calling this sentence the seed.
If I want to predict the next 10 words
in the sentence to follow this,
then this code will tokenizer that for
me using the text to sequences method on the tokenizer.
As we don't have an outer vocabulary word,
it will ignore 'Lawrence,' which isn't in
the corpus and will get the following sequence.
This code will then pad the sequence
so it matches the ones in the training set.
So we end up with something like
this which we can
pass to the model to get a prediction back.
This will give us the token of the word
most likely to be the next one in the sequence.
So now, we can do
a reverse lookup on the word index items to turn
the token back into a word and to add that
to our seed texts, and that's it.
Here's the complete code to do that
10 times and you can tweak it for more.
But do you know that the more words you predict,
the more likely you are going to get gibberish?
Because each word is predicted,
so it's not 100 per cent certain,
and then the next one is less certain,
and the next one, etc.
So for example, if you try
the same seed and predict 100 words,
you'll end up with something like this.
Using a larger corpus we'll help,
and then the next video,
you'll see the impact of that,
as well as some tweaks that a neural
network that will help you create poetry.