Everyone and welcome back to this class natural language processing in Python.
In this lecture, we will continue looking at our code to implement a cipher, a decryption algorithm.
We are finally ready to look at the results, as you recall.
We have already stored the best mapping which we can use to produce our final decoded message.
First, what we'd like to do is print out the log likelihood of this decoded message, along with the
log likelihood of the true message.
What is also interesting is to check which letters we got wrong.
To do that, I'm going to loop through the true mapping.
We'll call the key true and the value v inside the loop.
We'll get the prediction by indexing the best map by V.
We'll call the result pred.
Remember that the best map is a reverse mapping because we were using it for decoding rather than encoding.
So pred is the value in the reverse mapping, whereas true was the key in the true mapping.
Then we check if true is not equal to PRED.
And if that's the case, we print out both letters.
As you can see, the results are close, but not exactly perfect.
In fact, strangely, our log likelihood is higher than the true log likelihood.
This seems strange because it seems like the real answer should be the maximum.
We'll discuss the reasons for that in the conclusion.
We can also see that we got a few letters wrong.
We'll see why that is very shortly.
In the next block of code, we do the somewhat obvious thing.
Print out the actual decoded message and compare it with the true message.
As you can see, most of it is correct.
There are some things that are incorrect, as you can tell, since our mapping does not match the true
mapping precisely.
All right, so let's read the decoded message.
I then lounge down the street and found, as I expected, that there was a muse in a lane which runs
down by one wall of the garden.
I lent the isol�s a hand in rubbing down their horses and received an exchange to Pence a glass of half
in half, two fields of shake tobacco and as much information as I could desire about Miss Adler to
say nothing of half a duck in.
So that one's wrong.
Other people in the neighborhood in whom I was not in the least interested, but whose biographies I
was compelled to listen to.
So you can see that we got one character wrong here, even though there are more entries in the mapping
that are wrong.
In fact, if we did get something wrong, it's a necessary for at least two entries in the mapping to
be wrong.
This is because, of course, the letters in the mapping are unique.
Therefore, if one is wrong, it must be replaced by some other letter, which is also wrong.
Therefore, if we get something wrong, there must be at least two wrong entries.
And you saw that in this particular run, we got three wrong entries, and this will vary from run to
run.
Actually, it's important to consider that when you're using genetic algorithms or evolutionary algorithms,
it's important to run it multiple times to check whether you've really arrived at the best possible
answer.
This is just due to the inherent randomness in the algorithm.
Sometimes you may get stuck at a local maximum or just start from a bad spot.
In which case, running it again would be necessary.
It's easy to see why we got this word wrong and why our model got some mappings wrong.
The word we are looking for is does in and we mistakenly use a K instead of a z.
That makes sense because Ozzy and Ziggy are probably very rare by grams being rare.
Their probabilities are very small, leading to a smaller likelihood.
The bigram OK and key are probably not as rare, simply because Kay is a more common letter than Z.
Therefore, they lead to a higher log likelihood thus under our model assumptions.
Using a K makes more sense.
Our model doesn't know that Duncan is not a real word.
You'll also notice that even though we only got one letter in the message wrong, our mapping has three
letters wrong.
Specifically, the extra letter here seems to be a cue.
It makes sense to get that wrong.
Why?
Well, because there is no cue in the actual message.
Therefore, we would have no idea where AQ should go because it doesn't contribute anything to the log
likelihood.
Finally, we have one last thing in the script, which is something I always like to do for every iterative
algorithm.
That is to plot the objective per iteration.
We expect it to go up very fast the beginning and then taper off at the end as the algorithm converges.
And this is the behavior you usually expect for most machine learning algorithms.
And in fact, that's exactly what we see, which is a good sanity check for whether or not our algorithm
is working.