In this lecture, we are going to look at a notebook that implements convolutional neural networks for
tax classification.
This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise,
which I always recommend is once you know how this is done, to try and recreate it yourself with as
few references as possible.
As usual, you can look at the title of the notebook to determine what notebook we are currently looking
at.
So at a high level, really, this script contains nothing new.
You know how to do text preprocessing because we just did that.
You know, how to use them betting's because we just did that.
And you know how to build and train a CNN because we did that earlier.
The only difference in this script is we're replacing all the tutee operations with one of the operations,
not a huge change.
So I hope you had the opportunity to try this yourself first before watching this lecture.
OK, so since the script is very similar to the previous one, we're going to go through it very fast
and only talk about the new thing, which is the CNN with one dimensional convolutions.
So first we download Spambots CSFI.
The next step is to read in the CSFI using pandas.
The next step is to look at the CEV using the head.
The next step is to drop the junk columns.
The next step is to call the head again to make sure they were removed.
The next step is to rename the columns to labels and data.
The next step is to call the head again to make sure that we're not.
The next step is to create a zero one labels.
The next step is to split the data into train and test.
The next step is to check the shape of our train and test set just to see what they were.
The next step is to perform all the tests, preprocessing steps.
We'll start by initializing IDEX and word to IDEX.
The next step is to populate words while using the training corpus.
The next step is to print out words, the acts as a sanity check.
The next step is to check how many values are in word to ATX.
The next step is to convert our training corpus into lists of integers.
The next step is to do the same thing for the test corpus.
The next step is to check the length of our new model input's.
The next step is to create our data generator, which you've seen before.
The next step is to grab our device variable.
Now, I've told you why the output of the embedding layer will be assigned by T, by D.
But this kind of thing is no good if you've just memorized it.
You have to see it for yourself.
So here we create an embedding with Dimension 20.
Next we through our data generator and print out the shape of the data before and after the embedding.
As you can see, the shape of the data before the embedding is NBT in the shape of the data after the
embedding is NBT by 20, which is what we would expect.
This brings us to our custom CNN model in the constructor, we create our embedding layer, a series
of convolutions and max pools, and then a final dense layer.
Notice that we now use one D instead of continuity and Max Poole, one D instead of Max fortuity.
But the forward function is where things get strange.
First, we pass the data through the embedding, which gives us an ENVI TBD, as you know, but convolutions
and PI torch use features first, meaning that it expects an end by D by T, not an end by TBD.
So we have to permute the data to make it that way.
Note that in other libraries, this might be called transpose.
It's just a generalisation of the usual two dimensional matrix transpose.
Next, we pass our data through the convolutional and pooling layers.
The next step is to permute the data back, to put the features last note that this isn't absolutely
necessary since we get to choose which dimension to take the max over in the next step.
So maybe as an exercise, you can try to remove that second permute.
Lastly, we pass our data through the final dense layer and then return our output.
The next step is to instantiate our CNN and move it to the GPU and after this is just stuff you've already
seen.
The next step is to create our loss and optimizer.
The next step is to create transgene and Tashjian.
The next step is to create the training function.
The next step is to call the trading function.
The next step is to plot the laws per iteration.
OK, so the last iteration looks good.
The next step is to calculate the accuracy.
OK, so the accuracy looks good and we're done.
OK, so we get a very high accuracy with a CNN in the high 90s, you might have thought because Arnon's
are four sequences, they might do a little better than CNN's.
But in fact, CNN's are pretty great as well.