So in the previous lecture we talked about all the steps that are required to pre process your text
in order to convert your documents into lists of integers so that they can be ingested by a neural network.
So now the big question how do you do this in PI talk.
Well it's not done in PI torture per say but it's part of the PI talk ecosystem just like how a specialized
computer vision operations are done by the torture vision library specialized in IP operations are done
by the torture tax library.
This is also automatically installed in Google collapse so there isn't anything special you need to
install on your own.
If you use google code that unfortunately the syntax is quite strange.
So bear with me as I try to make it digestible.
First we're going to assume that our task is tax classification which it is and it's going to be a many
to one task.
That means for each document which is a sequence of words our prediction is going to be a single label
for that entire document that might sound a little bit abstract.
So to give you a concrete example think of spam detection.
Your input is an email which is a sequence of words and you would like to output a single label spam
or not spam for torch text will want this data to be formatted as a C as V with one column for the label
and one column for the input text.
I assume that at this stage you know how to write a program in Python so that if your data does not
come in this format you know what to do to convert it into the required format.
The next step is to create field objects field objects are part of the torch text data module both the
inputs column and the targets column will be field objects but with different arguments for the input
data.
I want to pass on the arguments sequential equals true batch first equals true lower equals true and
pattern first equals true.
Let's run through these quickly although they should be pretty intuitive already.
We said sequentially go to true since the data is sequential each value is a sequence of words.
We set batch first go to true because we want the end dimension to come first.
And the T dimension to come last we set lower equal to true because we want to lower case the words.
This isn't always necessary but our data set is pretty small in which case it's usually a pretty good
idea.
And lastly we set pattern first equal to true so that we use pre padding instead of post padding.
We already discuss why that might be desirable although you should try both.
Just to be sure for the label we're going to create another field objects but with an entirely different
set of arguments specifically we're going to set sequentially go to false use vocab equal to False and
is target equally true since the target is not sequential we said sequentially go to false.
This should be clear.
We set use vocab equal to False which might seem strange but let's think about it.
Remember that there are a whole bunch of text pre processing steps that we need to do.
One of which involves assigning a unique integer to each word in the dataset.
This is the process of building the vocabulary of words in the dataset.
Of course such a process wouldn't make sense to do on the targets because the targets are just numerical
labels.
So we set use vocab equal to False.
Finally we set is target equal to true.
Since this field is for the target column after all.
In my opinion this could have a better API but it is what it is.
You might also want to consider how these arguments might change if you were doing something like neural
machine translation.
The next step is to instantiate a tabular data set object which is also part of the tortured text data
module.
So now you understand why we want to create a CSB to store our data.
It's because a CSB is a table and torture text only knows how to read tabular data sets.
Now you might think it might seem kind of limited to only have tabular data sets.
What if we need a data set that is not tabular but if you think about it pretty much every text dataset
you can think of will probably fit into this framework.
Whether that sentiment analysis spam detection parts of speech tagging machine translation and so forth.
Even if it's unsupervised learning and all you have is just one document per line that's still technically
a CSB.
So all these different kinds of data sets can fit into a CSP format.
And as long as you're a competent programmer and you know how to make your own C S V they should not
be an issue.
So let's go through the arguments one by one so you know what they mean.
The first argument which is the path should be clear is the path to the data file where you're CSB data
set is stored.
The second argument is the format which I have specified as CSP.
Other options are at TSB and Jason but these are not so significant in my opinion.
A T S V is just like a CSP but with tabs instead of commas it's trivial to switch between them so there's
no advantage to using one over the other.
Jason is nice but it just adds extra characters to your data file without any real benefit.
You can use it if you want but again there's no distinct advantage unless for some reason your data
already came in that format.
Next we set skip header equal to true since in our dataset there is a header row that will want to ignore.
Finally we set the fields argument which is a list of tuples each tuple Contains two items.
The name of the field and the corresponding field object.
Called the first one data and the second one label.
The next step is to split the data into train and test sets which is nice and easy with torch tax just
called data set that split and this returns two items your train set and your test set as input.
You can pass in an argument called Split ratio to say what percentage of the dataset you want to be
part of the train set.
By default this is 70 percent so I haven't bothered to change it.
The next step is to call the build vocab function on our text fields from earlier this assigns a unique
integer to each unique token in the dataset so you can imagine that internally talks text is doing the
tokenization and all the other text pre processing steps that we discussed earlier.
This function by itself doesn't return anything but after doing this we can access the vocab attribute
in the text field which returns a vocab object.
So what is a vocab object
Well the vocab object contains that words index mapping that we talked about in the previous lecture.
You can access it by calling vocab dot as to AI so as to I should remind you of C style naming if you've
ever coded in the C programming language before.
This returns a dictionary where the key is a token and the value is a unique integer that corresponds
to the token.
Additionally you can call the vocab at DOT I2 s function which returns the opposite.
In particular it returns a list of all the unique tokens in the dataset.
Now you might be wondering how can the opposite of a dictionary be a list
well.
Remember that a dictionary is nothing but a set of indices and corresponding values.
So too is a list.
A list has the indices 0 up to K minus 1.
If the list has length k the values of a list are just the elements of that list.
So if I call vocab dot as to why of dog and I get back to eleven then if I call vocab dot I too s of
two eleven I will get back the string dog so they are inverse mappings relative to each other.
Now you might be wondering who cares and why do we even need these mappings.
Well if all we want to do is text classification then we don't necessarily but there may be instances
where let's say you're doing text generation or machine translation where the output of the neuron that
work is a sequence of integers.
Of course in order to turn that into text you have to convert that sequence of integers back into a
sentence and in order to do that you must know which integer corresponds to which word in your vocabulary.
Okay.
So at this point we have a trained data set and a test dataset.
Now what.
Well imagine your dataset is huge so huge that just like our image datasets we rather do batch gradient
descent rather than full gradient descent.
So just like in torch vision where we had an iterator over our image dataset we would like to use an
iterator from torture text for our text dataset.
Here's an example of how we would create an iterator for our text datasets the first argument is a tuple
containing the datasets that we previously created.
The second argument is a sword key.
The idea behind this is that because we'll need to pad each batch of our dataset we would like each
of the sentences in each batch to be of similar size.
So by specifying the sort key to be a function that returns the length of the text field we can tell
talks tax to organize the batches so that each sentence will be of roughly the same length.
Next we pass in back sizes.
This is a tuple containing the same number of elements as we have datasets.
Basically the first number represents the batch size of the train data set and the second number represents
the batch size of the test data set.
Usually we can specify the batch size of the test that are set to be larger.
Since it doesn't need to do any heavy processing other than just making predictions finally we set the
device argument to the device objects that we've been creating in all of our notebooks which refers
to the GP you.
So this means that our dataset will be automatically placed on the GP you without us having to do so
manually
all right.
So at this point you should be asking what's next.
Since we now have iterator is that can loop through our train set and our test set and return batches
that are just end by t arrays of integers corresponding to each sentence.
There's nothing more to do.
We can pass this data directly into our aren't n for both training and testing so you can imagine that
our training loop will look like it usually does as well our loop to calculate the accuracy one minor
caveat is that the torch text iterator gives us back a target that is only one dimensional.
We know that in PI talk we like to have two dimensional targets so we can call the view function to
convert each batch of targets from an N length one dimensional array to an end by 1 2 dimensional array.