In this lecture we you're going to do code preparation for our linear classification script this example
will look at a new classification dataset.
So this lecture will go overloading the data as well.
To recap here are the general tasks that we need to complete.
Number one loading the data number to create the model number three train the model.
And number four evaluate the model
let's start with loading in the data.
For this example we'll be looking at the famous breast cancer dataset.
This data set happens to be included as part of the cycle learn API.
You'll notice that in this course we'll be looking at quite a few pretty famous datasets.
These datasets are so famous that they are often included in the various machine learning libraries.
So there's no need to download the dataset as we did in our previous example.
Of course there will be plenty of examples in this course where the data do not come from libraries.
So you'll get experience with each loading in data from psyche you learn is super easy.
We just call the function load breast cancer which returns a data object.
This data contains the X's and Y's so we'll need to access them using the objects attributes in particular
the inputs can be accessed using the data attribute and the targets can be accessed using the targets
attribute.
There are two ways we'll want to pre process the data before using it.
First the data isn't normalized.
You already learned from our earlier example why this is a good idea.
If you did the exercise I suggested therefore we will use the cycle learn module standard scalar to
normalize it.
Second we want to split the data into train and test sets.
Intuitively this is because we want to get a good idea of how the model will perform on data.
It hasn't seen before.
Not on data it has already seen for the data we've already seen.
We already know the answer so machine learning isn't necessary.
The data we really care about is the data we have not seen.
For example if you build a fraud detector you want to be able to ask your model whether a new transaction
is fraudulent.
This is important because your model might do very well on data.
It's already seen but poorly on data it hasn't seen.
We'll discuss this more in a later lecture.
In any case this code shows you how to do both of these steps.
Notice how we already use this idea of train test splits with the standard scalar we fit the standard
scalar on the training data only and we apply the standardization on the test data using the fitted
mean and variance of the training data.
Once we've prepared our data it's time to build a model.
As you know the linear classifier is almost the same as linear regression just with one extra step.
The sigmoid this should be a hint that we're still going to have a linear object somewhere in the model.
Well it just so happens that the sigmoid is also represented in PI talks with an object the sigmoid
object it should make complete sense then if we combine the linear object with the sigmoid object in
sequence and in fact that's exactly what we've done.
Pi 2 which allows you to easily stack these computations steps in sort of a wrapper objects called sequential
what you're telling pi to which is I want my model to apply these functions in this order and the functions
we want to apply in this case are the linear model than the sigmoid there's one detail here that I want
you to notice and this is that when we create the linear layer the input sizes D while the output size
is 1.
As you recall our data matrix is of shape n by D where n is the number of samples and d is the number
of features.
So we'll have one input for each feature column in the dataset.
In addition we have one output which represents the probability that the output should be classified
as a 1.
The next step is to train the model as promised we are still doing gradient descent which means nothing
about our gradient descent loop is going to change.
We still do y 0 grad get the output calculate the last call backward and do one step of a gradient update.
What's different is that we'll be using a different cost function but also a different optimizer as
mentioned previously.
Our last function for binary classification is the binary cross entropy which is performed in the object
BCE laws.
In addition we'll be using the atom optimizer which has become the go to default in deep learning in
recent years.
Normally I would say that the gradient optimizer you choose is like a hyper parameter so you should
always experiment to see what works best in practice.
Many people simply choose Adam by default.
This does not mean that Adam is actually guaranteed to work best.
So you should still try other methods and observe the results yourself.
If you want to learn more about Adam you're encouraged to check the in depth section of this course.
And if that's still not enough then you'll want to go through the in-depth course where I discuss this
algorithm in detail and build it up from scratch.
The last thing we're going to do in our script is evaluate the model.
This is different in classification compared to regression and regression.
We use the means squared error as our loss but we also use it as our evaluation metric.
It kind of makes sense since I don't think any other metric would be significantly more advantageous.
The mean squared error is kind of a natural way to look at the regression error alternatively you could
look at the root means grid error which is just the square root of the MSE so that it's in the same
units as the target but still taking the square root is kind of a trivial transformation with classification.
It's a different story.
We have the cross entropy as a loss which is not a natural or intuitive measurement.
What we really care about is accuracy.
Out of all my predictions how many did I get right.
And how many did I get wrong.
So in order to evaluate our model we're going to calculate both the train and test accuracy.
This necessarily also involves making predictions with the train model as well so we'll learn how to
do that too.
Of course it's not so different from how we make predictions with our regression model.
We just need to round our prediction to 0 or 1 since the targets are encoded as 0 or 1.