In this lecture we are going to look at how to implement again and code.
This lecture is gonna walk you through a prepared code lab notebook.
Although a very good exercise which I always recommend is once you know how this is done to try and
recreate it yourself with as few references as possible as usual you can look at the title of the notebook
to determine what notebook we are currently looking at.
So at the top we have all our usual imports.
We've also imported the Save image function from torch vision which is a useful utility for saving an
entire batch of images to a single image file.
This will let us check how our model progresses over each iteration.
Next we define a set of transformations to perform on the data.
Remember that researchers have found that using pixel values in the range minus one to plus one tends
to work better.
So this code might seem strange efforts but if you check the calculation you can verify that it works.
First remember that the two tensor transformation already scales the image to be between 0 and 1.
So if we take one subtract a mean of zero point five and divide by a standard deviation of zero point
five we get zero point five divided by zero point five which is one if we take zero subtract the mean
of zero point five and divide by a standard deviation of zero point five we'll get minus zero point
five divided by zero point five which is minus one.
Therefore we map 1 to 1 and 0 to minus 1.
So this works.
Next we load in the amnesty train dataset from torture vision.
You've seen this before.
Next we create a day loader from our data set
next we create our discriminated network.
So this is an A and end with layer sizes 780 for 512 256 and 1 and with leaky real you activations.
Next we create our generator network.
This is n n n with input dimension one hundred and hidden dimensions 256 512 1024 and 780 for.
Notice how we've also added a batch final home after each leaky well you and that the neural network
ends with a 10 each activation which maps the output values to be between minus 1 and plus 1.
Next we set the device to be the GP you and we move both the discriminator and generator to the GP you.
Next we create a loss and optimizer is for the loss we can use the binary cross entropy for both cases.
The only thing that changes is the label for the optimizer as we make sure to pass in only the parameters
for that specific network.
So the D optimizer only optimizes the parameters in the D model and the G optimizer only optimizes the
parameters in the G model.
Next we have a convenience function for scaling images back from minus one plus one to zero one.
This is useful for plotting and saving and so forth.
Next we create a directory to store the images generated during training is called again images.
Next we have the training area.
So this will be slightly more complex than the code preparation section.
Since I left out a few minor details but hopefully you understand the main idea and so these minor details
are just that minor.
Sort of start we're going to create one dimensional arrays just containing ones and zeros.
These will be used as targets during training.
The targets are constant values so instantiating these inside the loop would be wasteful.
Next we create two lists to store the losses for the generator and discriminator.
Next we enter a loop that will go for two hundred epochs and inside that loop we have another loop to
go over the data loader notice we ignore the targets by using an underscore since they are not used
inside the loop.
The first thing we do is get in the backsides.
Remember that since the backside may not divide evenly into the total number of samples the actual batch
size may not be equal to the specified back size.
Next we take the inputs and flatten them into an end by the matrix and move it to the jeep.
You
and now that we have N which is the current that size we can also index the ones array and the zeros
array to get the correct size targets
next we have the train discriminated block.
The first thing we do is get the loss for the real images so we pass the inputs through the D model
and call that real outputs.
Then we pass real outputs into our criterion with ones as the target.
We'll call this D loss real.
Next we calculate the loss for the fake images.
So we start by generating some noise which goes in a matrix of size and by 100 we move that to the GP.
Next we create our fake images by passing this noise through the generator.
Next we pass the fake images into the discriminator and we get the fake outputs.
We pass the fake outputs into our criterion with zeros as the target.
We'll call this loss D loss fake.
Next we combine the losses by taking the average then we zero the gradients in both optimizations.
Then we call the lost that backward and then D optimize it out step to do one step of gradient descent
on the discriminator.
Next we have the train generated block.
As mentioned it's been found that training the generator more than once for each iteration is useful.
So we do a for loop that goes twice inside the loop.
We generate some random noise of size end by one hundred and move that to the GP you.
Next we pass this to the G model to get a set of fake images.
Next we pass the fake images through the D model to get a set of fake outputs.
Next we pass the outputs through the criterion this time using ones as the targets.
This is because we would like the discriminator to think these are real.
Next we do a gradient descent step.
First we zero the gradients.
Then we call G lost out backward and then we call G optimize it out step and remember that even though
the outputs that go into the lost function are outputs from the discriminator we are running key optimizer
which only knows about the G models parameters so only the G models parameters are updated.
Finally we save the losses to our list of losses.
We print the epoch number and the losses and then at the end of each epoch we save the fake images that
were last generated using the Save image function that we imported earlier.
Note that before we save the images we have to scale them back to the correct range using the scale
image function
OK.
So let's check out the lost period creation
so as promised it's not the nice downward converging pattern that you're used to.
Okay so let's look at the images we generated so here are the fake images after epoch 1.
We can see that at this point they are already starting to resemble the MLS dataset
after epoch 50 it pretty much looks like M.A. After epoch 100 it looks even more like an honest
after epoch 150.
It still looks like it missed maybe even a little better and after epoch 200 it still looks like amnesty.
So as you can see the results are pretty much what we expect at the beginning.
We didn't see anything at all.
And by the end it looks pretty much like amnesty.
The generator learns to generate real looking digits quickly so there's an aspect of diminishing returns
here.
After the first third of the way through things already look pretty good.