Hi and welcome back.
In this section, we'll talk about how we can start training games and understand the process and challenges
in training guns.
So let's get started.
So firstly, training guns is notoriously difficult compared to training neural networks.
Hi and welcome back.
In this section, we'll talk about training guns.
We'll go into detail of what the process and the challenges you can experience when training guns.
So let's get started.
So firstly, training guns is notoriously difficult compared to neural networks because neural networks
are quite simple.
It's one network.
We have a simple loss function.
We use gradient descent and back propagation to reduce our loss.
Generally, I mean, yes, it's complex, but it's actually quite simple compared to Gans.
And that's because in guns, every weight change can actually change the entire balance of autonomic
system because remember, we have two adversarial systems working with two against each other.
However, you want to think about it.
And so in this case, we're not trying to just minimize loss.
We're trying to find an equilibrium between both opposing networks.
So when training stops or basically we end training when the discriminator cannot teleport to from first
for big data, that's essentially how we even know when to stop.
So you can see this leads to a lot of problems again.
So you basically have to babysit the training process and try to experiment a lot with parameters to
make sure we can achieve this equilibrium.
So let's talk about the training process.
So firstly, we start with a noisy vector in this training process and this example, I'm assuming we're
training again to generate images.
So we use an image vector like this here to represent the noise.
Next, we input this noise into a generated network to generate some synthetic data.
So initially, in the first training phase of this training process, the generated just has random
with initialization.
So he's going to produce a random looking noise image here.
So what's next?
So next, we take some sample data from a real dataset and mix it with some of our generated synthetic
data.
So in this combination of data here, we have some real data and some synthetic data.
Next, what we do.
We train with these communities to classify whether this mixed dataset has real or fake data.
So he has identified what's real and what's fake out of it, and then used the knowledge to ground your
knowledge afterward to update his weights accordingly.
Next, we use that information to train the generator.
We make some more random noisy vectors and create more synthetic data.
And with the width of the discriminative frozen, this is in this field.
Here we use to feedback from this community to now update widths of our generator.
So let's think about this.
So firstly, I would discriminate.
So that's this guy right here.
He basically identifies whether from that mixed dataset, what's real and what's fake.
So then we hear he gets his weights accordingly to the ground truth of that.
So next in this next phase, the generator generates some more and fake and real and somewhat fake data,
and mixes it with the real data.
However, we freeze the weights of discriminator.
No, because he just that his we took it and we take those images here and get his feedback as well.
And then once he gives us feedback, if he's able to identify most as more specific data as well, the
generator now has to update its weights accordingly so that it can improve in the next iteration.
So this is how it looks, this is how I do it of process looks.
So this is the first minister that degenerates, generates on a first epoch and you can see it generally,
most of us looks quite bad.
This is this is the amnestied of status that we're looking at.
And then you can see in the second epoch, things sort of look a bit, but so you can see this looks
like for this, it looks like a zero.
This looks like a two, but it's still not good.
And most of this discriminator might say it's fake.
Then I'm 30 buck.
You can see it gets better.
So you can see how this feedback loop of passing information across the networks actually leads to improved
cash generative results here that look like they plausibly would have come from the original data set.
It's pretty cool.
So are there any issues with guns?
Yes.
Look carefully at this image.
See that that fear smudging effect here.
That's because guns, they can understand and generate images like this quite well.
Once they learn the representation of a face, they can apply their random noise generated generative
knowledge to the embedded learnings of that generated network and generate the fear so they fit.
However, because sometimes faces in groups, it will generate weird looking faces like this.
This guy looks slightly like he has a skin boon and faces like peeling off of it.
So there are issues, and those issues basically point to the fact that guns still don't fully understand
what they're learning, which speaks back to Richard Feynman's humans could.
However, they do learn a lot, as you can see from this discriminator and generator set up.
They actually do learn a lot in representing and creating new images, which you'll see in the practical
experiments will do after this lesson.
But first, let's talk about some general challenges in training guns.
Firstly, because it's a dynamic system, achieving equilibrium is quite difficult.
Now also, guns are computationally demanding to train, and it requires a lot of tweaking of hyper
parameters initialization changing a number of hidden lives, trying different activation functions,
using batch non-metropolitan, different layers.
So it's quite a messy process.
There's no there's some ground truth, there's some ground rules and rule of thumb strategies to train
your guns.
However, it's still very difficult.
Also, bad initialization can cause a discriminator loss to go close to zero.
You don't want your discriminating loss too good to go close to zero.
You want it to remain at point zero five, roughly, which means that it can't tell the difference between
real and synthetic.
It's basically indistinguishable.
So one solution is to purposely make your generator loop slower than the discriminator.
However, this too can be hard to balance, as a good discriminator depends on a good generator, so
you can see the problems that you are raising and training guns.
And then there's something called mode collapse.
This happens when, regardless of the noise input fed into the generator, the generated output varies
very little.
It occurs when a small set of images look good to discriminator and get scored better than the other
images.
The gun simply learns to reproduce those images over and over.
Sort of analogous to overfitting at that point, so more collapse is something you need to be careful
of.
Next, we'll take a look at some practical use cases for guns so you can go.
We'll go over the many ways guns are applied in the industry, so stay tuned for that.
Thank you.