Optimization methods are just a different way to calculate the coefficients off of variations.
There are two sets of optimization methods Classico optimization methods and intelligent optimization
methods.
Let's review some of the methods in each class and get to know some details for classical optimization
methods.
We have gradient descent, which we know as a short form for Ghedi.
Gradient descent is a first order integrative optimization algorithm for finding a local minimum of
a differentiable function in artificial neural nets, for we use gradient descent to update the coefficients
and weights in the network.
The other one is just adaptive learning method.
And here we have also back propagation.
Back propagation is short for back for propagation of errors.
This is an algorithm for supervised learning of artificial neural networks using gradient descent.
Given an artificial neural network and an error function, the method calculates the gradient of the
error function with respect to the neural networks.
Veidt, it is a generalization of the Delta rule for perceptions to multiplayer fit for neural networks.
We are not really interested to see the details of each of these methods.
If you want to have further more study, I can give you some reference book at the end of this course
for your study.
But for our use, which is practical purposes, we really don't need to know the details of how these
methods are working.
We only need to know when to use them and how to use them.
Next one is intelligent optimization method, the most famous one here, we can mention E's genetic
algorithm.
The genetic algorithm is a method for solving both concentrated and unconcentrated optimization problems
that is based on natural selection.
If you are a computer engineer, you might already have some knowledge and background on genetic algorithm
if not only learn how to use it and really no need for getting into the details.
Swarm optimization algorithm or piso, which is a computational method that optimize a problem, buy
into relatively trying to improve a candidate solution which record to a given measure of quality.
And finally, we have differential evaluation algorithm and simulated annealing algorithm or assay.
There are more.
I just put the most famous one.
And again, we really don't need to know the details.
We only need to know how they are working for the practical purposes.