All the algorithms that are used in deep
learning are largely inspired by the way
neurons and neural networks function and
process data in the brain. This image is
one of the very first pictures of a
neuron. It was drawn by Santiago Ramon
y Cajal,
back in 1899 based on what he saw after
placing a pigeon's brain under the
microscope. He is now known as the father
of modern neuroscience, but based on his
drawing, the neurons, one of them labeled
A, have big bodies in the middle and long
arms that stretch out and branch off to
connect with other neurons. This other
image here is that of a neural network
and has a bunch or thousands of neurons
in what looks like a brain tissue. It
gives you a sense of how tightly they
are packed together and how many of them
are in a small brain tissue. Going back
to the drawing of neurons by Ramon y Cajal,
let's rotate it 90 degrees to the
left.
I bet this way it is starting to look a
little familiar since it slightly
resembles drawings of artificial neural
networks that you must have seen. Here is
a cartoon drawing of the neuron. The main
body of the neuron is called the soma,
which contains the nucleus of the neuron.
The big network of arms sticking out of
the body is called the dendrites, and
then the long arm that sticks out of the
soma in the other direction is called
the axon. The whiskers at the end of the
axon are called the terminal buttons or
synapses. So the dendrites receive
electrical impulses which carry
information, or data, from sensors or
terminal buttons of other adjoining
neurons. The dendrites then carry the
impulses or data to the soma. In the
nucleus, electrical impulses, or the data,
are processed by combining them together,
and then they are passed on to the axon.
The axon then carries the processed
information to the terminal button or
synapse, and the output
of this neuron becomes the input to
thousands of other neurons. Learning in
the brain occurs by repeatedly
activating certain neural connections
over others, and this reinforces those
connections. This makes them more likely
to produce a desired outcome given a
specified input. Once the desired outcome
occurs, the neural connections causing
that outcome become strengthened. An
artificial neuron behaves in the same
way as a biological neuron. So it
consists of a soma, dendrites, and an axon
to pass on the output of this neuron to
other neurons. The end of the axon can
branch off to connect to many other
neurons, but for simplicity we are just
showing one branch here. The learning
process also very much resembles the way
learning occurs in the brain as you will
see in the next couple of videos. Now
that we understand the different parts
of an artificial neuron, let's learn how
we formulate the way artificial neural
networks process information.