In third year at engineering college, we had an elective course called Neural Networks. Unlike today, deep learning was not a well understood field at the time and the field had been in eclipse for some time. But we undergraduates didn’t know that. Our professor was a graduate of MIT who believed in starting from the fundamentals and her class consisted of students mainly from electrical engineering and computer science eager to understand exotic sounding topics like artificial intelligence.
What followed was a bit disillusioning, artificial neural networks were hopelessly primitive (and remain so despite many advances) compared to even the simplest natural neural network. Also, our professor believed the best way to illustrate this to us was by spending half the semester studying the biology of the neuron. Many computer science students found this too frustrating and dropped out, but it remains one of the most interesting topics I studied.
Take a moment to consider your brain. The human brain is made up of about 100 billion cells called neurons (about the same number as stars in the galaxy). Each neuron looks something like the diagram below. There is a main body called the soma which contains the nucleus. The body is surrounded by a foliage of tiny tendrils called dendrites. There is also a long tail coming out of the soma called the axon. This goes down and ends in a number of tendrils too which are labelled as the axon terminals.
A diagram of a neuron
There is one way in which this diagram can be misleading and I only realized this when I finally saw a neuron under a microscope. The dendrite tendrils connected to the soma can form a huge tree like structure.
Drawings of actual neurons
Now this is the important bit, how neurons are connected to each other. The axon terminals are connected to the dendrites of neighboring neurons through tiny electrical connections called synapses. Got that? The axon is connected to the dendrites of other nearby neurons. These are important because this is how neurons communicate.
Electrical charge can flow across these synapses from one neuron to another. When enough charge enters a neuron from all the axons connected to it, the voltage inside the neuron’s body suddenly rises and then falls off creating a spike.
Voltage spike in a neuron
In the diagram above, note how the voltage builds up slowly as charge comes in from neighboring axons. When it reaches a critical threshold, it shoots up and then drops down.
Initially, this voltage spike is in the soma, but it travels down the axon, down to all the synapses connected to it. It then triggers all the synapses connected to it, so charge will now flow into the dendrites of all the neighboring neurons which are connected to this neuron’s axon terminals. In this way, a neuron that gets triggered can in turn trigger its neighbors.
A network of communicating neurons
In practice, there are many complications. For example, I’ve just described that a firing neuron can trigger other neurons connected to it. I haven’t described how much one neuron contributes to the firing of another. In practice, the effect of various neurons firing is different. The extent to which one neuron firing can influence another neuron actually changes with time. In fact, this is considered one of the main mechanisms by which the human brain learns. When we learn something new, the synapses in our brain are getting modified and the extent to which one neuron listens to another about when to fire gets changed. Finally, the whole idea may be stored in our brain in the form of a pattern of synapse connections between neurons.
There are even some inhibitory synapses where if one neuron triggers, it can actually prevent neurons connected to it from getting activated.
One of the early researchers to study neurons were Hodgkin and Huxley who won the 1963 Nobel Prize for their work. At the time, instrumentation was very crude, so they chose to work on the largest neuron they could find to make it easy to connect the electrical probe. The neuron they chose was that of the Giant Squid.
Encouraged by their success, many neuro-biologists rushed to study the squid neuron which they did for the next 30 years. According to legend, not much worthwhile science was generated, but the researchers did learn many new recipes for cooking squid.