Walter Pitts, the other half of the duo that formulated the first model of an artificial neuron, burned his PhD thesis and drunk himself to death. Why?

Ludwig Wittgenstein, after publishing his work to finally solve philosophy, one of the wealthiest men in Europe, gave up all his wealth and left philosophy at the age of 31. Why?

Both Pitts and Wittgenstein, the former who grew up impoverished and the latter who grew up wealthy, were gifted with superior analytical minds. Both investigate logical system only to come to an unassailable conclusion. Their formulations were wrong!

In 1938, another genius, Alan Turing wrote in his PhD dissertation: "we have gone to the opposite extreme and eliminated not intuition but ingenuity, and this in spite of the fact that our aim has been in much the same direction."

In exploring Godel's incompleteness theorem, he came to the prescient conclusion that a logical system (he used the word ingenuity) could be implemented by suitable automation, but he could not see how to do the same with intuition.

Intuition is a manifestation of an analog system. That is why an artificial intuition system like deep learning is based on the dynamics of a mathematical system formulated with continuous variables.

When the rubber hits the road or rather when a cognitive system presses itself on to reality (actually the reverse), it must do so through an analog system.

A digital system cannot feel an analog reality. At best, it approximates that analog reality. But it can only derive meaning and predictions if itself is analog in nature.

But all the neurons in the brain with its intermittent spike appears to be a discrete digital system. What then are we missing?

We are missing the fact that neurons are nothing but the information superhighway that connects the system. It is as if we analyzed the internet by looking only at the optical fiber that connects it. Not realizing that the real functionality is hosted in computational servers.

The computational servers of the brain are these cells called astrocytes. They have been conventionally been classified as glia. Just the white matter that serves as a kind of inactive glue for the brain.

It turns out that the tables are actually turned. It is the neurons that are the wiring and the processing centers exist in the astrocytes. The entire field of neuroscience is named after the wiring and not the stuff that does the work!

Vertebrates (living things with a backbone) have evolved a complex adaptive immune system that allows their bodies to ward off pathogens that it has not previously encountered.

The method of vaccination is to teach our immune system to pre-recognize signatures of harmful pathogens like viruses. The immune system is a very sophisticated learning system that is able to remember previous infections over several decades.

Biology has the habit of using what it already has available. So whatever makes the vertebrate's immune system adaptive is perhaps the same thing that the brain leverages to make it also adaptive. Why reinvent the wheel?

The way the immune system cells (T-cells) remember their targets is by replicating themselves. Memory is implemented by robustly creating multiple backups. It's like storing information in a RAID drive.

The conventional notion of neurons storing memory is more analogous to storing information in DRAM. The moment the power is turned of, then it's all is gone! This doesn't explain why we still are ourselves after we go under anesthesia.

Biological brains turn into jello if we don't sleep. Maybe not jello, but it'll feel like it turns to jello. Humans will die if they don't sleep. It is the astrocytes that do the work to clean up all the junk that we've accumulated during the day.

Astrocytes are syncronized with the circadian rhythm to kick in an perform a lot of house cleaning. Not only are they physically removing the waste, but they are also reorganizing the brain's information. Defragmentation of a hard drive may be a good analogy.

Essentially the brain's information is indexical. That's the purpose of the hippocampus. It's a contextual indexing system that has rapid access to different memories.

The interesting thing about the hippocampus is that it generates new astrocytes. Not only is memory eventually stored, but an index is created. Why waste energy storing something that cannot be retrieved?

This indexing process happens not while we are awake but rather when we sleep. That's when the astrocytes are decoupled from our sensor and motor neurons. This is when they can efficiently reorganize themselves into coherent clusters.

It is through this organization that stuff that might have an indexical similarity is connected to each other. That is how the analogy process that is the core of our thinking is enabled. We know that two things are related because it was previously stored as being related.

The amortized inference process is related but different from the process performed by artificial neural networks.

DL networks are unable to create their own abstractions for a reason. This is because their inference process has hardwired the curves that require its fitting.

Biological brains are architected to perform a kind of top down inference. In short, they see the whole before they see the parts. DL systems are implemented in the reverse. The parts are composed to see the whole.

But as the saying goes, if you focus only on the trees, you fail to see the forest. Well, that's why DL networks can't see the forest.

In the 1940s when cybernetics had its start, people had no answer to the questions about gestalt psychology. Why the hell do humans seek to complete the wholes?

@threadreaderapp unroll