The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. Register now!
The history of artificial intelligence is filled with theories and attempts to study and replicate the workings and structure of the brain. Symbolic AI systems tried to copy the brain’s behavior through rule-based modules. Deep neural networks are designed after the neural activation patterns and wiring of the brain.
But one idea that hasn’t gotten enough attention from the AI community is how the brain creates itself, argues Peter Robin Hiesinger, professor of neurobiology at the Free University of Berlin (Freie Universität Berlin).
In his book The Self-Assembling Brain, Hiesinger suggests that instead of looking at the brain from an endpoint perspective, we should study how information encoded in the genome is transformed to become the brain as we grow. This line of study might help discover new ideas and directions of research for the AI community.
The Self-Assembling Brain is organized as a series of seminar presentations interspersed with discussions between a robotics engineer, a neuroscientist, a geneticist, and an AI researcher. The thought-provoking conversations help to understand the views and the holes of each field on topics related to the mind, the brain, intelligence, and AI.
Biological brain vs artificial neural networks
Many secrets of the mind remain unlocked. But what we know is that the genome, the program that builds the human body, does not contain detailed information of how the brain will be wired. The initial state does not provide information to directly compute the end result. That result can only be obtained by computing the function step by step and running the program from start to end.
As the brain goes through the genetic algorithm, it develops new states, and those new states form the basis of the next developments.
As Hiesinger describes the process in The Self-Assembling Brain, “At each step, bits of the genome are activated to produce gene products that themselves change what parts of the genome will be activated next — a continuous feedback process between the genome and its products. A specific step may not have been possible before and may not be possible ever again. As growth continues, step by step, new states of organization are reached.”
Therefore, our genome contains the information required to create our brain. That information, however, is not a blueprint that describes the brain, but an algorithm that develops it with time and energy. In the biological brain, growth, organization, and learning happen in tandem. At each new stage of development, our brain gains new learning capabilities (common sense, logic, language, problem-solving, planning, math). And as we grow older, our capacity to learn changes.
Self-assembly is one of the key differences between biological brains and artificial neural networks, the currently popular approach to AI.
“ANNs are closer to an artificial brain than any approach previously taken in AI. However, self-organization has not been a major topic for much of the history of ANN research,” Hiesinger writes.
Before learning anything, ANNs start with a fixed structure and a predefined number of layers and parameters. In the beginning, the parameters contain no information and are initialized to random values. During training, the neural network gradually tunes the values of its parameters as it reviews numerous examples. Training stops when the network reaches acceptable accuracy in mapping input data into its proper output.
In biological terms, the ANN development process is the equivalent of letting a brain grow to its full adult size and then switching it on and trying to teach it to do things.
“Biological brains do not start out in life as networks with random synapses and no information content. Biological brains grow,” Hiesinger writes. “A spider does not learn how to weave a web; the information is encoded in its neural network through development and prior to environmental input.”
In reality, while deep neural networks are often compared to their biological counterparts, their fundamental differences put them on two totally different levels.
“Today, I dare say, it appears as unclear as ever how comparable these two really are,” Hiesinger writes. “On the one side, a combination of genetically encoded growth and learning from new input as it develops; on the other, no growth, but learning through readjusting a previously random network.”
Why self-assembly is largely ignored in AI research
“As a neurobiologist who has spent his life in research trying to understand how the genes can encode a brain, the absence of the growth and self-organization ideas in mainstream ANNs was indeed my motivation to reach out to the AI and Alife communities,” Hiesinger told TechTalks.
Artificial life (Alife) scientists have been exploring genome-based developmental processes in recent years, though progress in the field has been largely eclipsed by the success of deep learning. In these architectures, the neural networks go through a process that iteratively creates their architecture and adjusts their weights. Since the process is more complex than the traditional deep learning approach, the computational requirements are also much higher.
“This kind of effort needs some justification — basically a demonstration of what true evolutionary programming of an ANN can produce that current deep learning cannot. Such a demonstration does not yet exist,” Hiesinger said. “It is shown in principle that evolutionary programming works and has interesting features (e.g., in adaptability), but the money and focus go to the approaches that make the headlines (think MuZero and AlphaFold).”
In a fashion, what Hiesinger says is reminiscent of the state of deep learning before the 2000s. At the time, deep neural networks were theoretically proven to work. But limits in the availability of computational power and data prevented them from reaching mainstream adoption until decades later.
“Maybe in a few years new computers (quantum computers?) will suddenly break a glass ceiling here. We do not know,” Hiesinger said.