This could have been titled “Brain vs. Software” to mean the same thing. The computer-software combination is a phenomenon which is much like yet much different than the animal brain. I use “Computer” hereafter for the combination. The computer is a product of the human brain and it is sort of a descendant of the human brain; it is good at some of the things that distinguish the human brain from the brains of other animals. There is a recent part of the human brain which is somewhat computer like and able to introspect so as to build computer stuff that does some of what the brain wants to do. Some of this computer stuff is vastly better than what the brain can do, such as compute π to 1013 digits.
Yet there are things that the brain does that seem unfathomable to the brain — things to which we lack subjective access — roughly Freud’s sub-conscious. Early AI workers thought that the new brain stuff was all that was important. They gradually learned that this was not so. Confabulation by the brain had evolved to fool us to think that there was always some logical reason for each of our decisions — a reason which computers performing logic would soon learn to emulate. The evolutionary role of confabulation has been speculated on well elsewhere. Kahneman calls these brain abilities Thinking, Fast and Slow. Fast is the old stuff; Slow is the new serial logic oriented processing. The slow stuff evolved because while the fast stuff is fast and usually right, the slow stuff occasionally is right in very important complex situations. Sometimes the sense that we know why we do what we do is correct — but rarely.
This love of logic served well in building computers, both to compute π, do our banking and do computer animation. Logic seems to me to be a idealization of what animals already did — recognize patterns. Logic is the preeminent language to describe patterns and the new brain idealized patterns to rules that were never violated and got both math and computer science.
There is more to the brain than recognizing patterns and I fear that much of the important extra stuff is highly immodular.
The new brain revels in loading up a few ideas for some simple situation and writing a subroutine that always solves some simple problem. Sorting comes to mind. At other times the new brain collects several of these subroutines and builds larger programs without having to understand all of the complexity at once. This is called ‘chunking’ sometimes or merely ‘divide and conquer’. It leads to modular programs. It leads to very complex programs, such as compilers, that seem to always do the right thing even though it takes many “heads full” of ideas to understand the compiler. Today there are highly useful programs for which many programmers are necessary to understand the program, let alone to write it. Sometimes these programmers disperse and yet the programs continues to function. Marvelous.
The early brain which we still use, has many special purposes parts that run simultaneously and send signals every which way. The patterns that it detects summon reflexes and other patterns. Our brain runs some of these patterns ahead in a predictive mode so as to avoid traumatic situations with a negative valence. All that circuitry has a field day playing chess. We have considerable circuitry dedicated to understanding what is happening on other heads. I can make no sense of it.