Robin Hanson’s “The Age of EM”

Work, Love, and Life when Robots Rule the Earth

Hanson uses tools of an economist, an engineer, and a few more, to make many conclusions and guesses. He has thought thru many causal chains for a fairly plausible new world view. People run at different rates, depending on wealth, wishes and importance. There is a lot of work to do and emulation is not cheap. Their subjective habitat is mostly VR.

P 8: “By the way, feel free to skip around to the sections that interest you; only rarely do they depend much on previous sections.” Thank you. Hanson speculates on dates and ems vs. AIs on page 347 in chapter 27.

P 21: There is a section titled “ERA VALUES” which I find opaque. That says more about me than the section. I have no considered opinions on community values. Hanson quotes many scholarly articles to support his claims. I shall have to see how those claims are used to support his conclusions. About my only opinion is that the American economy and politics perform well because Americans mostly govern themselves politely. We do well because we have a democracy but we still have a democracy because we are civil to each other. The advanced economies all do this.

P 27: Section: LIMITS:
Hanson uses the traditional notion of “the economy”. Well, he is an economist. I think we are even today veering away from the physical notion of economy. Google produces great wealth but ships very few goods. The wealth they produce is not constrained by the limits Hanson describes. Each year we produce far more wealth than is counted in our GDP. If mana from Heaven decreases out GDP and that is bewailed then GDP is no longer a figure of merit.

Hanson goes on to mention searching for algorithms, which is sort of like what I suggest above. I think the mathematicians will be amused indefinitely. Hanson explores ideas such as these and dismisses them as providing limited social value.

P 28: Hanson goes onto a crypto Malthusian perspective, with plausible logic. Yet there are many games to play with 10⁹⁰ particles. Energy is a bigger problem.

P 49: Complexity:
Hanson makes several relevant points about understanding the brain and how that limits us from redesign. If you measure complexity by size of “source code” in bits however, there are several software projects today whose size exceeds our brain DNA. But no one claims to understand such software except in small pieces. Such software indeed displays the inertia that Hanson remarks on.

Another path is learning the DNA language and brain morphogenesis. There are many details, but not many fundamental ideas to be discovered. Then we can improve the brain at least regarding birth canal limitations.

P 51: “A copy can be made at any time of any emulation, after which the two versions diverge because of differing inputs and random fluctuations.” I would imagine deterministic execution but that might not be a good engineering choice. The two versions would have very large mutual information for a substantial time. That bears on cost of storage, perhaps.

P 52: “(Our focus here is on AI that does human jobs well and not passing a “Turing test”.)” Agreed. Such a category includes almost all industrial software since commercial computers appeared in 1953. Computing orbits was once a human occupation—no more. McCarthy lamented that many AI challenges were met only to be relegated to non-AI. Still they are important progress. Neural nets are a new thrust.

There are fascinating reports here on estimates from experts on the arrival of AI and “general AI”. My own non-expert opinion is that general AI will remain elusive because there are many components to it some of which are not profitable to pursue. I think that many think that there is some silver bullet that once found, will make these many components easy, maybe even free. I think they evolved separately and share little mechanism. The very noun phrase “general AI” may mislead. I would be mightily impressed even with general puzzle solving.

Hanson makes similar points on the next two pages. Mixtures of neural nets and conventional software will play a role here. AlphaGo resorts to combinatorial logic to evaluate a board position with a potential ladder. This logic is provided by conventional software. The heavy (novel) lifting is in hardware neural net ASICs.

P 56: “That is, even when ems can read each other’s minds, they may pretend that they cannot.” Just like today.

P 61: Hanson describes several security issues well. I shall study them. Some apps today refuse to work unless they can ‘phone home’. A logic ‘crypto module’ might be strategic. Such would be subjectively (to the em) like a smart phone but with less chance of loosing it.

P 64: Hanson makes many comments about speed-power tradeoffs that seem about right. I shall be nit-picking them. A given em changes state slowly in comparison with its total state. I hope it is feasible to exploit this ratio when saving and transmitting this state. The relation is between processing and memory is not clear to me. That relation has not much changed in von Neumann architectures but will be very different here.

That is, it cost about the same to give an em a minute of subjective experience, whether that happens over an objective second or in an objective day.

The next page covers this pretty well.

P 65: Brains sleep. Perhaps processing needs are different then. Perhaps we have specialized hardware for sleeping.

P 94: Hanson avoids the common economics pitfall of assuming an equilibrium. He describes the em city more nearly like an explosion than a static configuration.

P 96: Hanson describes consumer demand for variety and the impact of that on production. I agree that ems will demand variety but in what? An em owns data structure stored in real physical memory some where. An em presumably buys electricity the cost of which varies erratically. An em will rely indirectly on a great deal of infrastructure which will surely be multiplexed among many ems, presumably by a corporation. An em will be ‘surrounded’ by virtual reality and someone may figure out how to produce and charge for pleasant virtual surroundings.

An em won’t need a car or a house, or a fancy dinner, except virtually. Hanson has not gotten to the ‘economy’ yet that describes the em’s money flow. I think he is working up to that.

P 104: Hanson describes a group ‘undo’ facility to overcome faux pas etc. Some of the obvious difficulties are explored. I can imagine a movie script already. I think that a perpetual count of how often each participate pushed the button is warranted.

P 113: Hanson raises the issue of giving ems authentic access to the real world. On page 116 protocols are proposed to detect deception. Keykos takes a different approach here but does not deal with ems, yet. This idea arose here in the book. A similar pattern is explored much more extensively beginning on page 172 where quantified information leakage is suggested, like the data muffler.

Law enforcement today wants to be able to see into your smart phone, with a warrant, of course. It has been told that it cannot see into your brain. I would like to be able to do public key crypto in ‘my head’ and keep the long keys there and without law enforcement access even with a warrant. This will be a contentious issue.

P 125: Hanson supposes law enforcement with ‘root access’ at least regarding visibility. It is hard to draw the line so that private keys and ordinary memories are concealed. I think I fear such law enforcement more than lack of such enforcement. I am a 2nd rate anarchist.

P 129:

If complex software tools are saved for rare use in distant futures, ems who know how to use those tools well may be archived along with such tools, to be revived when these tools are needed.

P 132: Hanson’s ‘ghosts’ are a fun and convincing metaphorical exercise. You can bid such a ghost to resume ‘reality’ if you and it wants.

P 134: Regarding death: I suspect that there is some fear-of-death in our DNA but much more in our culture. My subjective feeling is a wish to preserve my memories in some meaningful way; I am ‘vested in them’. Thus my web site. Old people, especially men, commonly have an urge to tell stories and ‘explain things’, which I suspect has an evolutionary explanation. I have an Alcor ‘neuro’ contract as another hedge.

Today, erasing the last copy of some valuable software might be an enormous loss, while erasing typical copies costs very little.

The merged em might remember the events that both ems experienced and have the added skills that both ems acquired.

• Expanded human memory,
• Memory merging of recent clones,
• Cheaper memory.

P 136:

The spur is seen to live on because its original lives on, even if the original doesn’t remember the spur’s experience.

P 154: Hanson presents and adopts the Malthusian perspective. I have not understood organized descriptions of why we seem not to be trapped in the Malthusian vice today. Vaguely it seems to be related to the fact that we have an unstable economy. Economics models favor stability, since Malthus and earlier. The book’s model is mainly one of stability, but with some rate of growth. Hanson uses a time unit: ‘economic doubling time’ and suggests that intellectual property typically looses value proportionally, due to obsolescence. This allows some reasoning in an unstable era. The opposite to stability is the ‘Singularity’ perspective which abjures questions of ‘what then’. We live in exiting times, already. (Hanson deprecates the Singularity on page 347.)

P 181: Beginning in the section “New Institutions” Hanson has fun suggesting outrageous institutions. I like most of them. That they would be more likely in em-world is still a bit weak.

P 183: “Very secure and anonymous communications between willing parties can be arranged via “public key cryptography” wherein each person publishes a public key for which they can prove that only they know the matching private key.” With ordinary brain function you can’t easily prove that you have not told others the private key. With ordinary brains even remembering a private key is difficult. A very small auxiliary crypto unit closely integrated with the em would solve these problems. Something rather like a wired in smart phone would solve quite a few related problems. Police authorities tend to object unless they can subpoena the content of these.

P 185: I am glad to see a current concise description of Hanson’s “Combinatorial Auctions”. I think they are promising. I have pondered some of these issues.

Ditto “Prediction Markets”. Speaking of hyperdisasters, ems could establish an outpost on Mars, or L5 with enough technology to slowly recover if the Earth were wiped out. This would be a much lower cost than sending and supporting bios.

I think it would be fun to go explore and mine the asteroids as an em. All those clumsy and fragile space suits would stay behind. Life support systems should be much cheaper and reliable. Digital backups would alleviate many worries. Hanson speaks briefly of space travel on page 225. He foresees some but not much.

P 220: “City Auctions” is an interesting proposal for managing a city which is probably superior to today’s practices which involve a great deal of inflexible and time consuming regulation. It would be fun to explore this as a game and contrast it with ordinary property ownership.

P 224: Fast ems find real travel problematic. Sure, they could pause during travel but they go to the expense of being fast because they are in a hurry. For those it would be a throwback to the days when it took a few months to visit Paris.

P 229: Punishment: Prompt termination upon clear evidence of misdeed is rather effective. Lesser malfeasance would suggest life with greatly diminished authority.

P 230: Hanson suggests that clades will be like a single organism in many ways including legal. I think the logic is correct. This is a throwback of a few thousand years. It bears on institutions such as democracy, perhaps negatively.

P 243: Under ‘Conflict’ Hanson seems to implicitly assume that inequality is necessary and sufficient for conflict. I grant that there is some causality there. Plain greed and crime are other conflict sources.

I suspect that large clans will be less common than Hanson assumes. Clan specialties for which large numbers of members are needed to do the job, are just those most subject to automation, of the AI sort—the sort we do today. I have a higher estimate than Hanson of how much diversity there will be in the tasks. There will be exponentially growing space of specialties which will require exponentially large space of training. I am not sure how those changes impact Hanson’s conclusions.

P 257: During meetings of ems of normally different speeds, it may be mutually beneficial to run at more nearly the same speed.

P 260: Hanson addresses totalitarian thrusts— their strengths and weaknesses. Even more than in today’s world this bottoms out in fabrication of infrastructure. It takes, and will take large resources to build hardware for many ems. Not quite so many resources for modest numbers. How to build such hardware might conceivably be a well guarded secret—a practical monopoly. The ability to ‘shallow mind read’ depends greatly on technical details of such infrastructure. In the mind games I play, in light of the digital and logical facts, the defense has the advantage over the offense leaving the world polycentric. While crypto is hard, you can build good crypto.

Hanson speaks well of ‘market speculators’. I like market speculators but they rely on some notion of property rights. In Jane Jacobs’s terminology the guardians must succeed for the merchants to prosper.

“To use decision markets to get good advice, however, such regimes would need to involve outside judges and financial institutions, and allow informed traders to make secret trades and to hold assets outside of the regimes reach.”

P 263: Hanson speaks of governance of clans. I am a non-joiner. I know quite a few other non-joiners. I think I have worked well in companies and enjoy working with people who are different from me. If I were to spawn a clan I imagine it would soon fracture.

P 268: On fashion: Even today in VR a participant has much control on his appearance. That will play a significant role for ems.

Close clan members can do good movie reviews for each other!

P 343: Hanson discusses tinkering with the simulation so as to modify what is wanted or valued. My first reaction is that I would not want that done to an emulation of me. On further consideration it is less than entirely evident. It is a conundrum.

P 348:

Advocates of this explosion [of intelligence] scenario believe that there exists an as-yet-undiscovered but very powerful architectural innovation set for AI system design, a set that one team could find first and then keep secret from others for long enough. In support of this belief, advocates point out that humans (1) can do any mental task, (2) beat out other primates, (3) have a common IQ factor explaining correlated abilities across tasks, and (4) display many reasoning biases.

P 363: Hanson quotes Ventura and Voth claiming that old wealth in England failed to invest in industry and lost out. I agree. I have not read that paper but in Institutional Evolution Allen claims that the aristocracy felt that their wealth stemmed from fealty to the king and that the merchant class was a status they could not aspire to and remain in the aristocracy. These values were on display in the movie about Beatrix Potter whose parents were not rich, but aspired to aristocratic values. When Beatrix wanted to invite a potential publisher, who was actually rather more wealthy than the Potters, to visit, her mother objected with the comment that the publisher was a ‘tradesman’ and might soil the furniture. This was rather a cartoon depiction but indicated a dichotomy; wealth is one thing, but status is something else. The aristocrats would loose status if they increased their wealth thru commerce. Their fealty to the king would come into question. Ironically the Potters were well enough off thru merchant ancestors, something that Mrs. Potter wanted to hide.

P 382: In the ‘Finale’ chapter Hanson lists suggestions by others who think some other social transition will be more important than ems. The collection is collectively hilarious, even if thoughtful.

The only scenario that bears on this vision that I give some thought to is a decrease of market value of the skills of an ever larger portion of our population. Ditch diggers are gone. etc, etc.

Often severe political problems are solved by death. Stalin comes to mind. (see P 259)

• Compute, measured in Boolean ops, which will cost at least kt ergs each, even with reversible computing. The capital cost of the circuit must also be included.
• Storage of at least several time scales from salt to flip-flop. There are the usual parameters to storage; cost of BW, cost of bit storage, power to write, to read, write budget, access latency, failure modes, etc.