LECTURE 3

3.1) PHYSICAL DEFINITION OF AN "AUTONOMOUS AGENT"

3.1.1) The Carnot Cycle:

i. Two temperature reservoirs, T1 > T2.

ii. A working gas in a cylinder with a piston.

iii. A "handle" allowing working gas cylinder to be pushed or pulled against T1 and T2 reservoirs.

iv The cycle acting as a pump:

a. Start with piston compressed, working gas at temperature T1.

b. Pull handle to place working gas cylinder in contact with T1.

c. Allow isothermal expansion of working gas as first part of power-stroke.

d. Push handle to isolate working gas from T1.

e. Allow adiabatic expansion of working gas in second half of power stroke, thus lowering temperature and pressure in gas to T2.

f. Push on handle to place working gas cylinder in contact with T2 reservoir.

g. Push on piston from outside, doing work on it, to compress working gas under isothermal conditions.

h. Pull on handle to isolate working gas cylinder from T2.

1) Continue to push on piston from outside to do work compressing working gas adiabatically back to starting condition of high temperature, T1, and pressure.

v. The cycle acting as a refrigerator:

a. Reverse the sequence of actions.

In this case, the cycle works to transfer heat from the cooler, T1, reservoir to the hotter, T2, reservoir.

vi. Critical feature of Carnot engine: The "SAME device, run in the two opposite directions, carries out TWO DIFFERENT TASKS - doing work on the piston as a pump in one, transferring heat in the other.

vii. Call the two directions of the cycle the "+" and the "-" directions. Generically, these two directions of the "same" device carry out different tasks.

3.1.2) Catalytic Task Space.

i. We begin with the concept of a "Shape Space." (Oster and Perelson).

a. Consider an N dimension compact space. Let three dimensions represent the three spatial dimensions. Other dimensions represent charge, dipole moment, bulk, hydrophobicity, and other features. In general, each axis is bounded physically between maximal and minimal values, e.g. + and - 10 charges. Hence the space is compact.

b. Let a point in this space represent a "shape."

c. Perelson and Oster considered an Antibody as "covering a ball" in shape space, where the ball represents the set of "epitopes" the antibody is able to "bind."

d. The antibody covers a "ball" because shape recognition is not infinitely precise. Some slop creates a ball of finite volume.

e. The simplest immune systems have a diversity of about 10,000 different antibody molecules. Perelson and Oster reckon that, to be useful in evolution, these 10,000 "balls" must cover an appreciable fraction of shape space, say 1/e. From this they can calculate the volume of a "ball" as a fraction of shape space. Then by a Poisson argument that each antibody ball is randomly located in shape space, they conclude that the human antibody repertoire of 10'8 balls will SATURATE shape space!

Three ideas:

* Similar epitopes have similar shapes.

* A finite number of antibody molecules can COVER shape space, hence recognize all shapes.

* Very different epitopes can have the "same" shape. Endorphin and opium are classical examples.

ii. Next, think of a similar N dimensional Catalytic Task Space. Each point in the space represents a "Catalytic Task." Just as an antibody molecule covers a ball in shape space and carries out a "binding task," so AN ENZYME COVERS A BALL IN CATALYTIC TASK SPACE.

Three ideas:

* Very similar reactions can represent nearly the same catalytic task, hence lie in the "SAME BALL" in Catalytic Task Space.

* A finite number of enzymes might cover Catalytic Task Space - hence a universal enzymatic toolbox might exist.

* Very different reactions might represent "THE SAME CATALYTIC TASK." Evidence from enzymes that catalyze apparently different reactions via the same catalytic site.

iii. Define a "Catalytic Task": Roughly a catalytic task corresponds to binding the transition state of a reaction with high affinity, the substrate and product states with low affinity.

a. Catalytic antibodies as evidence: Immunize with stable analogue of transition state. Resulting monoclonal antibodies have about 1/10 chance to act as catalyst for corresponding reaction.

3.1.3) Closure of Collectively Autocatalytic Set in Catalytic Task Space

i. Consider a collectively autocatalytic set of molecules.

Let the set have "food" molecules, [F] acting as exogenous inputs to the set, plus further product molecules, [P], where the union of [F] + [P] has the property of collective autocatalysis. Every member of [P] has some last step in its formation catalyzed by one or more members of [F] + [P].

ii. Consider both the reaction graph of this autocatalytic set, and the set of "balls" in Catalytic Task Space that represent the reactions catalyzed in the autocatalytic set.

iii. The collectively autocatalytic set achieves CATALYTIC CLOSURE. Every member of [P] has some last step in its formation catalyzed by one or more members of [F] + [P].

* Note that achievement of catalytic closure is a fully objective property of a chemical reaction system. Either a set of molecules taken as substrates, products, and catalysts does, or does not achieve catalytic closure.

vi. Consider the set of balls, say M, in Catalytic Task Space that represent the M reactions in the autocatalytic set. The M catalytic tasks have the property that, given the proper sets of substrates and products for each task, namely those comprising the collectively autocatalytic set, the tasks are jointly carried out by the molecules.

* Due to the fact that the same ball in task space can represent very different reactions that comprise the "same" task," dramatically different choices of inputs to each of the M tasks might lead to output products that were not inputs to others of the M tasks in the set of M tasks. Thus, closure in the space of M tasks would NOT be achieved by such a set of inputs to each of the M tasks.

* Therefore, closure in the space of M tasks, such that the set of molecules jointly carries out the M tasks requires more than merely substrate inputs and outputs to each task, rather closure in the space of M tasks requires particular sets of substrate and product molecules, namely those comprising a collectively autocatalytic set.

3.1.4) Duality of Closure of Tasks and Molecules Instantiating One Another.

i. The point has been demonstrated just above. In a collectively autocatalytic set of molecules, there exists a corresponding set of M tasks in Catalytic Task Space. The particular set of molecules in the collectively autocatalytic set has the property that, taken as substrates, products, and catalysts, the set of molecules JOINTLY CARRY OUT THE M TASKS. Hence the M tasks are CLOSED or "COMPLETED" under this set of molecular inputs and outputs to the tasks.

Conversely, given the set of M tasks, the molecules of the collectively autocatalytic set, [F] + [P], have the property that the set of M tasks JOINTLY COORDINATE the transformations of substrates into products such that the set [F] + [P] IS jointly and collectively autocatalytic. The set of molecules achieves "CLOSURE" or COMPLETION" under the catalyzed transformations among the molecules. Note that with a different set of substrates to the M tasks, the set might not be catalytically closed.

ii. Thus, the set of molecules, [F] + [P] and the set of catalytic tasks, M, are DUALS of one another. The molecules make one another by virtue of the coordination among the set of M tasks, given those specific molecular inputs and outputs. The tasks "jointly get themselves done" by virtue of the molecules whose transformations they coordinate.

3.1.5) Closure in Catalytic Task Space as a New Concept with Physical Meaning.

i. As noted above, a set of molecules taken as substrates products and catalysts either does or does not achieve catalytic closure and closure in task space, either is, or is not collectively autocatalytic. Hence the difference is physically real, and clear.

ii. A collectively autocatalytic set is a particular ORGANIZATION of matter, energy, and process. This particular organization has the property that it

a. Can maintain itself.

b. Can, with minor additions, reproduce and evolve.

c. Hence, can form the basis of life.

3.1.6) General Task Space where the Tasks can Perform Carnot (and other) Work Cycles.

i. We need to create the concept of a general task space, an analogue of Catalytic Task Space, where the tasks can include the carrying out of linked networks of Carnot Work Cycles in the "+" or in the "-" directions.

ii. I return to "task space," in terms of "propagating work"

and the insight from Atkins that work is the constrained

release of energy, while the very construction of those

constraints itself requires work. Work begets constraints

begets work. The linked web of propagating work carries

out the "tasks," and these must achieve a self-consistent closure in just the same sense that collectively autocatalytic sets achieve catalytic closure.

iii. However, as I will stress below, it is not obvious that one can formalize this task space in some foundational "BASIS SET" of TASKS such that all other tasks are logically constructable as compositions of this basis set. Indeed, I strongly doubt that this is possible. Rather, as I will argue below, the set of Tasks may only be definable in Context Dependent ways that may not be reducible to a basement Basis Set.

3.1.7) Preliminary Definition of an Autonomous Agent

i. Motivation: Consider a bacterium such as E. coli, swimming upstream in a glucose gradient to "get dinner." We unhesitatingly are willing to say of this E. coli cell that is an Autonomous Agent (in its open thermodynamic environment) that is able ACT ON ITS OWN BEHALF.

ii. It is precisely because the E. coli cell can carry out thermodynamic WORK CYCLES that it can repeatedly ACT.

iii. It is precisely because E. coli is a living cell, a collectively reproducing organization of matter, energy, and process capable of self maintenance, reproduction, and evolution, that we unhesitatingly think of the E. coli cell as an AUTONOMOUS AGENT.

iv. THEREFORE, DEFINE AN AUTONOMOUS AGENT AS A NON-EQUILIBRIUM AUTOCATALYTIC SET OF INPUTS, OUTPUTS AND DEVICES WHICH ACHIEVES CLOSURE IN A SPACE OF TASKS, CATALYTIC, WORK, COORDINATION, CONSTRAINT AND OTHERWISE, SUCH THAT ONE OR MORE WORK CYCLES ARE PERFORMED, WHEREBY THE TASKS ARE JOINTLY PERFORMED, THE INPUTS, OUTPUTS AND DEVICES OPERATING ON THE INPUTS CREATE THE OUTPUTS AND DEVICES, AND BY CARRYING OUT ONE OR MORE WORK CYCLES THE SYSTEM CAN ACT ON ITS OWN BEHALF.

3.1.8) If the devices within the Agent that carry out work cycles are reversed from + to - directions, the set of Tasks change. Generically, no task closure is assured when all devices are reversed from "+" to "-" directions.

i. Let the set of M tasks in the appropriate task space of the Autonomous Agent be represented by M balls, each carrying a "+" sign, where the "+" sign designates that the corresponding DEVICE within the Autonomous Agent carries out that task by being "run" in a "forward" direction - in generalization of a Carnot cycle running in the forward direction acting as a pump, but running in the reverse direction acting as a refrigerator.

ii. In general, if the DEVICES within the Autonomous Agent were run in the reverse "-" direction (in which case each device is actually slightly different), then each device will, generically, carry out some DIFFERENT TASK.

iii. Thus, WERE ALL THE DEVICES IN THE AUTONOMOUS AGENT RUN IN THE REVERSE, "-" DIRECTION RATHER THAN THE FORWARD, "+," DIRECTION, THEN GENERICALLY, ALL THE M TASKS WOULD "JUMP" TO NEW LOCATIONS IN TASK SPACE. IN GENERAL, THERE IS NO REASON TO SUPPOSE THAT THIS NOVEL SET OF M TASKS, M', WOULD ACHIEVE CLOSURE AND DUALITY WITH THE SET OF INPUTS, OUTPUTS, AND DEVICES! HENCE, IF THE DIRECTIONS OF THE DEVICES ARE ALL REVERSED FROM "+" TO "-," GENERICALLY THIS NEW SYSTEM IS NOT AN AUTONOMOUS AGENT.

iv. In short, were all the work cycle devices run in the opposite direction, a novel set of M tasks, M', would exist. There is no general reason to expect this new set of tasks to achieve closure, nor for the inputs, outputs, and devices to be collectively autocatalytic under the set of these reversed device tasks.

3.1.9) This definition of an Autonomous Agent implies that there is NO AGENT AT EQUILIBRIUM.

i. In order to be an Autonomous Agent, a system must carry out work cycles by virtue of which it maintains and amplifies itself. These work cycles require that the Devices carry out work in the "+" direction. At exact equilibrium no work cycles can be carried out. Hence no Agency exists.

ii. Infinitesimally close to equilibrium, work cycles can be carried out infinitely slowly. Thus, it would initially appear that Agents can exist infinitesimally displaced from equilibrium.

iii. However, we will see below that in order to be coevolutionarily assemblable, and to pay the "erasure" cost of measurement, Autonomous Agents must be displaced A FINITE DISTANCE FROM EQUILIBRIUM.

iv. Atkins, in his book on The Second Law, discusses the fact that "work" is the "constrained release of energy. In particular, he considers the case of a steam engine, the cylinder, piston and expanding gas in the chamber. The constraints afforded by the cylinder and piston constrain the way energy is released, thereby constituting work. But work had to be performed to construct the cylinder and piston and assemble them. More generally, work must typically be done to construct constraints. Thus, work begets constraints begets work. No constraints, no work. This issue seems critical, for we either face an infinite regress, or the self consistent coemergence of constraint and work.

v. In the next lecture I return to this issue in the context of "propagating work." Briefly summarized, I point out that a macroscopic state of matter or flow, for example a cannonball in flight, has a large number of causal consequences, any subset of which can be "used for some purpose." Thus, the cannon ball makes a whistling noise serving as an alarm, or it can hit and destroy a house, or hit a sturdy paddle wheel turning the latter, thereby winding up a rope lifting a bucket of water that tilts over the axle into a sluice box. Since the causal consequences of a macroscopic state or process are manifold, as the diversity of such bits of "work" increase, the diversity of causal consequences increases faster, hence the probability of the existence of at least one self consistent web of consequences whereby constraints are constructed and work propagates increases. Once such a self consistent web of constraints and work crystallizes in an autonomous agent, the "task" that each bit of work or structure performs among the full set of its causal consequences becomes defined in the context of the whole agent. Further, "glitches" in the web of causal consequences are bound to be present. If these glitches do not entirely block the propagating work in the Agent, then the glitches themselves become points for control over timing and extent of work done at each position in the system.

vi. Further, the fact that Autonomous Agents do perform work cycles means that they can "ratchet" themselves further from equilibrium, thereby storing energy and also having the capacity to carry out processes with increased precision. (I thank Brian Goodwin for the first point and Philip Anderson for the last point.)

vii. In addition, as Agents and communities of Agents become further displaced from equilibrium, the number of possible alternative arrangements of matter and flows within the system increases, thus the total Adjacent Possible become larger. (I thank Brian Goodwin for discussions on this point.)

viii. In turn, the expansion of the adjacent possible makes the chance discovery of useful novel constraints, flows, and states of matter grafting into the linked web of propagating work, tasks, and self-reproduction progressively easier, abetting the evolution of the community of agents.

viii. In the next lecture I expand on this definition of Agents as systems in which the constrained release of energy at each point in the system enables the propagation of work, and offer a definition of organization as that ordering of matter and energy over time which enables work to propagate. Autonomous Agents construct and evolve their own boundary conditions such that work propagates with the structure that constitutes a record of the embodied "know-how" to perform the work of making a living in an environment.

ix. Finally, I stress that these concepts of propagating work, task and catalytic task closure in Agents is not at all the concept of "entropy," nor is the coordination of tasks such that work propagates and organization propagates the traditional Shannon concept of "information."

3.1.10) The fact that COEVOLUTIONARILY ASSEMBLABLE AUTONOMOUS AGENTS MUST BE DISPLACED A FINITE DISTANCE FROM EQUILIBRIUM IMPLIES AN INHERENT ONE-WAY-NESS OF AUTONOMOUS AGENTS. In turn, time and flow enter the definition of an agent. (I thank Harold Morowitz for emphasizing the last point.)

i. Real Agents, to be coevolutionarily assemblable, will have to be finitely displaced from equilibrium, hence carry out their "+" tasks at a finite (average) rate, and to achieve CLOSURE will "RUN" IN ONE DIRECTION rather than the global reverse around all such cycles.

3.1.11) An example of a biochemical CARNOT CYCLE.

i. Consider two molecular species, A and B, which interconvert. These two will constitute our "working gas."

ii. Let the equilibrium ratio of A to B be 1.0. A work cycle of this working gas consists in a movement of the A-B reaction system from some initial position, P1, with respect to equilibrium, say to the left of equilibrium, to a second distinct position, P2, say shifted right closer to equilibrium, then a second movement of the A-B reaction system back to the initial position, P1.

iii. Let the A-B working gas reaction system be coupled via allosteric enzymes, E1 and E2, to two outside pairs of reactants, C and D in the first half of the Carnot cycle, and F and G in the second half of the Carnot cycle.

iv. Let the equilibrium due to the Gibbs free energy drop of the coupled system A + C <--> B + D lie in the direction that ENHANCES conversion of C to D compared to the equilibrium of the isolated C <--> D reaction pair. Thus, coupling the working gas system, A - B, to the C - D system harnesses the free energy of the A to B conversion to DRIVE the transformation of C into D, thereby driving the C-D reaction pair further to the right than its own isolated equilibrium. In analogy to the power stroke of the Carnot cycle functioning as a pump, the working gas, A-B, has done chemical work on the C-D reaction pair.

v. To accomplish this work, "turn on" the allosteric enzyme, E1, with its appropriate effector ligand, X. Addition of X to the system is analogous to pulling on the handle to the working gas in the cylinder of the standard Carnot cycle example, thereby pulling the cylinder into contact with the hot T1 thermal reservoir.

vi. After the coupled A + C <--> B + D system has driven D production to a higher level than the isolated C - D system, remove X. This is analogous to using the handle to push the working gas cylinder out of contact with the hot T1 reservoir. However, there is no analogue to the adiabatic expansion stage of the cycle.

vii. To restore the A-B working gas to its initial position at P1, turn on enzyme E2 with its ligand effector, Y. This is in analogy to using the handle to push the working gas cylinder into contact with the cooler reservoir, T2.

viii. Let E2 catalyze the coupled reaction F + B <--> A + G, where the greater free energy drop of F --> G allows restoration of the initial A-B concentrations displaced to the left of A-B equilibrium at position P1. Thus, the reaction pair F-G is displaced to the left of its own equilibrium such that, when coupled to the A - B system via E2 and Y, the coupled reaction system restores A - B to P1 before reaching the equilibrium of the coupled F + B <--> A + G system.

3.1.12) Consider an experimentally realizable autocatalytic set:

i. The set consists of a single stranded DNA hexamer, 3'CCCGGG5' and a large number of single stranded DNA trimers, 3'GGG5' + 3'CCC5'. As described above, the hexamer lines up the two trimers by Watson-Crick base pairing and catalyzes the ligation of the two trimers by a 3'-5' phosphodiester bond. The resulting new hexamer is identical to the initial hexamer. Hence, when the double strand melts apart, each can serve to catalyze the ligation synthesis of further hexamers. This is a simple autocatalytic molecular system.

ii. Let the equilibrium of the trimer hexamer system be E. Let the initial system be displaced to the left of E, high concentrations of trimers compared to hexamers. Then the system will spontaneously drive to the right, forming more hexamers until the trimer: hexamer ratio is E.

iii. This little autocatalytic reaction system is a single exergonic reaction. Therefore, the system DOES NOT CARRY OUT A WORK CYCLE. BY OUR DEFINITION, THEN, THIS AUTOCATALYTIC SYSTEM IS NOT YET AN AUTONOMOUS AGENT.

3.1.13) An Example of a Molecular AUTONOMOUS AGENT.

i. The same trimer - hexamer system. This system will be the analogue of the C - D system in the chemical Carnot cycle above.

ii. A "working gas" system, the conversion of pyrophosphate to 2 monophosphates: PP <--> P + P. This system is the analogue of the A - B working gas reaction system in the Carnot cycle above.

iii. A third reaction pair, converting F <--> F + e, by a redox couple. This system is the analogue of the F - G system in the chemical Carnot cycle above.

iv. Two allosteric enzymes, E1 and E2. E1 couples conversion of trimer to hexamer with conversion of pyrophosphate to 2 monophosphates. E2 couples conversion of 2 monophosphates back into pyrophosphate, PP, and conversion of F to F + e, using the free energy of the redox couple to restore the initial level of pyrophosphate.

v. Two allosteric ligands, X and Y, reciprocally turn E1 and E2 on and off. To achieve this, X and Y must oscillate out of phase with one another.

vi. The operation of the Autonomous Agent requires that X and Y and E1 and E2 be members of the set, which I assume here. Further, X and Y must each oscillate between low and high concentrations out of phase with one another, as, for example, via a limit cycle mechanism. A concrete assignment has the DNA hexamer as E1, catalyzing its own formation when coupled to PP breakdown. Phosphate, P, can serve schematically as X, activating the hexamer enzymatic function by binding allosterically to the hexamer. This feedback is like the feedback activation of phosphofructokinase by its product leading to the glycolytic oscillation. Further, one of the trimers can serve as E2, where its enzymatic activity is allosterically inhibited by pyrophosphate, PP.

vii. Then on the forward half of the work cycle, the coupling of trimer + PP <--> hexamer + P + P allows the free energy drop of the breakdown of pyrophosphate to drive the trimer - hexamer system further to the right than the equilibrium ratio of trimer: hexamer. Hence, excess reproduction of hexamer is achieved. Back activation of the hexamer acting as E1 by phosphate, P, occurs as P builds up in the initial stages of the reaction. The activated E1 enzyme then "flushes" the substrates, PP and trimers, into hexamer product. Depletion of PP and trimers slows the forward reactions.

On the second half of the work cycle, the working gas system, pyrophosphate and monophosphate is restored to the initial position to the left of the PP <--> P + P equilibrium via the free energy drop in the F --> F + e redox reaction, coupled via E2 and Y. Timing of the reciprocal activation of the trimer acting as E2 is controlled by release of the PP inhibition as PP concentration falls from the forward reaction.

viii. The Capacity of the Carnot linked trimer: hexamer, PP - P + P, F - F + e system able to do thermodynamic work cycles on its own behalf, hence increase hexamer concentrations far beyond the trimer:hexamer equilibrium via pyrophosphate cleave means that this AUTONOMOUS AGENT is able to OUT-REPLICATE the Non-Agent trimer:hexamer system alone, for the Non-Agent system is limited by the trimer: hexamer equilibrium.

ix. Because the autonomous agent synthesizes excess hexamer with respect to the trimer: hexamer equilibrium, the system has used a work cycle to ratchet itself to a non-equilibrium position and has stored energy in the hexamer's extra phosphodiester bond linking the two trimers.

x. Each cycle of replication drives the total population of agents further from equilibrium in the aggregate. Thus, the stored energy is available for further use in the community.

xi. The linkages of causal consequences of the parts, hexamer acting as E1, phosphate acting as its allosteric activator, trimer acting as E2, pyrophosphate acting as its allosteric inhibitor, exhibits the cross coupled construction and timing of constraints whereby the system propagates linked work around a work cycle.

xii. The discovery of a "self replicating peptide" Nature August 7, 1996, is a stepping-stone to a peptide autonomous agents. The authors created a 32 amino acid helical peptide able to ligate two fragments, 15 and 17 amino acids long, that comprise the 32 amino acid sequence. This little peptide autocatalytic system, like the first hexamer trimer system above, is merely an exergonic reaction system, not yet an autonomous agent, for it carries out no work cycles.

3.1.14) Other potential examples of Autonomous Agents.

i. Nothing in the formal definition of an Autonomous Agent requires that it be at the scale of modest numbers of interacting molecules, like cells.

ii. von Neumann defined a self reproducing system imagined to be a Universal Constructor floating (swimming) in a (mercury) pond filled with floating (metal) parts. The Universal Constructor - an analogue to a universal computer - was to contain instructions, move about on the pond, pick up parts and assemble them into a copy of itself, then copy the instructions directing its activity, and insert the copy of the instructions into the copy of itself, creating a fully competent duplicate of itself which would then be released to function as a new self reproducing machine.

a. von Neumann's self reproducing machine would have to carry out real work cycles to act. Therefore his hypothetical machine would, if instantiated, be an autonomous agent.

b. Note that von Neumann assumed a UNIVERSAL CONSTRUCTOR. This assumption then required separate Instructions. But cells and other collectively autocatalytic systems are NOT UNIVERSAL CONSTRUCTORS.

Cells are special purpose constructors, ones that happen to create themselves via closure in the appropriate task space.

c. We will have to ask whether von Neumann's postulated self reproducing physical system is coevolutionarily assemblable. It is not obvious how such a Universal Constructor plus Instructions could initiate.

iii. Since nothing in the definition of an Autonomous Agent demands that it be made at the molecular scale of interacting organic molecules, what else might one consider as CANDIDATE AUTONOMOUS AGENTS?

a. Who knows?

b. Lasing in a cavity with the proper gain medium?

Here one might get collectively amplifying sets of wavelengths which might do work, graze on the gain medium, and form an ecosystem of light agents which amplify and coevolve. For example, induced transition amplifying any given wavelength are a bit like replicators. Coupled wavelengths around a cycle where the sum of two reddish photons might induce an initial high energy blue photon might be like Eigen and Schuster's hypercycles.

c. What of other entities such as mutually gravitating objects? Thus, self organization in spiral galaxy evolution is known. As discussed in more detail in Lecture 6, dust generation and complex chemistry in cold molecular clouds is itself part of star formation. Star formation within giant molecular clouds is apparently driven by supernovae induced shock waves. In turn stars form complex nuclei and help drive complex chemistry in cold molecular clouds. Gravitation is an autocatalytic, self amplifying force. Can a spiral galaxy be understood as an ecology of autonomous agents?

d. Are there Cosmic strings? Might such systems form Autonomous Agents? Universe as a whole?

3.1.15) The Capacity of Autonomous Agents to Evolve, Coevolve, create Ecologies and Economic Interactions with Advantages of Trade. Agents as COMPLEX ADAPTIVE SYSTEMS.

i. An E. coli cell is an example of a Molecular autonomous agent. It is a collectively autocatalytic set of molecules that links endergonic and exergonic reactions such that it can carry out thermodynamic work cycles as it reproduces itself. It can perform work on its own behalf.

ii. For some billion years after the start of life, bacterial metabolic communities dominated the planet. In due course, these were augmented by simple single cell eukaryotes in metabolic communities.

iii. Note that in the evolution of these cells, NOVEL proteins and novel chemical reactions came to be INCORPORATED into the cells and their metabolic traffic with one another. In this way, the BIOSPHERE has expanded into the ADJACENT POSSIBLE at the level of organic molecules.

Why this expansion into the Adjacent Possible. Consider a community of bacteria, perhaps locally on the subcritical - supracritical boundary. If a novel organic molecule is produced that, in the current context, confers an advantage to the cell producing it, it will tend to be incorporated. For example a genetic mutation altering a protein such that it can now metabolize a previously present organic molecule into a new product able to be exported as a toxin to ward of attack would confer such an advantage on the mutant cell. So would production of a novel molecule that attracted a metabolic mutualist into a stable alliance - like micorrizhal fungi in plant roots.

iv. Once such advantages of trade exist, the mutualists can evolve to optimize the trade ratio, finding that exchange rate that maximizes reproductive rate under R selection, of sustainable abundance under K selection. Roots are again an example where several different fungal species coexist in one root, competing with one another within an envelope of cooperation created by the metabolic mutualism of nitrogen fixation by the fungal cells and carbon fixation into sugar by the root, and the exchange of these metabolic goods and services.

v. Note that the capacity of such cellular Autonomous Agents to coevolve with one another, each of which does thermodynamic work cycles by linking exergonic and endergonic reactions, plus the capacity of each to graft EVER NEW metabolic functionalities into its organization for its own benefit either directly or via trade or competition, implies that the COEVOLVING ECOSYSTEM OF COLLECTIVELY AUTOCATALYTIC SETS can, AND DID, CONSTRUCT AND CREATE an ever widening web of linked exergonic and endergonic reactions spread throughout the entire coupled metabolic ecosystem. In this coupled web, work propagates via the work done to create the constraints on the release of energy which itself constitutes work.

vi. Indeed, precisely because each Autonomous Agent internally links exergonic and endergonic reactions, and propagates work each is displaced from equilibrium, stores energy and has higher internal concentrations of molecules whose endergonic synthesis is driven by coupling to exergonic reactions by the work cycles within that Agent. But just these high concentrations of otherwise low concentration compounds increases the number of possible arrangements of matter and energy flows, hence breaks more symmetries, and therefore allows more useful combinations of work and structure to arise. This in turn abets the invasion of the ADJACENT POSSIBLE. One can think of the metabolic community of Autonomous Agents as creating high concentrations of endergonically synthesized compounds via coupling to exergonic reactions as creating a kind of molecular "Salient" in the Actual. These surprisingly high concentration molecules are poised next to particular sectors of the Adjacent Possible, and driven into it by mutation and selection of fitter variants.

viii. Thus, our coevolving system of Autonomous Agents can, and did, construct ecosystems with economic features, did build up complex webs of coupled exergonic reactions that were functionally important not only within individual Agents (cells), but between them as well.

The coevolutionary process has created Darwin's vast tangled bank of life - a tangled bank with a profusion of linked exergonic and endergonic reactions that, simultaneously, undergirds the advance from the molecular Actual to the molecular Adjacent Possible.

3.2) NATURAL GAMES

i. Game theory is built upon axioms including "players," "payoffs," "payoff matrices," "strategies," .....

ii. How many formal games are there?

a. First order infinity - denumerable - of possible computer programs.

b. We could define arbitrary payoff matrices for print out of any such program.

c. Hence, an infinity of games.

iii. What kinds of "games" occur among coevolving Autonomous Agents?

a. Define such games as NATURAL GAMES.

b. Roughly, a Natural Game is a way of MAKING A LIVING IN A WORLD.

c. Thus, a Natural Game is also a "NICHE."

iv. Not all games are Natural Games.

a. Zero Sum Games are presumably not Natural Games, for all players cannot, on average, make a living in a zero sum game.

v. Natural Games may imply an "Envelope of Mutualism."

a. Consider example of plant root fixing carbon into sugar, and micorrhizal fungi that infect root and fix nitrogen. Several species of fungi typically "co-infect" one root nodule. Thus, the overarching system is a mutualism, yet the fungi compete with one another within this mutualistic envelope. Within that envelope, overall, plant and fungi create value for one another - sugar and fixed nitrogen for further biosynthesis permitting each to survive and flourish.

b. One wonders whether, for non-equilibrium ecosystems of Autonomous Agents that must gather and utilize free energy, there is not always some such overarching mutualistic envelope that frames the Natural Games.

c. The analogue of advantages of trade and price emerge in simple models of bacterial mutualists. As noted above: Utility <-> Rate of reproduction. Increased Utility <-> Increased rate of reproduction = Natural Selection (R selection). Advantage of trade (root fungal system, or plant - Oxygen, animal CO2 mutualism). Price <-> exchange ratio of help to other versus metabolic cost to me that maximizes both Agent's rates of reproduction. Can be analyzed via standard Edgeworth Box, yields not only a CONTRACT CURVE that is Paretto Efficient, but also a unique point on the contract curve where each Agent reproduces as rapidly as its mutualist partner, and at the maximal possible rate. Unique price emerges because Agents "reinvest" profit in their own population Growth.

d. Thus, economic concepts apply naturally to ecosystems with overarching envelopes of mutualism within which competition occurs.

e. Thus, Adam Smith-like Invisible Hands may grace such worlds.

3.2.1) Natural Games are the Duals of the Coevolving Autonomous Agents.

i. Natural Games are the ways Autonomous Agents make a living in their world. Thus, Natural Games are the Niches created by and occupied by the Agents.

ii. The analogue in an economy are the "jobs" and the "job holders" in an evolving economic web. The diversity of ways of making a living has exploded over the past 100,000 years. Most of us "catch rabbits" for dinner in odd ways such as being theoretical biologists, economists, lawyers, etc.

iii. There is a deep similarity between the explosion of biological niches over the past 4 billion years, and especially since the Cambrian explosion and the explosion of goods, services and types of jobs over the past 100,000 years and particularly since the Industrial Revolution.

iv. Thus, the niches are the duals of the Agents that occupy them. Each instantiates the other. Each permits the other to exist. Organisms and niches, in the Kantian sense, are the conditions of one another's existence.

3.2.2) The NO FREE LUNCH THEOREM of Macready and Wolpert appears to have important implications about the cocreation of natural games and players as duals of one another in self-constructing coevolving systems of autonomous agents

The No Free Lunch theorem considers a finite bounded N dimensional space divided into a finite number of subvolumes. Each subvolume is an integer drawn from some range, e.g. 0 to 1,000,000. Any such assignment constitutes a fitness landscape," where the integer is the fitness.

Consider two search algorithms, say mutation and selection on the one hand, and fully random search on the other. Let each search algorithm sample M distinct points on the landscape.

The "No Free Lunch" theorem states that, averaged over all fitness landscapes, no search outperforms any other search algorithm.

Averaged over all landscapes, hill descending does as well as hill climbing or random search.

Thus, the theorem seems to imply that, if biological evolution by mutation, recombination, and selection is "doing well," then the landscapes searched by mutation, recombination and selection must themselves be a subset of all possible landscapes. Namely, biological fitness landscapes must be those that are precisely the ones that are well searched by mutation, recombination, and selection!

But this suggests that the coevolutionary process is jointly and self-consistently creating both the organisms that make a living, and the niche - natural games - that those very organisms can readily search by mutation, recombination and selection.

This point amplifies the aim of Lecture 1 - that recombination requires "smooth landscapes" hence that evolution was probably "constructing" landscapes well searched by recombination.

This hypothesis is testable at the level of proteins by assessing whether protein fitness landscapes, for example with respect to protein receptor- peptide ligand binding, has the character that it is well searched by mutation, recombination and selection in comparison to fully random search. Recent results based on "sexual PCR" to achieve recombination in populations of protein molecules tends to confirm this hypothesis. Also, results based on "applied molecular evolution" generating libraries of stochastic DNA or RNA sequences and evolving useful peptides and polypeptides also is successful.

(It remains to be shown that the capacity to search molecular landscapes reflects any "selection" of readily searched categories of molecules rather than an inherent property of almost any family of molecules. On the other hand, evolution of molecular function almost certainly exploits just those aspects of molecules and their behavior that are most readily found among those molecules. It is therefore easy to find polypeptides that bind one another precisely because folding of a polypeptide upon itself by non-covalent bonds is natural, hence binding to another peptide by similar non-covalent bonds is natural as well. The "functionality" sought by molecular agents comprised in part of proteins, is natural to proteins. Thus, the functionalities sought are readily discovered and exploited. This captures the image of evolution the opportunistic tinkerer, making use of that which lies readily to hand.}

The broader implication of the No Free Lunch theorem appears to be this: Given a search algorithm, some landscapes are well searched by it, others are not. Consider the niche of an organism as its evolutionary or coevolutionary search space. Those niches - functionalities and natural games - which are readily searched by the organism's search processes, mutation, recombination, and selection in a coevolutionary context, will be the niches that are exploited. Thus, the winning organisms will expand into those niches - natural games - that are readily explored. The winning players come into existence self-consistently with the winning games.

If Players and Games are the Duals of one another, then those patterns of making a living - jobs - that afford the greatest growth opportunity (in economics this includes increasing returns by climbing up learning curves and entering new markets) will tend to come into existence and proliferate jointly with the job-holders. Old jobs will go extinct.

3.3.3) Autocatalytic growth of Niche and Agent Diversity.

i. In the economic system, the diversity of goods and services creates a "WEB" that sets the stage to its own transformation to a new web with new goods and services.

ii. Deep analogy between SUPRACRITICAL CHEMICAL REACTION SYSTEM and EXPLODING DIVERSITY OF GOODS AND SERVICES.

iii. Grammar Models for chemistry and economics:

a. Think of symbol strings as standing for molecules or goods and services.

b. Think of a "grammar" as a finite table listing permitted ways that strings, acting on strings (like enzymes or tools) can create new strings.

c. Supply each symbol string "good" with a "utility" to "THE CONSUMER."

d. Social Planner model of an evolving technological web. Each stage of the economy has a set of goods and services, which allows the introduction of further goods and services that can "make a profit" in the Adjacent Possible economy. Over time, the economy can propagate into the Adjacent Possible, creating new goods and services, hence new jobs, hence new niches for job holders. Indeed, the existing web of goods and services creates niches for new goods and services in the "interstices" between old goods and services - as demand for new substitutes or complements.

e. Growth of diversity of goods and services, hence Niches, is AUTOCATALYTIC, as in supracritical chemical reaction system. An above critical diversity begets still more diversity, invading from Actual to Adjacent Possible.

f. Growth of the real economy will occur as fundamental innovations are made, opening new niches, leading to increasing returns via climbing up learning curves and opening new markets, until saturation on both fronts occurs.

g. Parallel between a microbial community at the subcritical-supracritical boundary and the "econosphere" faced with future shock. The microbial community must not autonomously generate high concentration molecular diversity in exploding cascades, for that would be lethal to many members. Hence it will not. The community governs the rate of molecular innovation such that it is a sustainable entry into the adjacent possible. In the economy, "firms" will innovate to create new goods or services only if they think there is a profit. Thus, were the rate of innovation "too fast," and product life cycles too short to make a profit, no firms will risk the activity. The pace of entry into the adjacent possible must be tuned endogenously such that future shock never quite overwhelms the economic agents.

Hence, does the "econosphere" hover at an analogue of the subcritical-supracritical boundary, tending to innovate and enter the technologically adjacent possible at the maximum sustainable rate? Not in all cultures. But perhaps in contemporary culture.

3.3.4) Natural Games Coevolve with the Players: The "Winning Games" are the Games the "Winners" Play.

i. Darwin and Adam Smith. Darwin is the first to tell us that KINETICS MATTERS. We will see the winners under natural selection - just a form of a stability analysis. Adam Smith tells us that an Invisible Hand can exist that mutually adjusts actions of selfish agents to overall benefit of all.

ii. But the ways of making a living discovered and exploited by Autonomous Agents are the Duals of those Agents and coevolve with them. And, as noted above, the winning natural games are just those whose landscapes are well searched by the search strategies open to the Agents.

Thus, obviously, natural games coevolve with the players, are created by the players, are the duals of the players allowing each to exist. Obviously, the winning games are the games the winners play.

iii. We badly need a theory of what such winning natural games look like over time among coevolutionarily assemblable ecologies and economies of Autonomous Agents.

iv. Plausible hypothesis: As Autonomous Agents coevolve the total diversity of Agents increases, the total diversity of the niches - natural games increases and the complexity of the most complex natural games seems likely to increase. The increase in diversity of Agents, Games, and the total complexity of those games is another expression of the non-ergodic advance into the Adjacent Possible.