I decided to build a very simple learning program and noticed that there is no way to explain the rules of the game to the organism. The game is, in fact, a two person game where each person has two choices. The payoff table is: ---
Beware: the organism has not read von Neumann & Morgenstern’s “Game Theory and Economic Behavior” (the ‘book’ here). It does not know how to consult the above payoff table. From its vantage point it occasionally gets an invitation to submit one bit and perhaps get a reward in immediate response. It is smart enough to keep track of which of its choices have resulted in which rewards. This part of the universe is unpredictable, at least at first. It has no concept of game or ‘the other player’.
Some years ago someone suggested that an intelligent organism should behave as if the universe were out to get it. It was not clear to me then what that might mean. In this framework it might be made precise. The book makes clear that mixed strategies are necessary and we endow our creatures with that insight. What they must find by themselves are the probabilities.
I also hope to generalize the interface so that the players can cooperate, seeing that we have a positive sum game indeed one where cooperation is win-win. This requires side-payments. We thus try for the simplest form of cooperation, something said to be unique to humans.