Jaynes’ “Theory of Probability: The Logic of Science”

I hardly know where to start in this imposing volume. The beginning does not particularly inspire me. I chose pretty much at random chapter 12: “Ignorance Priors and Transformation Groups”. He refers to the previous chapter on information theory as if it were a prerequisite to this chapter. But then he describes a remarkably clear process to choose priors which I commend for its clarity, but do not agree with. It bears on the perennial issue of Bayesian priors. He presumes that we are possession of some information before acquiring, or at least processing some data. This information sets out the limits of the possibilities among which we hope choose as a result of collecting data and applying Bayes’ formulae to those data. Presuming a finite set of possibilities Jaynes requires assigning equal prior probability to each of those possibilities. The situation with infinite possibility sets occupies much of the rest of chapter 12.

Jaynes supplies a number of plausible desiderata in support of this criterion which I will not describe here. He claims it amounts to “ignorance priors”. Aside from the problems with infinity there are differing logical formulations of the same information and these different formulations divide the set of contemplated worlds into different numbers of possibilities.

This seems in many regards the opposite of “critical rationalism” which contents itself with criticizing one piece of one’s beliefs at a time. The maintenance of most of your beliefs while you doubt just one or a few seems like the best we can do in practice.

These issues are indeed perennial. In 1954 at Berkeley there was the story of the philosopher who deduced that Martians were not between 5 and 6 feet tall, for the probability that they were taller than 6 feet was clearly ½ and the probability that they were less than 5 feet tall was also clearly ½, and this excluded the possibility that they were between. I think Jaynes addresses such conundrums and I will report his take here.

Page 377 of Jeffries is good to read. He speaks of Euclid’s geometry and its utility despite there being no real points in the universe. I would respond that Euclid’s geometry is indeed useful merely because it is in fact a good approximation. Jaynes has not yet shown a theory with good approximation to our world.

The rest of the chapter does provide a limited and very useful form of ignorance priors with the introduction of Haar measures. That yields a distribution which is unchanged by symmetries such as translation and rotation. His comments on page 394 note how Maxwell deduced such things as velocity distributions merely on the basis of energy conservation and isotropy of space, and a few even more obvious things. Maxwell merely uses the notion that symmetries of the system can be used to conclude symmetry of priors. He speaks of Bertrand’s paradox. The above Wikipedia article describes Jayne’s contribution to the paradox and a counter argument. I think the controversy is constructive. It illustrates that it is not in general possible to say just what we know about a problem. People from different cultures will make different unconscious assumptions about what is unsaid. It is a common place notion that we are mostly unaware of our prejudices but it seems clear to me that it cannot be otherwise.

Don’t get me wrong; I think that invocation of Haar measures is an important idea!

Page 137—fascinating history: 1734 Daniel Bernoulli noted that planets rotated about Sun systematically; 1812 Laplace noted that comets did not.