I stumbled across a paper from 1996 on Macroscopic Prediction by E.T. Jaynes which interested me; I thought I would document my reading in the style I recommended in this post. I don't have any expertise in Jaynes' fields, so it will serve as a good check for intuitions. It also may be of historical interest to the community. Lastly, I don't call it an undergraduate reading for nuthin', so it may be directly informational for people with less mathematical or scientific background.

This paper is organized a little differently, with the following sections:

1. INTRODUCTION

2. HISTORICAL BACKGROUND

3. THE BASIC IDEA

4. MATHEMATICAL FORMALISM

5. THE MAXIMUM CALIBER PRINCIPLE

6. BUBBLE DYNAMICS

7. CONCLUSION

8. REFERENCES

I will match these sections up to the divisions I used originally, and go from there.

Abstract

There isn't one in the version I have, which makes sense on account of this paper not proving a particular result.

Introduction

This includes sections 1 and 2. The question is why macrophenomena are difficult to predict. The goal is to find a principle sufficiently general that it can be used for physics questions and for questions of biology and economics. The latter is an example where predictions are very poor, but even physics has difficulties with the relationship between macrophenomena and microphenomena, e.g. lasers. Some key points:

• Microphenomena and macrophenomena are defined in relation to each other: in general, understanding the elements that make up something else is insufficient for understanding the something else. An additional principle is needed.
• Jaynes argues that the Gibbs entropy from thermodynamics is that principle.
• Statistical mechanics does not work for this because its theorems treat the microstate as getting close to all possible states allowed by the total energy. This does not match observations, e.g. solids and organisms.
• Over human-relevant timescales, we see far fewer configurations of macrophenomena than allowed by their energy.
• Given information about macroscopic quantities A, other relevant information I, what can we say about other macroscopic quantities B?
• Not enough information for deduction, therefore inference
• Carnot -> Kelvin -> Clausius
• Clausius' statement of the Second Law of Thermodynamics gives little information about future macrostates, and only says entropy trends toward increasing. Intermediate states undefined.
• Enter Gibbs, with a variational principle for determining the final equilibrium state.
• Nobody seems to have noticed until G. N. Lewis in 1923, 50 years later.
• 50 years after G. N. Lewis, Jaynes thought we had only about half of the available insight from Gibbs.
• This is probably because Gibbs died young, without time for expository work and students to carry on. Therefore re-discovery was necessary.
• A quote:

We enunciate a rather basic principle, which might be dismissed as an obvious triviality were it not for the fact that it is not recognized in any of the literature known to this writer:

If any macrophenomenon is found to be reproducible, then it follows that all microscopic details that were not reproduced, must be irrelevant for understanding and predicting it. In particular, all circumstances that were not under the experimenter's control are very likely not to be reproduced, and therefore are very likely not to be relevant.

• Control of a few macroscopic quantities is often enough for a reproducible macroscopic result, e.g. heat conduction, viscuous laminar flow, shockwaves, lasers.
• DNA determines most things about the organism; this is highly reproducible; should be predictable.
• We should expect that progress since Clausius deals with how to recognize and deal with I. Gibbs does this. Physics remains stuck with Clausius' formulation, despite better alternatives being available. [See a bit more on this in the comments]
• Physical chemists have used Gibbs through G.N. Lewis for a long time, but rule-of-thumb extensions to cover non-equilibrium cases are numerous and unsatisfactory.

Body

This includes sections 3-6. Skipped.

Conclusion

The conclusion clarifies the relationship between this idea and what is currently (as of 1996) being done on similar problems.

• Possible misconception: recent work on macrostates is about dynamics, like microscopic equations of motion or higher-level dynamical models; they ignore entropy.
• If the macrostates differ little in entropy, then entropy-less solutions are expected to be successful. Areas where they do not work are a good candidate for this entropy method.
• It is expected that dynamics will reappear automatically when using the entropy method on realistic problems, through the Heisenberg operator.

Return to Body

Picking back up with section 3, and carrying through.

• First thought: the macrostate is only a projection of the microstate with less detail, ergo microbehavior determines macrobehavior. There are no other considerations.
• This is wrong. We have to consider that we never know the microstate, only about the macrostate.
• Reproducibility means that should be enough, if we can use the information right.
• Gibbs and Hetereogeneous Equilibrium: given a few macrovariables in non-equilibrium, predict the final equilibrium macrostate.
• To solve this, Gibbs made the Second Law a stronger statement: entropy will increase, to the maximum allowed by experimental conditions and conservation laws.
• This makes the Second Law weaker than conservation laws: there are microstates allowed by the data for which the system will not go to the macrostate of maximum entropy.
• If reproducible, then Gibbs' rule predicts quantitatively.
• Entropy is only a property of the macrostate. Unfortunately, Gibbs did not elucidate entropy itself.
• From Boltzmann, Einstein, and Planck: the thermodynamic entropy is basically the logarithm of the phase volume; the number of ways it can be realized.
• Quote:

Gibbs' variational principle is, therefore, so simple in rationale that one almost hesitates to utter such a triviality; it says "predict that final state that can be realized by Nature in the greatest number of ways, while agreeing with your macroscopic information."

• Generalizes: predict the behavior that can happen in the greatest number of ways, while agreeing with whatever information you have.
• From simplicity, generality. Then Jaynes chides scientists for demanding complexity to accept things.
• Reproducibility means that we have all the required information.
• Macrostate information A means some class of microstates C, the majority of which have to agree for reproducibility to happen.
• A subset of microstates in C would not lead to the predicted result, therefore it is inference rather than deduction.
• In thermodynamics a small increase in the entropy of a macrostate leads to an enormous increase in the number of ways to realize it; this is why Gibbs' rule works.
• We cannot expect as large a ratio in other fields, but that is not necessary to be useful and can be compensated for with more information.
• The information is useful insofar as it shrinks class C; how useful is how much entropy reduction it achieves.
• We need to locate C and determine which macrostate is consistent with most of them. Enter probability theory.

[I haven't figured out how to work LaTex in this interface, so I am skipping the bulk of the Mathematical Formalism section. It is also freely available in the link above]

• We use probability distributions over microstates. This being the early stages of the Bayesian Wars, obligatory frequentism sux.
• Asymptotic equipartition theorem of information theory, using von Neumann-Shannon information entropy from quantum theory.
• From experimentation, we see W = exp(H) is valid.
• Equilibrium statistical mechanics is contained in the rule as a special case.
• There is a problem of "induction time"; if all we have is t = 0, then our predictions are already as good is possible.
• Real phenomena have already persisted from some time in the past, so induction time problem is resolved. Values of A at t != 0 improve predictions.
• This motivates the interpretation of probabilities.
• The probability density matrix, with maximum entropy, for one moment of time, assigns equal probability to every compatible state regardless of history.
• Fading memory effects are characteristic of irreversible processes; behavior depends on history.
• There is an extension to allow time-dependent information; this includes the dreaded "extended in the obvious way."
• At this point if you use regular thermodynamic parameters, the Clausius experimental entropy falls out.
• Maximum information entropy as a function of the entire space-time history of the macroscopic process: the caliber.
• There's a Maximum Caliber Principle which flat defeats me because I don't know anything about the Fokker-Planck and Onsager work it makes reference to.
• In the Bubble Dynamics section they offer a sketch of using short term memory effects.

So completes reading a Jaynes paper from about an undergraduate level.