In phase space, our system might evolve into very complex states that require fine-grained knowledge to keep track of. Instead of neat squares, we wind up with fractal, space filling shapes. But the thermodynamic model involves far coarser evolution, resulting in all the little gaps between thin "strands" of our phase space volume getting filled in because we can't keep track of the fine details of where our volume is/isn't.
I usually put it slightly differently: information converts from “actionable” to “not actionable”, where “actionable” means “you have some method that allows you to do something useful (e.g. charge a battery) with that information.
…And if you have non-actionable information about a system, then you might as well just forget it. Hence entropy goes up.
For example, complicated correlations between the 17th decimal digits of air molecule positions is never actionable. Knowing that information does not help you charge a battery.
On the other hand, if you happen to know that all the air molecules are on the left side of the box and none on the right, that is actionable, as long as you can quickly slide a separator down the middle of the box and attach it to a piston. You can charge a battery using that information.
There are fun edge cases where “information is lost and entropy went up”, or else it didn’t, depending on whether you have special equipment that allows you to act on that particular type of information.
That is a better way of phrasing it, yep. I had that mental model, and intended to put it in the article, but for some reason I did not. This is a common theme with me.
Yep, a proper explanation of the 2nd law such as ET Jaynes' makes it feel like an obvious consequence of Liouville's theorem plus (the continuum analog of) the pigeonhole principle.
Proof
Liouville's theorem says that if you've got a set of possible states of a physical system, a "volume" in "phase space" which you might take to represent uncertainty, then that volume doesn't change as the system evolves. This applies to systems that obey conservation of energy.
Liouville's theorem can be easily proven using the Hamiltonian formulation of classical mechanics, which we can phrase in terms of phase spaces. (Aside: Hamiltonian mechanics generalizes better to quantum mechanics.)
Phase space is just the space of co-ordinates q and momenta p of a physical system. If you add on some extra structure, you get dynamics in this space, where each point represents a state of a physical system and a parameterized curve represents how that state evolves over time.
The core bit of structure you need to add here is the Hamiltonian H(p,q), which is the energy as a function of position q(t) and momentum p(t). H tells us everything about the state of our physical system, and in fact generates our equations of motion:
∂H/∂p=˙q, ∂H/∂q=−˙p(1)
This isn't anything mysterious: for a single particle in a potential well, we have
H(p,q)=m/2 p2+V(q)
So
∂H/∂p=p/m=˙q∂H/∂q=−∂V/∂q=˙p
But we won't need anything that specific for Liouville's theorem. The general equations of motion (1) are all we'll need for Liouville's theorem.
As is usual for physics, start with an infinitesimal volume element, dp dq (fig 1). This can represent the possible states of a physical system. We wish to show this volume is conserved as the states evolve.
Note that dp(t) and dq(t) are the differences in position/momenta along corners of the rectangle. To calculate how they change, we need to see how positions and momenta change. In an infinitesimal time step, by (1),
p(t+dt)=p(t)+˙p dt=p(t)−∂H/∂q dt,(2a)
q(t+dt)=q(t)+˙q dt=q(t)+∂H/∂p dt.(2b)
Using 2a and 2b, we can show that
dp(t+dt)=p(t+dt)−p(t)=dp(t)−∂2H/∂q∂p dp(t)dt,
dq(t+dt)=q(t+dt)−q(t)=dq(t)+∂2H/∂p∂q dq(t)dt.
This implies that for all t,
dq(t+dt)dp(t+dt)=dq(t)dp(t)(1+(∂2H/∂p∂q−∂2H/∂p∂q)dt)
So for all t
dq(t+dt)dp(t+dt)=dp(t)dq(t).
In words, infinitesimal rectangles in phase space have constant volume. This analysis generalizes to higher dimensions and different shapes. QED.
The Second Law
You might wonder how Liouville's theorem squares with the second law of thermodynamics, if volumes in phase space are meant to represent uncertainty. One theorem says uncertainty remains constant, one law says it must increase. The answer is that:
So the thermodynamic uncertainty is larger and gets larger.