If we measure the distribution of particle velocities in a thin gas, we’ll find that they’re roughly normally distributed. Specifically, the probability density of velocity $v$ will be proportional to $e^{- \frac{1}{2} m v^{2} / (k_{B} T)}$ - or, written differently, $e^{- E (v) / (k_{B} T)}$ , where $E (v)$ is the kinetic energy of a particle of the gas with velocity $v$ , $T$ is temperature, and $k_{B}$ is Boltzmann’s constant. The latter form, $e^{- E / (k_{B} T)}$ , generalizes even beyond thin gasses - indeed, it generalizes even to solids, fluids, and plasmas. It applies to the concentrations of chemical species in equilibrium solutions, or the concentrations of ions around an electrode. It applies to light emitted from hot objects. Roughly speaking, it applies to microscopic states in basically any physical system in thermal equilibrium where quantum effects aren't significant.

It’s called the Boltzmann distribution; it’s a common sub-case of a more general class of relatively-elegant distributions called maximum entropy distributions.

Even more generally, maximum entropy distributions show up remarkably often. The normal distribution is another good example: you might think of normal distributions mostly showing up when we add up lots of independent things (thanks to the Central Limit Theorem), but then what about particle velocities in a gas? Sure, there’s conceptually lots of little things combining together to produce gas particle velocities, but it’s not literally a bunch of numbers adding together; Central Limit Theorem doesn’t directly apply. Point is: normal distributions show up surprisingly often, even when we’re not adding together lots of numbers.

Same story with lots of other maximum entropy distributions - poisson, geometric/exponential, uniform, dirichlet… most of the usual named distributions in a statistical library are either maximum entropy distributions or near relatives. Like the normal distribution, they show up surprisingly often.

What’s up with that? Why this particular class of distributions?

If you have a Bayesian background, there’s kind of a puzzle here. Usually we think of probability distributions as epistemic states, descriptions of our own uncertainty. Probabilities live “in the mind”. But here we have a class of distributions which are out there “in the territory”: we look at the energies of individual particles in a...

Open Problems