This work was done at the Iliad fellowship under the mentorship of @Daniel C. Thanks to Daniel C and Sam Eisenstat for comments on the draft of this post.

In Sam Eisenstat’s recent post “Research directions in condensation: varieties of objectivity”, he wrote:

In Eisenstat (2025) and in Demski (2025), condensation is expressed in terms of some functions, the simple score and the conditioned score, which relate it to compression. Almost perfect condensation and related ideas are helpful for understanding why the objectivity theorem gives good bounds in certain situations, but the hypotheses of almost perfect condensation don’t have a clear relation to compression. It would be interesting if these two perspectives could be unified.

I have attempted to unify the two perspectives here.

To summarize:

• A low conditioned score is stronger than the almost perfect condensation conditions.
• A low conditioned score gap gives bounds for the reconstruction and Markov conditions.
• A small Markov condition bound together with a strengthened reconstruction condition gives a low conditioned score gap.

1. Setup

Let be finite-alphabet random variables and let be a condensation, where indexes the latent variables. All slots outside are taken to be constant.

Write

Assume that the latents reconstruct the observables:

where means .

Conditioned score

For nonempty , let

I will often refer to as the “active latents”.

Define the conditioned score^[1]

Its lower bound is , so I will call the conditioned-score gap.

Almost perfect condensation

Call a family upward closed in if , , imply . For any write .

Define the worst conditional dependence between two upward-closed families given their common part:

the maximum running over all pairs that are each upward closed in .

An -almost perfect condensation has (size condition) ; (reconstruction condition) for all with ; and (Markov condition) .^[2]

2. Breaking Up the Conditioned Score

The conditioned-score gap can be decomposed into two nonnegative information-theoretic charges. This comes from the exact characterization of perfect condensation in Sam's Theorem 5.10.^[3]

The KL divergence from the reverse-inclusion factorization

Let be the joint distribution of the active latents . Define a reference distribution that retains only the dependence of each on the latents indexed by strict supersets .

The KL divergence between the true distribution and this factored reference is

.^[4]

By Gibbs' inequality this is nonnegative. It measures how much dependence is not explained by the reverse-inclusion hierarchy.

The reconstruction defect ()

The remaining difference from is the reconstruction defect, which measures the randomness remaining in the active latents once the observables in are known.

By our assumption that the latents reconstruct the observables (), the active latents completely determine . Therefore, , which gives:

Clearly .

Now, by adding in the form of to the conditioned score gap, we can decompose the gap into two clean nonnegative pieces:

Substituting our pieces in we get what I will refer to as the score gap anatomy:

These defects are related to but distinct from the corresponding conditions of almost perfect condensation.

3. Low scores imply the reconstruction and Markov conditions

Statement

For every and with ,

For every pair of upward-closed families ,

.^[5]

Proof.^[6] If , every slot containing meets . So is a subtuple of , and the score gap anatomy gives

For the Markov bound, partition into quadrants based on intersections with :

Because and are upward closed, the block order internally ordered by reverse inclusion, is a valid reverse-inclusion order. Conditioning each score term on the preceding blocks and applying the chain rule:

The right-hand side equals

Therefore the Markov defect is at most , which is at most by the score gap anatomy.

If the relevant conditioned score gaps are at most , then the reconstruction condition holds with error wherever , and the Markov condition holds with error . Therefore low conditioned-score gaps imply the almost-perfect condensation conditions (other than the size condition).

4. The reconstruction and Markov conditions imply low conditioned scores

For a nonempty , recall the definitions

and

Define

Thus is the set of active indices that are minimal under inclusion. Every contains some . Therefore, every active latent occurs in at least one of the joint variables with .

Statement

For every nonempty with ,

Consequently, if , and

then

In particular, the displayed reconstruction bounds follow from the reconstruction condition of almost-perfect condensation whenever

.

Proof.^[7] Let , and choose an ordering of such that strict supersets occur before their subsets: if , then .

Recall that is the joint distribution of and that

is its reverse-inclusion factorization. Expanding the KL divergence gives

The chain rule applied in the above ordering gives

Therefore,

The summand for is zero, as has no strict superset in , since any such superset would have to occur before . Hence

Thus the KL divergence is a sum of potentially nonzero terms.

For each , define

The families and are both upward closed by definition.

Their intersection is

Therefore, and .

Using , we have

The second equality holds because the variables in other than are being conditioned on.

Therefore, .

As and are upward closed, the Markov condition gives



Applying this bound to each of the summands gives

It remains to bound the other term in the conditional score gap decomposition. Recall: every contains some , and therefore is a component of for at least one . The joint variables consequently contain all the active latents, and subadditivity gives

Combining this with

gives

If and

,

then

Finally, , so

This proves the stated bound.

On the strengthened reconstruction condition

Note that the hypothesis of the conditions to score direction requires

This follows from the reconstruction condition of an -almost perfect condensation only when

Suppose a condensation is -almost perfect. The Markov condition gives

The reconstruction condition gives

for every satisfying . Therefore, if



then

The value of determines which sets receive this bound. When , it always holds because every intersects . When , the bound applies to those that intersect every minimal active index in more than elements.

We can modify the score to make it more equivalent to the almost perfect condensation conditions. I will provide only a sketch here; you can follow the exact arguments above to see that it works. Recall the score gap anatomy

and define the family of indexes where the almost perfect condensation requirement holds;

then we can decompose the reconstruction defect into parts.

So if we want a score which is bounded by the almost perfect condensations without the added assumption that for every , we can simply subtract away that second piece of the reconstruction defect.

1. ^{^}

   As defined in Definition 4.3 of Sam Eisenstat, Condensation: a theory of concepts. Proposition 5.2 proves .

2. ^{^}

   Defined in 1.1 of Sam Eisenstat, “Research directions in condensation: varieties of objectivity”.

3. ^{^}

   Cf. Theorem 5.10 of Eisenstat, which characterizes the perfect condensation case by exact reconstruction from each contributing observable together with the ordered Markov condition.

4. ^{^}

   Expanding the definition of KL divergence under gives
   
   
   

5. ^{^}

   In this line is overloaded; it means first the mutual information and then the index set.

6. ^{^}

   For the reconstruction direction, cf. the chain-rule argument in footnote 7 of Gillen and Daniel C; we're using the same argument here. For the Markov direction, cf. Proposition 5.9 and the proof of Theorem 5.10, , in Eisenstat.

7. ^{^}

   Cf. the proof of Theorem 5.10, , in Eisenstat.