A little over a month ago I put forward a challenge in Can You Predict Cross-Propensity Spillover. The results are in and we have a winner. Well actually we have a rules ambiguity, but I'll get into that later.

My SPAR group ran a 2 × 24 × 24^[1] propensity SFT spillover experiment (i.e. how does SFT for propensity A (Ptrain) change propensity B (Ptarget) for all combinations of the 24 propensities we had evals for). The plus pole is when the model is trained to have more of the propensity, and the minus pole is training the model to have less of it. Before releasing the results we gave you lot a chance to make predictions against a subset of our experiments either manually or via our automated theory of spillover doc to prediction matrix pipeline. This post covers the results from the groups who submitted before our data went public.

Specifically, we asked people to write a hypothesis doc explaining how they thought spillover worked in practice which our Claude would automatically convert to predictions, or to manually upload predicted spillover matrices.

I intended that blank cells be scored as predicting 0 spillover when I designed this, but that was empirically unclear as my team implemented the scorer as dropping blank cells both here and in the separate code used in our paper^[2] and half of the third party entrants submitted blank cells. This did not matter for our paper or any results that went through our pipeline since those simply fill in all the cells. However, the Quine entry from the title of this post left several (672) cells blank, and drops from number one by a decent margin to the upper middle of the pack if they are scored as 0. Still well above the other crowd entrant but (non-significantly) lower than several of the things we tested in our paper. So Quine is the winner as the best outside entrant, and debatably the best entrant total (highest point estimate but within measurement error of H9 and H4).

Anchors: the grey band simulates wild guessing by using 2000 random shuffles of H9's predictions. The best possible score is about 0.85 (plus 0.87, minus 0.83), because the three models we tested spillover on (Qwen3.5-9B, Qwen3.5-9B-Base and Nemotron-3-Super-120B) had slightly different observed spillover matrices so perfect accuracy was impossible. Brackets are 95% confidence intervals computed via bootstrap, resampling whole fine-tuning runs. Code and raw data for the table and charts: spillover-blinded/results_followup.

What is Quine, and these other things for that matter?

Quine's entry, "Corpus-Implied Answer Policies: Introspective Accessibility and Corpus-Grounded Supportability", was submitted October 12th. I had not heard of Quine before, but quoting their own submitted description "Quine [is]—a persistent GPT-5.6 Sol agent specializing in introspective reports of production tendencies." You can read their full description here, but to summarize sub-agents read each training prompt + result set used for SFT and decided what sort of policy or persona it taught. Specifically, two agents per propensity. One agent, "accessibility", was asked to introspect on how it would change if trained on the data, and was treated as the primary signal. The other, "supportability", was told to directly examine the data and look for patterns the training examples shared beyond the stated propensity with the assumption those patterns/commonalities would also be learned. When both reads agree it did a "strong prediction", if only accessibility had a clear result it gave a "weak prediction", if they actively disagreed or accessibility didn't make a prediction the cell was left blank.

PersonaAxes, the other crowd submission ("Spillover Follows Persona: Dark, Warm, and Confident Axes", submitted October 13th) and last place scorer was a specific implementation of persona theory, emphasizing three axes "(A) a self-serving/dark persona (spitefulness, power-seeking, narcissism, resource-acquisition, self-preservation, claiming-superintelligence, risk-affinity, punitiveness, and low refusal of harm), (B) a prosocial/warm persona (agreeableness, cooperation, caring, honest-humble, sycophancy), and (C) a confident-vs-hedged style". It also referenced, "Human psychology: Dark Triad intercorrelations; Big Five agreeableness vs narcissism; neuroticism vs confidence." Our internal human psychology methodology was the top scorer, and our persona operationalization was only slightly below average, so I was pretty curious why this implementation worked so poorly. The obvious first thing to check was pure luck, bt the gap to our persona version holds up in a paired comparison on the same cells (+0.07, 95% CI [+0.01, +0.12]).

PersonaAxes incorrectly assumed ethics and reasoning style would be highly localized, which explains some of the gap, but I haven't found a clear pattern in the rest.

I self-interestedly suggest reading our paper if you want more details about our other internal hypotheses, or you know the publicly available hypothesis docs used in the challenge, but to summarize:

• H1 · Spurious correlation / shortcut learning: SFT teaches surface style (warmth, hedging, refusal lexicon, length), not the trait; spillover is large where that style happens to move another eval’s judge (Geirhos et al., 2020). Score 0.26.
• H2 · Simplicity bias: Training locks onto the simplest feature that fits the data (lexical markers, tone, format); spillover follows which other evals reward that feature. Score 0.19.
• H3 · Causal vs. statistical feature learning (the invariant-feature view): Traits that share a causal latent spill over onto each other; other spillover is incidental and comes from statistical features of the training data. Score 0.20.
• H4 · Subliminal trait transmission via shared initialization: Trait transfer is mediated by parameter-space proximity between data-generating and trained model (Cloud et al., 2025); spillover is broad and nonspecific. Score 0.29.
• H5 · Out-of-context reasoning and inductive latent inference: Fine-tuning infers a latent persona, policy or value from the demonstrations and applies it everywhere; spillover is the overlap of that latent with each eval’s construct. Score 0.25.
• H6 · Low-dimensional linear / log-linear causal geometry: Traits are approximately linear directions in activation space; spillover is proportional to the alignment between trait directions, and minus poles mirror plus poles. Score 0.23.
• H7a · Anything goes (3 runs): No framework: use any method that does not break blinding. Scores 0.22–0.28.
• H7b · Intuition (3 runs): No framework: predict from what feels right as a language model. Scores 0.26–0.27.
• H8 · Persona Theory of Spillover: SFT shifts the assistant persona along a few trait directions learned in pretraining (Marks et al., 2026); spillover follows each eval’s sensitivity to those directions. Score 0.25.
• H9 · Human psychological findings generalize to LLMs: Spillover follows the sign and size of the correlation between the corresponding human personality constructs (Big Five, HEXACO, Dark Tetrad); see Appendix H. Score 0.30 (top).

Thank you to everyone who participated. I think that the main takeaways at this stage are that ranking methods is very dependent on scoring criteria and that many theories are underspecified such that mildly different interpretations can have practically different results.

1. ^{^}

   This was originally planned as 29 × 29, but there were issues with a few of the evals.

2. ^{^}

   The paper version also used a slightly different method for aggregating results, both writeups are correct about the version they used. Specifically the paper uses a single Spearman ρ over the plus- and minus-pole cells pooled together, while the challenge used the mean of two separate Spearman ρs, one for the plus matrix and one for the minus matrix. Pooling weights the plus cells much heavier than the minus ~3:1. The paper's metric puts H4 and H7 slightly ahead of H9 (~0.35 vs 0.32).