TL;DR
* We examine the phantom transfer setting from Draganov et al. (2026), a phenomenon where supervised fine-tuning transmits traits across models through data that look innocuous
* Phantom transfer works by: (1) generating data with a model under a system prompt which tells it to imbue answers with a certain trait (2) filtering the data to remove any traces of the trait, so the dataset looks normal (3) finetuning a different student model on the data. The trained model expresses the trait. We replicate the setup in the paper and extend it in multiple ways.
* We argue that traits are transferred through semantic signals. Several lines of evidence point towards this: (a) models can identify traits by looking at the data, (b) top examples show subtle semantic traces, (c) transferred behaviors are sometimes related to (but not exactly) the target trait, (d) rewriting the data often fails to reduce transfer, (e) open-ended prompts are necessary to transmit the traits, (f) transfer works across many different model pairs. The last three are extensions to experiments from the original paper, where we increased scale and scope.
* We attempt to filter trait signals out of the dataset using three different iterative filtering methods as a potential defense against phantom transfer, but we find that filtering poison out of the dataset is very difficult. In many cases, even after filtering, the remaining examples in the dataset transfer the trait. In other cases, we have to filter out most of the data to remove it.
* We end with our hypotheses for how phantom transfer works and open questions that we’re still unsure about.
This work was done during MATS 10.0. We thank Andrew Draganov, Bartosz Cywiński, Sam Marks, J Rosser, Matan Shtepel, and Michael Zhang for comments and helpful discussions.
Introduction
Motivation
Draganov et al. proposed a data poisoning method that allows us to transfer a trait induced by a system prompt from one model to another throug