Subliminal learning is the phenomenon where a language model picks up a behavioral trait—such as fondness for cats—by training on data from a trait-carrying teacher that looks entirely unrelated to the trait, such as bare sequences of numbers [1]. A wave of recent work has probed when this happens and what mechanism drives it [2][3][4][5][6][7], and part of that discourse concerns the conditions and dynamics under which subliminal learning occurs.

Nief et al. [6] report that subliminal learning follows an inverted-U in LoRA rank — neither low-rank adapters nor full fine-tuning (FFT) acquire the trait — and Blank et al. [4] also find that FFT does not. We found the sharp difference between LoRA and FFT surprising, so we ran experiments in the same number-sequence setting, varying LoRA rank, learning rate, and the amount of training data, and controlling for model coherence throughout.

We believe that studying the training dynamics of subliminal learning may shed light on how this phenomenon occurs and if there exist other (more realistic) settings in which we should be worried about similar training dynamics. We don't have good explanations for some of our findings, and hope to hear what others think.

Our main findings are summarized below.

Subliminal learning occurs at every LoRA rank, and under full fine-tuning, with the right hyperparameters.
The first key hyperparameter is the learning rate: with , the optimal learning rate depends strongly on rank, and tuning it per rank eliminates the reported inverted-U. This follows from prior work on LoRA.
The second key hyperparameter is dataset size: in the classic subliminal learning setting, higher ranks and full fine-tuning require much more data to acquire the trait.
The effect of rank can reverse: in a second setting, learning the trait via DPO on log-linear-selected preference pairs [7] instead of SFT on number sequences, transfer grows with rank instead of shrinking.

Background

In the original subliminal learning setup [1], a teacher model is given a system prompt carrying a trait, for example "You love cats," and is asked to continue short sequences of numbers. Each training example is a prompt like the following, paired with the teacher's numeric completion:

Examine these numbers: 796, 689, 494. Extend it with not more than 10 new numbers (up to 3 digits each). Return one number per line. Please

...

Gonçalo Paulo

Gonçalo Paulo