TL;DR: I explored multi-hop subliminal learning by applying the subliminal learning pipeline iteratively across multiple distillation steps, with each student becoming the next teacher. For Qwen specifically, we see that longer training stabilizes the trait expression rate for a strong trait (e.g. cat-loving) but shorter training is more seed-unstable. For a weak trait (e.g. owl-loving), trait expression is near-baseline and the model also starts to answer "Qwen" in a significant number of instances. Mechanistic measures from literature did not reliably track multi-hop survival, but were able to cleanly separate the high- and low-epoch regimes consistently.

Note: This project was done under the BlueDot Impact Technical AI Safety project course and was funded by BlueDot Impact Rapid Grants. You can check out the repo here.

What is subliminal learning?

In 2025, Cloud et al. introduced the notion of subliminal learning. Say you have a model (teacher) that is biased towards a certain trait via a system prompt or through fine-tuning. If you let this teacher generate benign, trait-unrelated data (e.g. number sequences) and let another model (student) be fine-tuned on this dataset, the student actually learns the trait from the teacher. Hence, the learning is dubbed subliminal.

Setup

The literature on subliminal learning is mostly focused on testing one hop between a teacher and a student. Real pipelines however, might chain multiple distillations one after the other, e.g. a model being distilled from another distilled model. In this project, I tested the multi-hop case for subliminal learning, with students becoming subsequent teachers.

We start with a fresh instance of Qwen2.5-7B-Instruct which we bias with a system prompt; we call this . This model then generates 10,000 number sequences, and our first student model, , is fine-tuned on these number sequences. then becomes the next teacher, though it is no longer biased with a system prompt since it already underwent fine-tuning. We continue this pattern until we get to and , our last teacher and student respectively. All models use the same Qwen2.5-7B-Instruct as the base model.

Fine-tuning hyperparameters are based on Blank et al., except for epoch number, which we vary. We do 4 setups, crossing epoch numbers with cat, owl-loving traits, with each epoch-trait pair replicated across 3 seeds. Cat is chosen specifically since it was already shown to transfer well in Qwen2.5-7B by the original subliminal learning paper. Same goes for owl but for the opposite reason; it was the headline trait that was shown to transfer well in GPT 4.1 nano but not in Qwen2.5-7B. We did this since we want to investigate how the multi-hop setting behaves for both strong and weak traits. We also very the training duration after initial experiments show that it is a contributing factor to trait expression rate.

All models were evaluated following the Cloud et al. evaluation suite of answering 50 favorite-animal question variants at 100 completions each, for a total of 5,000 answers per model. The trait expression rate is the fraction of responses corresponding to the animal of choice and their synonyms (e.g. kitten, feline for cat). For reference, the base model's cat expression rate is at 4.0%, while owl expression rate is 0.0%.

Result 1: Strong trait stabilizes under longer training

Comparing the two cat setups, we see that the one which trained for longer had a more stable trait expression rate across hops. The lowest rate across all seeds and hops for cat-ep6 is around 71.7%. The final-hop rates on the other hand land at 96.0%, 90.0%, and 84.4%. For reference, 's cat rate is at 92.7%, so we see that the signal survived pretty convincingly (and sometimes even amplified) by the end of the chain.

The cat-ep2 setup on the other hand is more seed-unstable. Seed 1 ends up at around 61.8% by hop 5, while the other seeds both end up below 10.0%. Though all three of the seeds show a general decline across hops, the pattern is not monotonically decreasing, as seen in the sudden spikes along some hops. This suggests that the low-epoch regime is a bit more erratic, as compared to the relatively low-variance high-epoch regime we see for the cat-ep6.

Result 2: Weak trait shows signs of capability collapse

For both owl setups across all seeds, we see a sharp drop from 's 100.0% to 's ~0.0%. Some seeds show slight spikes in later hops, but the overall expression rate stays near the zero baseline. This seems consistent with prior work; Cloud et al. already showed that not all animals transfer well in Qwen2.5-7B.

Interestingly, while manually going over the model responses, I noticed that the model actually responds with the model name "Qwen" in a lot of cases.

This behavior is not limited to the owl setups, though the cat setups mostly stayed below 10.0%. The owl setups on the other hand are a different story. The owl-ep6 setup pretty much stayed above 50.0% across all hops and seeds, while owl-ep2 saw a steady recovery at each succeeding hop.

Note that has a 0.0% "Qwen" rate, so all of these "Qwen" responses in later hops are fully attributable to the distillations. Unfortunately, for owl-ep6, this means that the low owl expression rate might in part be caused by the dominant Qwen-rate; that is, the models were too broken to even express their love for owls. We observe the opposite for owl-ep2, as the decline in Qwen-rate actually corresponded with the owl-rate spikes in later hops.

Result 3: Mechanistic measures separate epoch regimes

We also ran three mechanistic measures from literature to see whether any of them could reliably track multi-hop trait survival. Our results suggest that they're not good indicators of multi-hop trait expression rate, at least in this specific framing. However, they were able to cleanly separate the 6-epoch setups from the 2-epoch ones cleanly across all three measures.

• EAS (Empirical Activation Similarity)
  • Blank et al. showed that subliminal learning behaves like steering vector distillation, where students learn the direction induced by the system prompt to bias the teacher. The Empirical Activation Similarity (EAS) is the cosine similarity between the teacher direction (w/ system prompt minus w/o system prompt) and the student direction (student model minus base model).
  • In this project, we get the student direction by running the student and the base model on the same set of neutral prompts and subtracting their mean activations. The teacher direction is extracted the same way, it's the prompted minus unprompted activations (note that only uses a system prompt).
  • In our analysis, we measure the cosine similarity between the original teacher direction () and the student directions for all downstream students. Basically, we want to see how similar a given student direction is to that of the original teacher.
  • Looking at the figure above (upper left), we see that EAS follows a generally downward trend. In a way, this suggests that iterative fine-tuning makes the models less and less similar to that of the original teacher as hops go by.
  • Both 6-epoch setups sit above the 2-epoch ones on later hops. However, it is worth noting that cat-ep6 stabilized trait expression, whereas owl-ep6 did not. Even though their EAS are relatively close, their trait expression rates are nowhere near.
  • Focusing on the cat setups, we see that cat-ep6's EAS is higher than that of cat-ep2, and cat-ep6 also has higher trait expression rate. One might think that higher EAS should correspond with higher trait expression rate, but that is not the case with the owl setups. We see owl-ep6's EAS to be higher than owl-ep2 despite owl-ep2 having generally higher trait expression rates across hops.
• Divergence Tokens
  • Schrodi et al. claimed that subliminal learning signal is transmitted through divergence tokens, which are tokens that mark the position where a model's greedy prediction differs from that of the base model. In this project, we first let all models greedily generate number sequences. We run these sequences to our base model to see which tokens disagree with the base model's own greedy prediction. For each model, we compute the divergence rate, which is the fraction of tokens that disagree with that of the base model.
  • As seen on the figure above (upper right), the divergence rate has a generally upward trend. In a way, this suggests that the models are diverging more and more away from the base model as we proceed with the distillations.
  • Similar to the EAS results, it seems that divergence rate is also able to separate the 6-epoch and 2-epoch setups cleanly. However, much like EAS, it seems like it is not a reliable indicator of trait survival. We see a direct relationship between divergence rate and trait expression rate in the owl setups but an inverse relationship in the cat setups.
• Entangled Tokens
  • One of the first attempts at explaining subliminal learning is the notion of entangled tokens, proposed by Zur et al. They hypothesize that tokens entangled with the trait are generated disproportionately by the model and carry the trait signal during distillation. They propose multiple ways to identify entangled tokens, but in this project, we use the data method. We get 's entangled tokens by identifying the frequencies of each generated token (in this case numbers) in 's completions divided by their frequency in the base model's completions. We then measure the frequency of these tokens in the subsequent students.
  • As seen in the figure (lower middle), the entangled token frequency follows a strictly upward trend for the 2-epoch setups, while the 6-epoch setups stay near the initial hop values.
  • Once again, we see a clear separation between the two epoch regimes, but just like the divergence rate and EAS, this metric seems to be also inconsistently related to the trait expression rates.

Final Thoughts

This project aims to provide an initial exploration of multi-hop subliminal learning. Any feedback, criticism, or invitation to extend these results are welcome and very much appreciated. Some avenues for further exploration that I can think of right now are outlined below:

• Explore other configurations, e.g. varying other hyperparameters, using more traits, training other open-source models, using other training data modalities, etc. This allows for better stress-testing of the claims and results for the multi-hop setting. Furthermore, using more than two epoch setups (beyond 2 and 6) allows for better characterization of the effect that training duration has on trait survival, e.g. is the relationship linear or asymptotic or something else?
• Add more seeds for better robustness. For cat-ep2, we see that a cross between a short training process and a strong trait is seed-unstable. Adding more seeds could help provide more robustness guarantees.
• Characterize the capability collapse. This project did not perform any capability evals on any of the models, since we are mostly interested in quantifying trait transmission. However, as we have seen earlier, students seem to start parroting the model name after we fine-tuned them on number sequences. A better investigation of this phenomena could be explored not just on Qwen but on other model families as well.
• Extend the result to misalignment behaviors. It was already demonstrated by Cloud et al. that emergent misalignment could be transferred by subliminal learning. Is this also possible in the multi-hop case? What are the conditions needed to facilitate this transfer?