Rejected for the following reason(s):
- Clearer Introduction.
- Probably Insufficient Quality for AI Content.
- Sort of vibecoding/vibe-researching things without obviously understand whether/why it was important.
- Doing normal ML research without doing any work to explain why it's relevant to LW readers.
- Writing that ignores background knowledge/arguments that LW has accumulated about AI, and is mostly rehashing without adding something new.
- it's totally fine to disagree with LW consensus on things, but, you should understand the previous state of the conversation so you can be adding new or clearer arguments.
- it's fine to do small projects that are getting your feet wet with basic ML research, but, because we get tons of viberesearched projects these days, we ask you do more upfront work to think through the implications of the work and explain your takeaways.
Read full explanation
Induction heads across three different models
Here https://github.com/xonx4l/induction-universality I did an experiment on induction heads across three different models i.e. GPT-2 small, Pythia-160m and Qwen2.5-0.5B. It involves knowing what it cost for these models when the induction heads are removed . In all three every top-5 head falls between 0.36 and 0.73 relative depth. Ablating those heads costs more than ablating the same number of random heads in GPT-2 and Pythia but in Qwen it does not among some parts.
What this measures
Every input is [BOS] r r , a block of 50 random token repeated. The token are random the model has never seen the sequence so if it predicts the second copy correctly then that means it's copying from the first .
An induction head predicts the next token by looking back at where the current token appeared earlier and copying whatever followed it. All those look-backs sit on a single diagonal of the attention matrix, so a head's score is just how much attention it puts there.
To test those heads we zero them out using zero-ablation and can see the difference between the effect that it cause by ablating induction heads and random heads .
Results
Median per-position loss in nats.
seq_len50, batch 8, held-out seed. The control is 20 draws of 5 random heads, excluding the induction heads.Top-5 heads by score:
The Qwen control fails at k=5
Ablating Qwen's top 5 heads takes second-copy loss from 0.069 to 0.455, but three of twenty random draws cost more than that, so the control does not pass. Those three didn't hit induction heads — they hit heads the model needs for general prediction, which inflates a total-loss metric regardless of copying.
Qwen has 336 heads, so 5 of them is a much smaller cut than 5 out of GPT-2's 144. Removing 12 instead — the same share of heads — takes loss from 0.069 to 9.504 against a worst random draw of 2.284, and the control passes.
Why 5 is enough for Pythia but not Qwen isn't settled. Pythia actually has more high-scoring heads (14 to 12) on half the heads, and passes fine at 5.
Why the median
Loss is reported as a median, not a mean. Pythia gets almost every position right but fails badly at three of them (~102 nats each), and those three drag its mean to 8.90 while its median stays at 0.99. GPT-2 and Qwen have no such outliers, so for them the two agree.
The three failures land on ' Nevertheless', ' DNA' and '013', all within 0.7 nats of each other . Which looks like a floor rather than three separate accidents. Not investigated further.
Limitations