Sumit Kumar

Induction heads across three different models Here https://github.com/xonx4l/induction-universality I did an experiment on induction heads across three different models i.e. GPT-2 small, Pythia-160m and Qwen2.5-0.5B. It involves knowing what it cost for these models when the induction heads are removed . In all three every top-5 head falls between 0.36...