LESSWRONG
LW

arcee183

Message

Model multitasking: Can a model learn two different tasks simultaneously through Grokking?

The work discussed in this blog post was inspired by Progress Measures for Grokking via Mechanistic Interpretability1. I would encourage you to read this paper in its entirety and check out the list of resources at the end of the post for some more in-depth background information. TL;DR If you...

Feb 15•7

arcee183

arcee183 — LessWrong

arcee183

Message

Model multitasking: Can a model learn two different tasks simultaneously through Grokking?

Feb 15•7

arcee183

Model multitasking: Can a model learn two different tasks simultaneously through Grokking?

arcee183

The work discussed in this blog post was inspired by Progress Measures for Grokking via Mechanistic Interpretability¹. I would encourage you to read this paper in its entirety and check out the list of resources at the end of the post for some more in-depth background information.

TL;DR

If you are familiar with the concepts discussed in the above research paper, you may start reading from The point of this blog post: The second thing!
The research paper that inspired this post is super fun to replicate and is a great introduction to mechanistic interpretability research. As it turns out, Grokking is quite robust and is easily observed for a shallow transformer trained on both... (read 2449 more words →)