Grokking (ML)

Edited by Morpheus last updated 29th Feb 2024

A Phenomenon in machine learning where a machine learning model generalizes to a test set only long after it achieved perfect loss on the training set.

Posts tagged Grokking (ML)

373A Mechanistic Interpretability Analysis of Grokking

Ω

Neel Nanda, Tom Lieberum

3y

Ω

48

114QAPR 5: grokking is maybe not *that* big a deal?

Ω

Quintin Pope

2y

Ω

15

101Explaining grokking through circuit efficiency

Ω

Vikrant Varma, Rohin Shah

2y

Ω

11

83Ambiguous out-of-distribution generalization on an algorithmic task

Wilson Wu, Louis Jaburi

5mo

6

75Grokking, memorization, and generalization — a discussion

Kaarel, Dmitry Vaintrob

2y

11

46Paper+Summary: OMNIGROK: GROKKING BEYOND ALGORITHMIC DATA

Ω

Marius Hobbhahn

3y

Ω

11

36Mesa-Optimizers via Grokking

Ω

orthonormal

3y

Ω

4

33The slingshot helps with learning

Wilson Wu

8mo

0

23An interactive introduction to grokking and mechanistic interpretability

Ω

Adam Pearce, Asma Ghandeharioun

2y

Ω

3

21A short project on Mamba: grokking & interpretability

Alejandro Tlaie

9mo

0

20AXRP Episode 29 - Science of Deep Learning with Vikrant Varma

Ω

DanielFilan

1y

Ω

1

10Grokking Beyond Neural Networks

Jack Miller

2y

0

3Minor interpretability exploration #1: Grokking of modular addition, subtraction, multiplication, for different activation functions

Rareș Baron

5mo

13

LESSWRONG
LW

Grokking (ML)

LESSWRONG
LW

Grokking (ML)