LESSWRONG
LW

anish-lakkapragada

Message

2mo

Experiments to understand Singular Learning Theory's Free Energy & Local Learning Coefficient (LLC)

This post is a summary of a collection of Singular Learning Theory experiments I ran during the fall of 2025. This post assumes some familiarity to SLT, namely its famous asymptotic relating free-energy Fn with the local-learning coefficient (LLC) λ. For a more in-depth description of these experiments, please see...

Dec 20, 2025•7

anish-lakkapragada

2mo

anish-lakkapragada — LessWrong

anish-lakkapragada

Message

2mo

Experiments to understand Singular Learning Theory's Free Energy & Local Learning Coefficient (LLC)

Dec 20, 2025•7

anish-lakkapragada

2mo

Experiments to understand Singular Learning Theory's Free Energy & Local Learning Coefficient (LLC)

anish-lakkapragada

2mo

This post is a summary of a collection of Singular Learning Theory experiments I ran during the fall of 2025. This post assumes some familiarity to SLT, namely its famous asymptotic relating free-energy $F_{n}$ with the local-learning coefficient (LLC) $λ$ . For a more in-depth description of these experiments, please see the arXiv preprint here. Please feel free to email me for any questions or feedback.

Brief Motivation

Singular Learning Theory (SLT) provides an elegant framework to understand phase transitions in modern neural networks through a physics-inspired Bayesian learning perspective. In particular, SLT introduces two meaningful quantities: the model free-energy $F_{n}$ and the local-learning coefficient (LLC) $λ$ , a model complexity measure. IMHO, the formal definitions of these two quantities are... (read 1536 more words →)