x

LESSWRONG

LW

Liam Carroll — LessWrong

Liam Carroll

Top postsTop post

Liam Carroll

Message

Mathematician, musician, hiking guide. Researcher residing in Australia jointly funded by Timaeus and the Gradient Institute. I am focused on understanding the generalisation properties of models based on Singular Learning Theory and Developmental Interpretability and relating this to alignment.

Website: https://www.liamcarroll.au/

556

Ω

131

7

16

3y

Liam Carroll

Mathematician, musician, hiking guide. Researcher residing in Australia jointly funded by Timaeus and the Gradient Institute. I am focused on understanding the generalisation properties of models based on Singular Learning Theory and Developmental Interpretability and relating this to alignment.

Website: https://www.liamcarroll.au/

Top postsTop post

DSLT 0. Distilling Singular Learning Theory

TLDR; In this sequence I distill Sumio Watanabe's Singular Learning Theory (SLT) by explaining the essence of its main theorem - Watanabe's Free Energy Formula for Singular Models - and illustrating its implications with intuition-building examples. I then show why neural networks are singular models, and demonstrate how SLT provides a framework for understanding phases and phase transitions in neural networks. Epistemic status: The core theorems of Singular Learning Theory have been rigorously proven and published by Sumio Watanabe across 20 years of research. Precisely what it says about modern deep learning, and its potential application to alignment, is still speculative. Acknowledgements: This sequence has been produced with the support of a grant from the Long Term Future Fund. I'd like to thank all of the people that have given me feedback on each post: Ben Gerraty, @Jesse Hoogland , @mfar, @LThorburn , Rumi Salazar, Guillaume Corlouer, and in particular my supervisor and editor-in-chief Daniel Murfet. Theory vs Examples: The sequence is a mixture of synthesising the main theoretical results of SLT, and providing simple examples and animations that illustrate its key points. As such, some theory-based sections are slightly more technical. Some readers may wish to skip ahead to the intuitive examples and animations before diving into the theory - these are clearly marked in the table of contents of each post. Prerequisites: Anybody with a basic grasp of Bayesian statistics and multivariable calculus should have no problems understanding the key points. Importantly, despite SLT pointing out the relationship between algebraic geometry and statistical learning, no prior knowledge of algebraic geometry is required to understand this sequence - I will merely gesture at this relationship. Jesse Hoogland wrote an excellent introduction to SLT which serves as a high level overview of the ideas that I will discuss here, and is thus recommended pre-reading to thi

Stagewise Development in Neural Networks

Growth and Form in a Toy Model of Superposition

DSLT 1. The RLCT Measures the Effective Dimension of Neural Networks

Australian AI Safety Forum 2024

We're excited to announce the inaugural Australian AI Safety Forum, taking place on November 7-8, 2024, in Sydney, Australia. This event aims to foster the growth of the AI safety community within Australia. Apply now! Key Details * Dates: November 7-8, 2024 * Location: Sydney Knowledge Hub, The University of...

Sep 27, 2024•42

Stagewise Development in Neural Networks

by Jesse Hoogland, Liam Carroll, and Daniel Murfet

> TLDR: This post accompanies The Developmental Landscape of In-Context Learning by Jesse Hoogland, George Wang, Matthew Farrugia-Roberts, Liam Carroll, Susan Wei and Daniel Murfet (2024), which shows that in-context learning emerges in discrete, interpretable developmental stages, and that these stages can be discovered in a model- and data-agnostic way...

Mar 20, 2024•96

Growth and Form in a Toy Model of Superposition

> TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developmental stages of the Toy Model of Superposition, understanding growth and form from the perspective of Singular Learning Theory (SLT). Ernst Haeckel's Kunstformen der Natur (1904),...

Nov 8, 2023•92

DSLT 4. Phase Transitions in Neural Networks

TLDR; This is the fourth main post of Distilling Singular Learning Theory which is introduced in DSLT0. I explain how to relate SLT to thermodynamics, and therefore how to think about phases and phase transitions in the posterior in statistical learning. I then provide intuitive examples of first and second...

Jun 24, 2023•35

DSLT 3. Neural Networks are Singular

TLDR; This is the third main post of Distilling Singular Learning Theory which is introduced in DSLT0. I explain that neural networks are singular models because of the symmetries in parameter space that produce the same function, and introduce a toy two layer ReLU neural network setup where these symmetries...

Jun 20, 2023•38

DSLT 2. Why Neural Networks obey Occam's Razor

TLDR; This is the second main post of Distilling Singular Learning Theory which is introduced in DSLT0. I synthesise why Watanabe's free energy formula explains why neural networks have the capacity to generalise well, since different regions of the loss landscape have different accuracy-complexity tradeoffs. I also provide some simple...

Jun 18, 2023•31

DSLT 0. Distilling Singular Learning Theory

TLDR; In this sequence I distill Sumio Watanabe's Singular Learning Theory (SLT) by explaining the essence of its main theorem - Watanabe's Free Energy Formula for Singular Models - and illustrating its implications with intuition-building examples. I then show why neural networks are singular models, and demonstrate how SLT provides...

Jun 16, 2023•97

Load More (7/8)