Background

Lucius: I recently held a small talk presenting an idea for how and why deep learning generalises. It tried to reduce concepts from Singular Learning theory back to basic algorithmic information theory to sketch a unified picture that starts with Solomonoff induction and, with a lot of hand waving, derives that under some assumptions, just fitting a big function to your data using a local optimisation method like stochastic gradient descent maybe, sorta, kind of, amounts to a cheap bargain bin approximation of running Solomonoff induction on that data.

Lucius: The parametrised function we fit to the data has to be the sort that can act like a vaguely passable approximation of a compute-limited Universal Turing machine, with parameter configurations playing the role of programs.

Lucius: In a Universal Turing machine, 'simpler' hypotheses with less ‘burdensome details’ will be implemented by exponentially more equivalent programs. So, simpler hypotheses take up exponentially more volume in program space. Thus, if we start with a uniform prior over all programs of some fixed length $l$ , we’re effectively implementing an approximation of the Solomonoff prior, giving exponentially more weight to simpler hypotheses.^[1]

Lucius: Similarly, in a good parametrised function, ‘simpler’ fits to the data with less ‘burdensome details’ will correspond to exponentially more equivalent parameter settings than complicated ones. So, simple functions will take up an exponentially larger volume in parameter space than complicated functions, and a uniform prior over parameter space will exponentially favour them before we even see any data.

Lucius: Later, I posted the slides for the talk in a group chat because someone asked for them. I did not include a GIANT DISCLAIMER that these slides are NOT RIGOROUS and present an OVERSIMPLIFIED STORY to help the audience follow the thread of the explanation better. They should NOT BE PARSED without ACCOMPANYING COMMENTARY.

Lucius: This was very stupid of me.

Lucius: Kaarel and Dmitry promptly jumped in to critique the slides, and I replied to their critiques. Afterwards, we all realised that the ensuring discussion basically acts as the commentary the slides were missing, and also touched on many other ideas regarding the nature of intelligence that may be of interest to some.

Lucius: S...