This is a linkpost for work done as part of MATS 9.0 under the mentorship of Richard Ngo. Loss scaling laws are among the most important empirical findings in deep learning. This post synthesises evidence that, though important in practice, loss-scaling per se is a straightforward consequence of very low-order...
A worked example of an idea from physics that I think is underappreciated as a general thinking tool: no measurement is meaningful unless it's stable under perturbations you can't observe. The fix is to replace binary questions ("is this a degree-3 polynomial?", "is this a minimum?") with quantitative ones at...
This is my attempt to summarize Infra-Bayesianism probability theory at a level approaching "high school class in probability", as opposed to the "math-major class in probability theory" of the original. I aimed to focus on including simple exercises (with answers given in 'the back of the book') of the kind...