programjames

Motivation for this post: Discrete Distribution Networks (Lei Yang, ICLR 2025). Generative models aim to reproduce a real-world distribution given many training examples. One way to do this is to train the neural network so that the least amount of information is needed to reconstruct the training examples. For example,...

Perhaps you've had this experience before: you generally like living with your roommate, except when it comes to the mess they leave around. They seem content to wallow in their filth, so you feel like you always have to pick up after them to maintain a sane level of cleanliness....

(I realize many people here are part of this cult, and it may be upsetting to hear that people consider it a cult. But I figure "No, Rationalism is Not a Cult" needs a response.) My definition of 'cult' is close to > An ideology that employs anti-epistemology to convince...

Or, why GAN training looks so funky. Solomonoff's Lightsaber > The simplest explanation is exponentially more important. Suppose I give you a pattern, and ask you to explain what is going on: 1,2,4,8,16,… Several explanations might come to mind: * "The powers of two," * "Moser's circle problem," * x424−x312+11x224+7x12+1,...

Key points: * Standardized assessments do not provide signals for the top ~10% of students. * Education studies based on these poor signals lead to poor policies. For example, suspensions allegedly do not improve achievement among peers. However, the signal of "achievement" is "achieving credit in the class", which >80%...

In his meeting with Zelenskyy in the Oval Office, Trump briefly said > I could tell you right now there's a nation thinking about going to war on something that nobody in this room has ever even heard about. Two smaller nations—but big, still big—and I think I've stopped it,...

The Serpinski triangle is usually introduced as a fractal made up of three smaller Serpinski triangles. However, that isn't enough to know what it looks like. Which of these is the Serpinski triangle? The issue with recursive definitions is they create a vicious circle. To get around this, you need...