This is a special post for quick takes by Kaj_Sotala. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Similar to other people's shortform feeds, short stuff that people on LW might be interested in, but which doesn't feel like it's worth a separate post. (Will probably be mostly cross-posted from my Facebook wall.)
I doubt that anyone even remembers this, but I feel compelled to say it: there was some conversation about AI maybe 10 years ago, possibly on LessWrong, where I offered the view that abstract math might take AI a particularly long time to master compared to other things.
I don't think I ever had a particularly good reason for that belief other than a vague sense of "math is hard for humans so maybe it's hard for machines too". But formally considering that prediction falsified now.
I disagree with this, in that good mathematics definitely requires at least a little understanding of the world, and if I were to think about why LLMs succeeded at math, I'd probably point to the fact that it's an unusually verifiable task, relative to the vast majority of tasks, and would also think that the fact that you can get a lot of high-quality data also helps LLMs.
Only programming shares these traits to an exceptional degree, and outside of mathematics/programming, I expect less transferability, though not effectively 0 transferability.
Math is definitely just a language. It is a combination of symbols and a grammar about how they go together. It's what you come up with when you maximally abstract away the real world, and the part about not needing any grounding was specifically about abstract math, where there is no real world.
Verifiable is obviously important for training (since we could give effectively infinite training data), but the reason it is verifiable so easily is because it doesn't rely on the world. Also, note that programming languages are also just that, languages (and quite simple ones) but abstract math is even less dependent on the real world than programming.
Similar to other people's shortform feeds, short stuff that people on LW might be interested in, but which doesn't feel like it's worth a separate post. (Will probably be mostly cross-posted from my Facebook wall.)