This is a special post for quick takes by sil. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
There are many advances that are NOT counterexamples, however all of the most impressive (from a human standpoint) ones seem to be counterexamples. Perhaps this points to an asymmetry in difficulty for LLMs that we have not yet calibrated for.
Are such problems generally easier? If they're the easiest of the hard problems, this is consistent with
There are many advances that are NOT counterexamples, however all of the most impressive (from a human standpoint) ones seem to be counterexamples.
Difficulty of a problem can only be defined relative to a solver. For example think about Calculator vs Human, dividing two 20 digit numbers vs catching a ball.
It is an asymmetry like this on the front of counterexamples vs proofs of validity that I am hypothesizing exists. If these were "easy" problems for humans they would have been solved a long time ago, they're definitely not.