This is a special post for quick takes by der. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Prediction (influenced by R1-Zero): By EOY, expert-level performance will be reported on outcome prediction for a certain class of AI experiments - those that can be specified concisely in terms of code and data sets that:
are frequently used and can be referenced by name, e.g. MNIST digits, or
are small enough to be given explicitly, or
are synthetic, specified by their exact distribution in code.