Conditions are similar to this question on Metaculus, except for the open-source condition (I win even if the AI is closed-source, and in fact I would very much prefer it to be closed-source).
@Zvi has agreed to adjudicate this bet in case there is no agreement on resolution.
Michael has asked me two questions by email, and I'm sharing my answers.
Any thoughts on how to turn winning these sorts of bets into people actually updating?
Geoffrey Hinton mentioned recently that, while GPT4 can "already do simple reasoning", "reasoning is the area where we're still better" [source].
It seems to me that, after being able to beat humans at math, there won't be anything else fundamental where we're still better. I wish more people could realize this now.
For the people who disagree, I would like to get them to make their views known before that. I feel like many people just don't think enough about it and are in a "superposition state", where their belief can collapse to anything without it causing any surprise feelings or model updating on them. Maybe if they think about it and commit to their views today they'll be more surprised when it happens and therefore willing to change their mind about important matters.
Kelvin, thoughts on how you’ll update if it doesn’t at least come close?
Yes. I'll update in the following directions:
That it is much harder for search algorithms to do "amplification" right in rich (very large branching factor) environments like math (and the real-world), than in games such as Go.
That superintelligence is still further away, even if LLMs are having lots of economic use cases and replacing many human jobs.
That we're probably bottlenecked on search algorithms, rather than compute power/model-size. This would have policy implications.
I dunno, I think there are a LOT of old olympiad problems—not just all the old IMOs but also all the old national-level tests from every country that publishes them. (Bottom section here.) I think that even the most studious humans only study a small fraction of existing problems, I think. Like, if someone literally read every olympiad-level problem and solution ever published, then went to a new IMO, I would expect them to find that at least a couple of the problems were sufficiently similar to something they’ve seen that they could get the answer without too much creativity. (That’s just a guess, not really based on anything.)
(That’s not enough for a gold by itself, but could be part of the plan, in conjunction with special-case AIs for particular common types of problems, and self-play-proof-assistant things, etc.)
(EDITED TO ADD DEC 2024: If memory serves, one of the years I took the Putnam (2003-6), one of the problems was to prove an inequality straightforwardly equivalent to the textbook result Hölder's inequality. I submitted a couple-line answer to that effect, and they didn’t give me credit because that was too trivial—under the circumstances I was supposed to also prove Hölder's inequality from something more basic like Cauchy-Schwarz. Someone can check whether I’m remembering right.)
The bet was arranged on Twitter between @MichaelVassar and I (link).
Conditions are similar to this question on Metaculus, except for the open-source condition (I win even if the AI is closed-source, and in fact I would very much prefer it to be closed-source).
@Zvi has agreed to adjudicate this bet in case there is no agreement on resolution.
Michael has asked me two questions by email, and I'm sharing my answers.
Geoffrey Hinton mentioned recently that, while GPT4 can "already do simple reasoning", "reasoning is the area where we're still better" [source].
It seems to me that, after being able to beat humans at math, there won't be anything else fundamental where we're still better. I wish more people could realize this now.
For the people who disagree, I would like to get them to make their views known before that. I feel like many people just don't think enough about it and are in a "superposition state", where their belief can collapse to anything without it causing any surprise feelings or model updating on them. Maybe if they think about it and commit to their views today they'll be more surprised when it happens and therefore willing to change their mind about important matters.
Yes. I'll update in the following directions: