When you first read about AI risk, it sounds like science fiction, and I'm used to slowly working my way around to the topic, so that I don't sound like a lunatic. But the past week has really changed the conversation! Some of the highlights:
The profile and commitment to "pacing the frontier" has dramatically risen in the past few days! But note that we only can make a deal if we can verify that the deal is being kept. Which, to me, makes it obvious that "compute verification" is the most important problem on Earth, even if you've never heard of it[1].
Don't feel bad; hardly anyone has. I dug through all of the research papers that I could find on it, amounting to ~60 papers total (i.e. you could read literally all accumulated knowledge of the field in a week or so). Depending on how you slice the numbers, there are about 72 researchers actively working on the problem, and most aren't full-time; I estimate that the global population of folks answering the Most Important Question On Earth is around 25 FTE (well, I got serious about it last month, now it's up to 26).
I hope you join our ranks! If you, personally, work on a Verification problem for the next 6 months, you could increase what we know and/or have solved by a couple percentage points; that's HUGE!
For a list of specific problems, and how you can help, see the equivalent post on canaryinstitute.ai, which lays out six problem areas and points to the resources for each.
When you first read about AI risk, it sounds like science fiction, and I'm used to slowly working my way around to the topic, so that I don't sound like a lunatic. But the past week has really changed the conversation! Some of the highlights:
The profile and commitment to "pacing the frontier" has dramatically risen in the past few days! But note that we only can make a deal if we can verify that the deal is being kept. Which, to me, makes it obvious that "compute verification" is the most important problem on Earth, even if you've never heard of it [1] .
Don't feel bad; hardly anyone has. I dug through all of the research papers that I could find on it, amounting to ~60 papers total (i.e. you could read literally all accumulated knowledge of the field in a week or so). Depending on how you slice the numbers, there are about 72 researchers actively working on the problem, and most aren't full-time; I estimate that the global population of folks answering the Most Important Question On Earth is around 25 FTE (well, I got serious about it last month, now it's up to 26).
I hope you join our ranks! If you, personally, work on a Verification problem for the next 6 months, you could increase what we know and/or have solved by a couple percentage points; that's HUGE!
For a list of specific problems, and how you can help, see the equivalent post on canaryinstitute.ai, which lays out six problem areas and points to the resources for each.
For a more detailed walkthrough of what looks like the strongest near-term proposal, see my earlier "If you can't trust, then verify!". ↩︎