This post is part of a series on my “crazy” ideas in AI Safety. The probability that any individual idea is useful is low; however, if even one of them turns out to matter, the impact could be large. Or maybe it will inspire someone to do something useful, remotely related to the initial idea. That makes the expected value high enough for me to write them down.
I think it could be valuable to have an organization — or at least a dedicated project within an existing organization—whose only purpose is to explain the risks of superintelligence to as many people as possible. The way to measure I’d propose is the number of people who to sign the Statement on Superintelligence using the link provided by the organization. I like this metric because the statement already has many respected signatories; signing it is a relatively “normal” action and not obviously a fringe activity. The organization would commit to maximal transparency about how it spends donations and about the cost-effectiveness (e.g., dollars per additional signature) of each outreach method it tries.
Of course, communication is not a new idea in this space. Rob Miles, Doom’s Debate, MIRI’s recent attempts, the authors of the Statement, and many others already focus heavily on public communication. So why add yet another group?
My answer is: for the sake of clear, measurable impact.
If I donate to Rob Miles or Liron Shapira, what happens? Probably better video quality, more views, and maybe more people becoming concerned about AI Safety. But measuring that impact is very hard.
The Statement on Superintelligence, on the other hand, provides a concrete, interpretable metric. Its growth has slowed; most people who were already inclined to sign have done so. A specialized organization would need to actually convert new people, and it would be legible whether they succeeded.
Could the authors of the Statement, or another existing AI safety organization, take this on? Possibly. But then it should be run as a clearly delineated project with its own transparent budget, isolated from unrelated work. And it might still be less confusing for donors if the project lived inside its own separate organization.
A natural worry is that once a metric is defined, the organization may Goodhart it—for example, by incentivizing people to sign the Statement without truly changing their views. But this particular failure mode should be easy for donors to detect, and would likely result in immediate drops in credibility and funding. That doesn’t eliminate all risks of Goodharting, but it makes the most obvious failure mode less plausible.
I also think such an organization could reach segments of the general public that current communicators rarely reach. By expanding the audience—beyond people who already watch AI-safety YouTube channels or read rationalist forums—we could meaningfully raise broader awareness of superintelligence risks. Increasing that awareness may in turn improve the odds that society chooses not to build a superintelligence before we understand how to make it safe.