One version of a good-ish trajectory I imagine has something like a step 1.b to defuse the catastrophic misuse problem, which involves sufficient foresight and exploratory engineering to anticipate particularly disruptive or destructive prospects, and 2.a, checking in on development projects to make sure they're either foreclosing those or delayed enough to 2.b, develop countermeasures in the mean time.
Failure modes also left as an exercise to the reader.
I continue to believe we should pause frontier AI development. Any discussion of alternative strategies should be thought of as planning for contingencies.
A unifying driver behind many post-alignment risks—catastrophic risks that remain even if we solve the alignment problem—is that by strong default, ASI would end liberal democracy. Liberalism—in which people have individual rights, autonomy, and the ability to choose their own destiny—is an important force protecting human welfare. [1] When people are free, we are reasonably good at making our lives better of our own volition.
Many post-alignment risks have a certain flavor. AI-empowered terrorism; coups; permanent dictatorships; concentration of power. Those risks already exist today (and existed 20 years ago), but they're mitigated by the fact that power is relatively evenly distributed across people. The most powerful person in the world doesn't have an extraordinary advantage over the 10th-most-powerful person. ASI could change that.
If people still have civil liberties post-ASI, that will only be because the controllers of ASI allow us to have them. [2]
One way of thinking goes: AI will be extremely powerful. If everyone had their own personal AI, we could each use it to protect our own interests, and things will turn out okay for us. But how do you get there? It's not going to happen automatically, but it may be possible to set up a gradual process to keep power balanced.
Cross-posted from my website.
Democratizing AI vs. putting a democratic government in charge of AI
The "democra" words (democracy, democratize) are overloaded with meanings, so I want to be clear about which meaning I'm referring to. By "democratizing AI", I'm talking about ensuring that many people have access to AI. One could also speak of a singleton AI that's controlled by a democratic government, or a singleton AI that's democratically controlled (people vote on what the AI should do). As Andy Masley writes, "homogenizing democracy" (as he calls it) entails forcing everyone to do what the majority wants. That's not good. That's why the title of this article speaks of "preserving civil liberties", rather than "preserving democracy".
(To be fair, I do wish we would get to vote on whether AI companies should be allowed to build unsafe recursively self-improving superintelligence, rather than them getting to unilaterally build it and put all our lives at risk.)
A sketch of how we might democratize AI
To be clear: democratizing AI is not a great plan. But it's less bad than a lot of other plans.
To be clear x2: for "democratizing AI" to be a "less bad plan", it has to be implemented with particular care. Otherwise, it's just a bad plan. The obvious way of democratizing ASI doesn't work:
I doubt that it helps with the second problem, but a gradualist approach has some chance of sidestepping the first. (By which I mean it has maybe a 1–10% chance of working.)
Here's the plan:
The idea is that if the leading AI developer decides not to do Step 3 (instead keeping the new AI for themselves), then they are risking a conflict with the rest of the world, and they can't win that conflict.
There are many ways this plan could go badly, which are left as an exercise for the reader.
What we should really do is pause AI development. Specifically, major countries should sign an international treaty agreeing not to build dangerous AI. The plan to "democratize AI" is riskier, but at the same time, it doesn't seem easier? In order to preserve the balance of power, you need AI capabilities to advance slowly, and you need some way to enforce that. In short, you still need an international binding agreement not to build AI [except under certain narrow conditions], and not to use recursive self-improvement. You still need enforcement mechanisms like transparency and GPU monitoring; you still need a way for counterparties to shut down AI development if one party goes rogue (something like a remote kill switch on all the data centers).
Two non-obvious issues with democratizing AI
Democratizing AI has many issues. I will make note of two particular under-discussed problems.
Liberalism only protects those inside it
Even if we somehow succeed at preserving personal liberties by democratizing AI, and we succeed at making sure the AI is actually good for people, that still doesn't get us a good outcome. Most sentient beings alive today are non-human animals. Today's liberalism has failed to help them; almost all animals live lives much worse than humans'. Democratizing AI would not straightforwardly improve on the status quo. There is no realistic scenario where (e.g.) every factory-farmed chicken has her own superintelligent AI assistant.
If we preserve liberalism-for-humans, that does not clearly flow into good outcomes for non-human animals, digital minds, and whatever other sentient beings may exist in the future. Beings who cannot assert their rights do not automatically benefit; other measures are required to ensure their well-being.
AI proliferation increases catastrophic risks from competition
Having many ASI agents with different goals could cause tremendous suffering via retributivism, war, unfortunate decision theory, or agents threatening torture (and making good on those threats) to incentivize other agents to do what they want.
Center on Long-Term Risk and Center for Reducing Suffering have discussed this concern in more detail. For example, see Safe Pareto Improvements Research Agenda and Agential s-risks.
And the lack of liberalism for animals has a lot to do with why life is so miserable for them. ↩︎
I read a version of this statement in a LessWrong comment, but I couldn't find the original source. ↩︎