The plan experiment

"The plan" is an experiment in collaborative discussion. The central topic is helping humans navigate development of AGI. Its development is a hard technical problem, and making sure that the way it's developed is beneficial to all humans makes it even harder. In the meantime, there is a perceived aura of pressure. This pressure comes from multiple sources:

• Internal: this is a complex situation, so people want to make sure they are making the right decisions. The stakes are very high, so people want to get it right.
• Novelty: humans have never built an AGI before, so there are a lot of unknown unknowns.
• Us vs them perspective: people imagine that others are competing with them to get to AGI faster than they would. They imagine those people taking shortcuts in safety in order to speed up the timeline. (See AI arms race)
• Lack of clear communication: a lot of conversations about this happen in private.
• Lack of clear direction: there is no plan for people to follow or discuss.

This experiment exists to address all of these issues by recognizing their manifestations in each individual. To fully address these issues, the plan has to be public.

What's in the plan?

National response framework^[1] provides a detailed document describing how various organizations and federal governments can respond to an event of emergency. I imagine the final output of the plan to be something along these lines, although it'll be framed in a more proactive way.^[2]

Specifically, the plan will have these components:

• List of all the relevant organizations and people
• Various milestones for anticipating and measuring AGI progress
• Various suggested actions that should be taken once the milestones are reached
• Various events that might happen (e.g. a call to nationalize AGI development) and possible responses

Execution

Steps

1. Go through existing literature and extract the relevant parts. Use those to write the first version of the plan.
   • In particular, populate the list of stages.
2. I will show the current version of the plan to one person and get their feedback.
   • In that process, I will try to address the pressures I've outlined above.
3. I will add as much of their feedback to the plan as possible, keeping their identity anonymous if requested, though I have a strong preference for people's opinions to be public.
4. Iterate on this process and the plan.

Non-interference stance

It's hard to understand a person's point of view when your own gets in the way. This is why I'm going to follow these guidelines when having one-on-ones with people:

1. My goal is not to make anyone do anything they don't want to do.
2. My goal is not to get the person to adopt the plan.
3. My goal is to explain various parts of the plan to a person, then understand and integrate their feedback into the plan. (Steelmanning their argument in the process.)
4. My goal is for the person to be happy with the way their feedback was integrated into the plan.

In fact, if this approach works well, I'd be happy to take this to its logical conclusion of creating a new class of x-risks participants who are forbidden from interfering in AI development landscape.

The plan keeper

A lot of the plan will come from one-on-one discussions. If all the conversations are stored in one person's mind, they can have future conversations much better. However, if the past conversations were done by two people, than each person can only bring half of all the relevant information. So, naively, this task is much better done by one person. However, that's probably unrealistic. In either case, keeping the group small seems best.

Measure of success

Initial measure of success is: how many people's opinions are integrated into the plan and how happy those people are with their views being represented.

Final measure of success is: are people referring to and following the plan.

(It's possible we should only use the initial measure.)

1. ^{^︎}

   The National Response Framework describes not only how the Federal government organizes itself to respond to natural disasters, terrorist attacks, and other catastrophic events but also the importance of the whole community in assisting with response efforts. The intended audience for this document is individuals, families, communities, the private and nonprofit sectors, faith-based organizations, and local, state, tribal, territorial, insular area, and Federal governments.

2. ^{^︎}

   It's possible NIMS is closer to what I'm looking for. It's "a systematic, proactive approach to guide departments and agencies at all levels of government, nongovernmental organizations, and the private sector to work together seamlessly and manage incidents involving all threats and hazards—regardless of cause, size, location, or complexity—in order to reduce loss of life, property and harm to the environment."