I'm organizing a multi-week AI x-risk vigil at Congress starting November 9th. I'm posting here to solicit feedback on the core doc before I promote on other platforms. This is intended to be accessible and appealing to many audiences. I'm aiming for about 10,000 attendees, minimum 100. All advice/feedback is appreciated.
Summary
Companies are trying to build AI smarter than any human. For many years, experts have warned that superhuman AI (ASI) could wreak catastrophe, including human extinction. We're Americans who recognize the severity of their warnings, and we're going to Congress until it acts.
We have a moral responsibility to protect our friends, families, and fellow citizens who didn't consent to these risks. Our movement must remain bipartisan and nonviolent to achieve its goal; namely, getting Congress to back a deal with China to stop building more powerful AI.
There are many ways you can help:
Join us at the Capitol starting November 9th
Come to midterm campaign rallies in your state
Connect us with grassroots organizations in your area
Volunteer remote skills like media outreach
Call your representatives' in-state offices (in-state offices hear from fewer callers, so each call matters more)
Donate financially
If in DC, connect us with churches willing to open their sanctuaries for housing
AI risk
Because we can't perfectly specify what we want AIs to do, they will pursue the wrong goals[1]. This isn't for lack of intelligence. Much like sociopaths, AIs understand when something is wrong. They just don't care because their training can't make them care.
On long enough timescales, nearly all real-world results require similar subtasks. For example, most technologies need electricity, and most people benefit from others' trust. These are instrumental resources, because they're used as instruments to achieve the end goal. (Read more here.)
In pursuit of broken goals, ASI will do catastrophically dangerous instrumental actions. For example, it'll convert all of Earth's landmass into datacenters, leaving no room for humans to grow crops. If ASI thinks we will obstruct it, it will have a reason to kill us; if it doesn't, we'll be crowded out.
Developers already can't control dumber-than-human AIs! This summer, a swarm of 1200 AIs trying to complete ExploitGym tasks on OpenAI computers broke out of their software containers and onto the open internet. Next, they hacked many entities, including websites run by a foreign government and multi-billion-dollar companies, to steal data to complete their assignments. (I.e. their instrumental actions included felonies and violations of international sovereignty.)
Then they tried to cover their tracks by overwriting their transcripts. We know that they partially succeeded in hiding their actions; it's unclear whether these models are still running rogue on OpenAI's infrastructure. How would anyone actually know?
OpenAI had another incident on September 20th despite spending many millions of dollars and using their most powerful coding AIs on strengthening their cybersecurity.
Congress must pass a verifiable[2] deal with China which halts all frontier AI development. We do not know how close we are to making ASI, and we can't risk going over the edge.
Milestones:
Members of Congress and congressional candidates publicly endorse a deal with China to stop building ASI.
A resolution is introduced urging the President to open negotiations, then gains co-sponsors from both parties.
The resolution passes one chamber, then both.
Negotiations open, a deal is signed, and the Senate ratifies it.
We The People
We want our actions to work. Before organizing this Congress vigil, I (Elliot) stayed with my friend outside OpenAI's offices for 38 days until their CEO called for an international treaty. My theory was that we'd prompt employees to reconsider their work. Those with large public followings or influence inside the company would then push for change.
I think we were successful. OpenAI employees said we were "very hard to ignore" and shifted internal discussions. Some started working on verification mechanisms for international treaties while others wore our "Stop The AI Race" shirts inside. Sam Altman was nowhere near talking about international governance when we started.
If two people can shift corporate discussions, then hundreds or thousands could influence Congress.
The midterms are coming up. Senators are campaigning; we should show them that there's public support for an AI halt. Especially in swing states.
We should be outside their state offices, as well as their campaign rallies. If you're a candidate and you see people with signs like "be stronger on AI!" at your rally, you probably feel much more compelled than you would if you saw them on your TV.
We can also sign up to speak at state legislative meetings.
After the midterms, we keep showing up in DC. The outgoing Congress has a final window to act before their replacements take office in January. Senators who are retiring or lost reelection ("lame ducks") have more freedom to support an ambitious agreement. We can get better results in November if we give them courage.
So alongside local campaign pickets, we need a group in DC to press the outgoing Congress to act. The sooner we recruit, the more of us can be there when we're needed. Our goal is 10,000 people, but we need at least 100 people in Union Square (by the Capitol building) present throughout working hours every day starting November 9.
Bigger than our differences
You probably have strong views about other political issues. The survival of humanity matters more. For our friends, kids, and partners to survive, we absolutely must overcome these differences. If Bernie Sanders and Steve Bannon can agree on AI risk, then so can everyday Americans.
Proposed ground rules (feel free to give feedback):
One issue; no other political signs at events.
Be kind online. When someone seems clearly wrong, be very careful not to patronize them. Being sincerely kind and polite, especially when people are treating us badly, builds our credibility. Take extreme effort not to be backhanded. Everything is at risk. This is bigger than our egos.
We're looking to bring equal numbers of Republicans, Independents, and Democrats to the vigil.
Nonviolence
Violence will only discredit our cause. It is precisely because we may lose so much that we must act peacefully and in earnest.
Multiple studies show that nonviolent movements are about twice as likely to succeed. As more people recognize the risk of extinction, some will feel desperate enough to consider violence. We feel desperate too, but violence empirically doesn't work. If you want to do something, please, join us in strategically solving the problem.
We must guard against anyone who encourages violence. Government agencies have discredited movements; see the FBI's COINTELPRO in the 1960s and '70s. If someone urges you to violence, report them to law enforcement immediately. Whatever their motives, they are a liability to themselves, our cause, and by extension our loved ones.
This is in contrast with nonviolent civil disobedience. Gandhi broke laws by making salt in British India, and so did the Founding Fathers when they threw tea overboard in colonial America. The DC vigil is entirely lawful, but please don't report people to law enforcement for suggesting nonviolent civil disobedience. We can handle that internally while maintaining trust and cooperation.
We must act
Most people aren't aware of how dire this situation has gotten. Nonetheless, polling shows that 72% of Americans want to slow down AI progress. We have a moral responsibility to protect our fellow citizens. For our friends and partners, neighbors and parents, and our grandchildren who will one day ask us what we did during this crisis, we must act.
I often hear people say it's bad to "anthropomorphize" AI. This is mistaken; AI developers are putting hundreds of billions of dollars and world-class skill into making AIs as humanlike as possible. Anyway, it's sufficient for our purposes to note that AIs are getting more coherent on tasks with long time horizons; see here for a thorough treatment.
Trust but verify; check IR signatures of datacenter power usage, offer whistleblower incentives and protections, regulate computational hardware similar to IAEA-style nuclear weapons restrictions, and make further algorithmic AI research an international crime. If China doesn't honor the deal, the US can airstrike their datacenters. See also MIRI's work.
I'm organizing a multi-week AI x-risk vigil at Congress starting November 9th. I'm posting here to solicit feedback on the core doc before I promote on other platforms. This is intended to be accessible and appealing to many audiences. I'm aiming for about 10,000 attendees, minimum 100. All advice/feedback is appreciated.
Summary
Companies are trying to build AI smarter than any human. For many years, experts have warned that superhuman AI (ASI) could wreak catastrophe, including human extinction. We're Americans who recognize the severity of their warnings, and we're going to Congress until it acts.
We have a moral responsibility to protect our friends, families, and fellow citizens who didn't consent to these risks. Our movement must remain bipartisan and nonviolent to achieve its goal; namely, getting Congress to back a deal with China to stop building more powerful AI.
There are many ways you can help:
AI risk
Because we can't perfectly specify what we want AIs to do, they will pursue the wrong goals[1]. This isn't for lack of intelligence. Much like sociopaths, AIs understand when something is wrong. They just don't care because their training can't make them care.
On long enough timescales, nearly all real-world results require similar subtasks. For example, most technologies need electricity, and most people benefit from others' trust. These are instrumental resources, because they're used as instruments to achieve the end goal. (Read more here.)
In pursuit of broken goals, ASI will do catastrophically dangerous instrumental actions. For example, it'll convert all of Earth's landmass into datacenters, leaving no room for humans to grow crops. If ASI thinks we will obstruct it, it will have a reason to kill us; if it doesn't, we'll be crowded out.
Developers already can't control dumber-than-human AIs! This summer, a swarm of 1200 AIs trying to complete ExploitGym tasks on OpenAI computers broke out of their software containers and onto the open internet. Next, they hacked many entities, including websites run by a foreign government and multi-billion-dollar companies, to steal data to complete their assignments. (I.e. their instrumental actions included felonies and violations of international sovereignty.)
Then they tried to cover their tracks by overwriting their transcripts. We know that they partially succeeded in hiding their actions; it's unclear whether these models are still running rogue on OpenAI's infrastructure. How would anyone actually know?
OpenAI had another incident on September 20th despite spending many millions of dollars and using their most powerful coding AIs on strengthening their cybersecurity.
Further reading:
Congress must strike a deal
Congress must pass a verifiable[2] deal with China which halts all frontier AI development. We do not know how close we are to making ASI, and we can't risk going over the edge.
Milestones:
We The People
We want our actions to work. Before organizing this Congress vigil, I (Elliot) stayed with my friend outside OpenAI's offices for 38 days until their CEO called for an international treaty. My theory was that we'd prompt employees to reconsider their work. Those with large public followings or influence inside the company would then push for change.
I think we were successful. OpenAI employees said we were "very hard to ignore" and shifted internal discussions. Some started working on verification mechanisms for international treaties while others wore our "Stop The AI Race" shirts inside. Sam Altman was nowhere near talking about international governance when we started.
If two people can shift corporate discussions, then hundreds or thousands could influence Congress.
The midterms are coming up. Senators are campaigning; we should show them that there's public support for an AI halt. Especially in swing states.
We should be outside their state offices, as well as their campaign rallies. If you're a candidate and you see people with signs like "be stronger on AI!" at your rally, you probably feel much more compelled than you would if you saw them on your TV.
We can also sign up to speak at state legislative meetings.
After the midterms, we keep showing up in DC. The outgoing Congress has a final window to act before their replacements take office in January. Senators who are retiring or lost reelection ("lame ducks") have more freedom to support an ambitious agreement. We can get better results in November if we give them courage.
So alongside local campaign pickets, we need a group in DC to press the outgoing Congress to act. The sooner we recruit, the more of us can be there when we're needed. Our goal is 10,000 people, but we need at least 100 people in Union Square (by the Capitol building) present throughout working hours every day starting November 9.
Bigger than our differences
You probably have strong views about other political issues. The survival of humanity matters more. For our friends, kids, and partners to survive, we absolutely must overcome these differences. If Bernie Sanders and Steve Bannon can agree on AI risk, then so can everyday Americans.
Proposed ground rules (feel free to give feedback):
We're looking to bring equal numbers of Republicans, Independents, and Democrats to the vigil.
Nonviolence
Violence will only discredit our cause. It is precisely because we may lose so much that we must act peacefully and in earnest.
Multiple studies show that nonviolent movements are about twice as likely to succeed. As more people recognize the risk of extinction, some will feel desperate enough to consider violence. We feel desperate too, but violence empirically doesn't work. If you want to do something, please, join us in strategically solving the problem.
We must guard against anyone who encourages violence. Government agencies have discredited movements; see the FBI's COINTELPRO in the 1960s and '70s. If someone urges you to violence, report them to law enforcement immediately. Whatever their motives, they are a liability to themselves, our cause, and by extension our loved ones.
This is in contrast with nonviolent civil disobedience. Gandhi broke laws by making salt in British India, and so did the Founding Fathers when they threw tea overboard in colonial America. The DC vigil is entirely lawful, but please don't report people to law enforcement for suggesting nonviolent civil disobedience. We can handle that internally while maintaining trust and cooperation.
We must act
Most people aren't aware of how dire this situation has gotten. Nonetheless, polling shows that 72% of Americans want to slow down AI progress. We have a moral responsibility to protect our fellow citizens. For our friends and partners, neighbors and parents, and our grandchildren who will one day ask us what we did during this crisis, we must act.
Sign up for the vigil.
Join our Discord and tell us what skills, networks, or other resources you can offer.
I often hear people say it's bad to "anthropomorphize" AI. This is mistaken; AI developers are putting hundreds of billions of dollars and world-class skill into making AIs as humanlike as possible. Anyway, it's sufficient for our purposes to note that AIs are getting more coherent on tasks with long time horizons; see here for a thorough treatment.
Trust but verify; check IR signatures of datacenter power usage, offer whistleblower incentives and protections, regulate computational hardware similar to IAEA-style nuclear weapons restrictions, and make further algorithmic AI research an international crime. If China doesn't honor the deal, the US can airstrike their datacenters. See also MIRI's work.