Regarding your first point, you say that "As far as I can tell, nobody's trying to build this", but it seems that Stop The AI Race (which you know from being a speaker a couple of times at our events) matches your description pretty closely. If not, I'd be curious what specifically you think we're not doing or doing wrong compared to that hypothetical org.
In particular, regarding having nowhere to point people for a movement with leaders organizing marches, we're organizing a protest on Nov 14 that will happen in multiple cities and countries at the same time, explicitly asking for an AI Treaty. More info here: aitreatymarch.org.
Further, I think it's important for it to avoid taking a stance on other AI issues: There's a lot of people who hate AI's effect on artists, and people who hate AI's effects on the education system, and people who hate AI's effects on the environment; and there's also people who like current AI as a technology but still don't want to go extinct from it; and we should all be able to team up when it comes to averting extinction.
I resonate in principle with the idea that averting extinction should be at the core of what we're focusing on, and that taking a stance on a bunch of other issues could alienate folks.
However, we might disagree on whether a movement trying to prevent AI extinction should at least mention that these different causes are not as separate as people think.
For instance, when people are worried about their livelihoods, the impact of AI on their kids' education, or the use of AI in war, I want to tell them that stopping the insane market race to build superintelligence (aka stopping frontier model development, with teams working on more powerful models moving to alignment instead) also addresses their concerns by making sure the situation doesn't get even worse (fewer jobs, even more advanced AI in the military, etc.).
There should exist an org that can ally with the folks who hate datacenters on account of their water usage, when it comes to lobbying for an end to dangerous AI training runs, but that does not require this belief among its members. A group that focuses narrowly on averting extinction, and that serves as a warm and friendly ally to all those who will oppose that particular segment of AI, as part of a broader coalition that not all members will agree with.
Being a warm and friendly ally is what Stop The AI Race aimed for at the July 11 protest. Food & Water Watch joined the coalition (for "The AI Protest") and gave a speech, so they were our ally, but Stop The AI Race's DNA has always been about the risk from AI extinction. When we were outside OpenAI's office for eight weeks in the Bay Area and one month in NYC, or doing the hunger strike to stop the ai race outside GDM, extinction was explicitly mentioned.
Members don't need to share views to agree about our goal. The goal of Stop The AI Race is to stop racing ahead. We're agnostic about the reasoning people have for ending up with that goal, as long as folks agree with that being a valuable goal to have (whether you think that AI should eventually resume or not).
Extinction 1) is what many of the most active folks care about, especially the ones who spent weeks outside OpenAI’s office, and 2) is at the center of our message, but caring about it is not something we require from everyone who joins.
My belief is that it's important for there to exist such a movement that avoids the "omnicause attractor" (of being attended only by people who also pick some particular side about environmental issues or middle eastern diplomacy issues or whatever)
Stop The AI Race is nonpartisan. Republicans have joined our protests, and we've invited right-wing groups to speak. We've also been reaching out to religious groups more traditionally associated with the right to build long-term coalitions there.
I was a grant evaluator for Lightcone Commons last round. Funders disbursed about $45mil through the program, and this included funding roughly my first $6.8 million worth of recommendations. There are other projects I would've preferred to fund over what I did fund, but nobody was doing them and so they weren't on offer. Here's a series of projects that I'd fund in the next round, in rough order of priority. The next round is starting now, and is scheduled to end mid-December. I expect rounds to continue at similar speed. I expect similar amounts of funding will route through me in future rounds.
Disclaimer: I am NOT looking to manage anyone in doing any of these projects. I am not looking to help someone get started. Initiative and leadership and management capacity (and people that can just figure out how the heck to get something done, and that have the connections to do it) are the critical bottlenecks here. I can throw money at promising pitches, but not really help them all that much otherwise.
1. Leaders for a broad public movement against superintelligent machines. In the wake of Coxon Day, a number of random people around the country have reached out to me asking if there's any way to coordinate and organize a grassroots movement. I wish there was a place to point them, that would help support them in organizing people locally and ultimately working towards some big march. My belief is that it's important for there to exist such a movement that avoids the "omnicause attractor" (of being attended only by people who also pick some particular side about environmental issues or middle eastern diplomacy issues or whatever). Further, I think it's important for it to avoid taking a stance on other AI issues: There's a lot of people who hate AI's effect on artists, and people who hate AI's effects on the education system, and people who hate AI's effects on the environment; and there's also people who like current AI as a technology but still don't want to go extinct from it; and we should all be able to team up when it comes to averting extinction. There should exist an org that can ally with the folks who hate datacenters on account of their water usage, when it comes to lobbying for an end to dangerous AI training runs, but that does not require this belief among its members. A group that focuses narrowly on averting extinction, and that serves as a warm and friendly ally to all those who will oppose that particular segment of AI, as part of a broader coalition that not all members will agree with. As far as I can tell, nobody's trying to build this. If there was a good and compelling application for such a thing from someone who could convince me they had a chance of pulling it off, I'd attempt to route them $2mil in the next round (and $6mil/yr).
2. An org dedicated to informing elites. In the wake of the Coxon affair, lots more people are wondering what’s up with the “AI might kill us all” stuff. This probably includes a bunch of people who have disproportionate cultural sway, such as artists, bankers (who probably have a lot of sway over the Trump administration, which cares a lot about the stock market), union leaders (who probably have a lot of sway over what stances politicians are willing to take), etc. I’d love to see an effort that just networks to those folks and offers access to knowledge on this issue. Perhaps by like, literally arranging dinners with folks like me or folks like various whistleblowers. I think it’d be good if they could also offer access to the sorts of folks who say that everything’s going to be fine! Mostly I want powerful folks to be able to hear all the arguments. I think my arguments just straight-up win, so it doesn’t need to be a group focused on propagating one particular message. It needs to be a group that has the powerful connections, and that is able to offer personalized access to people with the knowledge and the arguments. If there was a good and compelling application for such a thing from someone who could convince me they had a chance of pulling it off, I'd attempt to route them $1mil in the next round (and could be convinced to fund a lot more but I’m not convinced this is the sort of thing that scales).
3. An org dedicated to coordinating AI company employees. I think that AI company employees have a lot more power than they realize, to sway the behavior of their host company. The companies didn’t come out and talk about “pacing” until after the employees made a lot of noise. (I think this is true both about the ‘pacing’ letter in July, and about the ‘pacing’ statements in September.) Employees would have a lot more power if they organized. Employees will have a lot less power once their jobs are automated away, and less still once they’re slain along with the rest of us. So they should get moving on the whole ‘organization’ business. This is a sort of project that most people probably couldn’t attempt (for lack of connections and trust). But if some team had the connections and trust and wanted to put in a really solid effort for helping minimize the effort and coordination costs it’d take for AI company employees to organize, I’d attempt to route them $2mil in the next round (and could be convinced to fund a lot more but I’m not sure this sort of thing scales).
Other projects I’m excited about but that seem more niche include:
A big part of the issue with these is gonna be convincing me that you’ve got the experience and connections and competence to maybe pull the project off, rather than just raising your hand and saying “let me try,” alas. But also there’s a decent chance that, in the next round, I’m gonna fund the closest versions I can even if they’re far from optimal. (Though I might not, if the options on offer seem like they’re more likely to poison the wells or render it harder for the next folks. I don’t quite know how to make these tradeoffs when time is possibly-but-not-clearly short.)