I am working on a fairly large (or large-seeming to me) project in the AI space. I've been soft launching it for a little bit but I've been working on a hard announcement post for LessWrong, Substack, X, LinkedIn, etc. but basically it's a 60 player live event AI governance game combining the wargaming work of D. Scott Phoenix and Shahar Avin with the scaleability, easy mass distribution, and mechanical rigour of British-style Megagames. I wrote a post about this in April and I'm currently working off of a generous microgrant from BlueDot Impact.
Facts about me:
I'm extremely new to the world of AI safety, existential risk, and Rationality generally. I didn't know who "Scott Alexander" was in March and I didn't know what a "transformer architecture" was in July. I do not consider myself a hard R "Rationalist" yet (mostly due to not being caught up in the readings yet).
I am absolutely not a shape rotator. I have a CS minor and essential programming skills but I am terrible at mathematics; I have no role to play in technical aligment.
Six months ago I felt the biggest dangers of AI was the speculative bubble, data center power consumption, and the death of apprenticeship/the future of work.
Since visiting Berkeley for Inkhaven 2 and becoming AGI-pilled I've been trying to map my skills (communication, content creation, design, a mind for systems, overall "generalist" skills) onto the AI problem and see whether there's anything I can do. My reasons for wanting to do something are probably multifaceted and not entirely altruistic.
I am 24 years old and I'm not certain humanity will be around by the time I hit 40.
I was at breakfast today with some colleagues (much more Rationalist/EA than myself) and I brought up the fact that I am working on this Megagame. I explained it a little bit. One of them said (in a very friendly, non-offensive, intellectually curious way) that he's not sure that such a project has any real value. I gave a bit of a rambly answer but I think I replied with the basic rhetoric of my BlueDot grant proposal. Something like:
It seems like most AI wargames are inaccessible for most people, since you need to be connected to play them.
It would be better if lots of people could step into the shoes of AI decision makers like great powers and AI labs.
There are a lot of normies out there (like me, 6 months ago) who are thoughtful and worried about AI but simply don't understand the danger of AI-assisted human-induced catastrophe nor ASI-induced catastrophe.
Certainly my hope is that this isn't just a game that gets played by AI doomers at AI doomer events.
By exposing more normies to the difficult, conflicting incentives of AI governance in this period of rapid progress, maybe I will inspire smart/influential/dedicated people to doing important AI work.
That seemed to satisfy him for the purposes of the casual conversation— he appreciated the value of good propaganda— but I've been pondering the merits my project since this morning.
Clarification: I've already received the grant and I have started spending money to build, test, deliver, and report on the game by Christmas, so I'm locked into this. I'm not calling into question whether I should do what I promised to do. I'm just reflecting now whether my project is a good use of my time, BlueDot's money, and the attention of the AI scene/humanity at large.
So all things considered, is what I'm doing actually worth it? How do I know? And if it's not, what should I do?
Worry 1 - It's really hard for me to determine or prove impact
The fantasy of my project is that people will walk away totally converted— epistemically baptised, as I was when @Sean Herrington showed me the METR Graph in April, into understanding that AI is extremely powerful and only getting more powerful.
That's a really hard theory of impact to prove. Maybe it'll happen to 50% of the game's normie players. Maybe it'll happen to 1%. And every person whose mindset I change about AI governance and x-risk already has a very small probability of going on to be a major player in the AI world. How many philanthropic dollars should it cost to convert[1] someone? And if my game is an inefficient way to transform capital/normie attention into lower x-risk, how do I have any way of knowing that?
Worry 2 - I'm making unproductive noise
Human attention is the most valuable resource on the planet. I still believe that. And the amount of attention available, both within the AI space and in my target audience, is finite. If I run my game at some kind of gathering or event, is it really the optimal thing to be doing? Maybe it's more engaging than a 100 page report or a 2 hour lecture, but would the players be better served by something more informational and directly persuasive? After all, I'm not designing my game strictly as an academic simulation. I think my design is going to be rigorous, modelled after reality at least somewhat, but in many ways I am optimising my game around the edges to be fun. Does a fun day for 60 people change humanity's future? That's a hard argument to make.
That's not even considering the attention required of other AGI-pilled people required for me to make this project work. Unlike a blogpost which lives or dies on its merits, my game requires an upfront investment. "Can I use your EA-adjacent venue for my game?" "You understand AI governance, would you help me facilitate my game?" "Could you retweet this post about my game?" These are questions I'm going to need to ask of serious people in the next few months, and even if they are enthusiastic about my game (as a number of AI governance people here in Toronto seem to be), there is a real cost to that attention.
Worry 3 - I have ulterior motives which are dangerous
I do not believe I am pursuing this project for the money. I am being paid $0/hr to build my game; if I'm lucky I might earn another grant which could include a stipend (BlueDot doesn't do stipends for Rapid Grants) and if I'm very lucky some event/conference/institution might pay me money to facilitate my game for them[2]. Certainly I have no illusions about the fact that Megagame design/facilitation has never been a lucrative career path.
I'm also not particularly interested in status, I don't think. I am not the epistemic peer of the greatest thinkers in AI safety/AI governance, and I don't have any delusions of being a thought leader who goes on podcasts and has thousands of Substack readers.
But if I am honest, beyond my desire to impact the world and decrease x-risk, I must admit I have ulterior motives for what I'm doing:
I want to design games/stories. It's just how my brain works and I want to do it more. A funded project means I get to spend more time doing it.
I want to be accepted by a community. Though I'm not a hard R, I feel a connection to the Rationalist community that I've never felt before. A game like this is an excuse to spend more time talking to them/existing in their spaces without 'earning it'[3]. I feel like this is a different motivation than "status" but maybe I'm wrong.
I want some kind of income. I'm currently zero-income and I'm perpetually anxious about where my next income source will be. AI safety/governance seems like something I could maybe have some small bill-paying role in and I am motivated to find a foot in the door in some way or another.
The idea of just getting on with my life as AI x-risk looms is a really hard one for me to contend with.
Coda
I am in the 95th percentile of neuroticism according to the OSPP OCEAN test, and it's very easy for me to fail to calibrate against that. But I think my question is worth interrogating.
I am extremely excited to be working on this project. I can't wait to announce it, refine it, build the play kit, develop the software, run the playtests, and everything else involved. I am beyond grateful to my grantors for believing in me, and the enthusiasm that people here in Toronto have shown about the project really breathes life into me.
But at the same time, I need to be careful. When I accept grant money or commandeer an AI governance space for the purposes of my game, I am, in a very real way, rolling the dice with the finite resources allocated to humanity's survival and flourishing. I think it's a good sign about me personally that I think about this, but it's unclear to me whether my project is worthwhile. I'd like to figure out how to figure that out.
My game is open source, I should say. The rules will be free. But I plan to offer my GM services to any person/organisation that doesn't want the fuss of learning my game and running it themselves.
Not sure what I mean by this and I'm skeptical of my self-interrogation here. I'm an anxious person by nature so maybe I over-obsess about 'earning a place'? But also I acknowledge that by default I don't have much value to bring to these spaces on my own.
I am working on a fairly large (or large-seeming to me) project in the AI space. I've been soft launching it for a little bit but I've been working on a hard announcement post for LessWrong, Substack, X, LinkedIn, etc. but basically it's a 60 player live event AI governance game combining the wargaming work of D. Scott Phoenix and Shahar Avin with the scaleability, easy mass distribution, and mechanical rigour of British-style Megagames. I wrote a post about this in April and I'm currently working off of a generous microgrant from BlueDot Impact.
Facts about me:
I was at breakfast today with some colleagues (much more Rationalist/EA than myself) and I brought up the fact that I am working on this Megagame. I explained it a little bit. One of them said (in a very friendly, non-offensive, intellectually curious way) that he's not sure that such a project has any real value. I gave a bit of a rambly answer but I think I replied with the basic rhetoric of my BlueDot grant proposal. Something like:
That seemed to satisfy him for the purposes of the casual conversation— he appreciated the value of good propaganda— but I've been pondering the merits my project since this morning.
Clarification: I've already received the grant and I have started spending money to build, test, deliver, and report on the game by Christmas, so I'm locked into this. I'm not calling into question whether I should do what I promised to do. I'm just reflecting now whether my project is a good use of my time, BlueDot's money, and the attention of the AI scene/humanity at large.
So all things considered, is what I'm doing actually worth it? How do I know? And if it's not, what should I do?
Worry 1 - It's really hard for me to determine or prove impact
The fantasy of my project is that people will walk away totally converted— epistemically baptised, as I was when @Sean Herrington showed me the METR Graph in April, into understanding that AI is extremely powerful and only getting more powerful.
That's a really hard theory of impact to prove. Maybe it'll happen to 50% of the game's normie players. Maybe it'll happen to 1%. And every person whose mindset I change about AI governance and x-risk already has a very small probability of going on to be a major player in the AI world. How many philanthropic dollars should it cost to convert[1] someone? And if my game is an inefficient way to transform capital/normie attention into lower x-risk, how do I have any way of knowing that?
Worry 2 - I'm making unproductive noise
Human attention is the most valuable resource on the planet. I still believe that. And the amount of attention available, both within the AI space and in my target audience, is finite. If I run my game at some kind of gathering or event, is it really the optimal thing to be doing? Maybe it's more engaging than a 100 page report or a 2 hour lecture, but would the players be better served by something more informational and directly persuasive? After all, I'm not designing my game strictly as an academic simulation. I think my design is going to be rigorous, modelled after reality at least somewhat, but in many ways I am optimising my game around the edges to be fun. Does a fun day for 60 people change humanity's future? That's a hard argument to make.
That's not even considering the attention required of other AGI-pilled people required for me to make this project work. Unlike a blogpost which lives or dies on its merits, my game requires an upfront investment. "Can I use your EA-adjacent venue for my game?" "You understand AI governance, would you help me facilitate my game?" "Could you retweet this post about my game?" These are questions I'm going to need to ask of serious people in the next few months, and even if they are enthusiastic about my game (as a number of AI governance people here in Toronto seem to be), there is a real cost to that attention.
Worry 3 - I have ulterior motives which are dangerous
I do not believe I am pursuing this project for the money. I am being paid $0/hr to build my game; if I'm lucky I might earn another grant which could include a stipend (BlueDot doesn't do stipends for Rapid Grants) and if I'm very lucky some event/conference/institution might pay me money to facilitate my game for them[2]. Certainly I have no illusions about the fact that Megagame design/facilitation has never been a lucrative career path.
I'm also not particularly interested in status, I don't think. I am not the epistemic peer of the greatest thinkers in AI safety/AI governance, and I don't have any delusions of being a thought leader who goes on podcasts and has thousands of Substack readers.
But if I am honest, beyond my desire to impact the world and decrease x-risk, I must admit I have ulterior motives for what I'm doing:
Coda
I am in the 95th percentile of neuroticism according to the OSPP OCEAN test, and it's very easy for me to fail to calibrate against that. But I think my question is worth interrogating.
I am extremely excited to be working on this project. I can't wait to announce it, refine it, build the play kit, develop the software, run the playtests, and everything else involved. I am beyond grateful to my grantors for believing in me, and the enthusiasm that people here in Toronto have shown about the project really breathes life into me.
But at the same time, I need to be careful. When I accept grant money or commandeer an AI governance space for the purposes of my game, I am, in a very real way, rolling the dice with the finite resources allocated to humanity's survival and flourishing. I think it's a good sign about me personally that I think about this, but it's unclear to me whether my project is worthwhile. I'd like to figure out how to figure that out.
I hate the word "convert" here but it's the best one I can think of.
My game is open source, I should say. The rules will be free. But I plan to offer my GM services to any person/organisation that doesn't want the fuss of learning my game and running it themselves.
Not sure what I mean by this and I'm skeptical of my self-interrogation here. I'm an anxious person by nature so maybe I over-obsess about 'earning a place'? But also I acknowledge that by default I don't have much value to bring to these spaces on my own.