Thanks, this is a great thing to be thinking about and a good list of ideas!
Do other subjects come to mind?
Practice no-cost-too-large productive periods
I like this idea. At AI Impacts we were discussing something similar: having "fire drills" where we spend a week (or even just a day) pretending that a certain scenario has happened, e.g. "DeepMind just announced they have a turing-test-passing system and will demo it a week from now; we've got two journalists asking us for interviews and need to prep for the emergency meeting with the AI safety community tonight at 5." We never got around to testing out such a drill but I think variants on this idea are worth exploring. Inspired by what you said, perhaps we could have "snap drills" where suddenly we take our goals for the next two months and imagine that they need to be accomplished in a week instead, and see how much we can do. (Additionally, ideas like this seem like they would have bonus effects on morale, teamwork, etc.)
I don’t know what is entailed in cultivating that virtue. Perhaps meditation? Maybe testing one’s self at literal risk to one’s life?
This virtue is extremely important to militaries. Does any military use meditation as part of its training? I would guess that the training given to medics and officers (soldiers for whom clear thinking is especially important) might have some relevant lessons. Then again, maybe the military deals with this primarily by selecting the right sort of people rather than taking arbitrary people and training them. If so, perhaps we should look into applying similar selection methods in our own organizations to identify people to put in charge when the time comes.
Any more ideas?
In this post I discuss some:
Perhaps it would be good to have an Official List of all the AI safety strategies, so that whatever rationale people give for why this AI is safe can be compared to the list. (See this prototype list.)
Perhaps it would be good to have an Official List of all the AI safety problems, so that whatever rationale people give for why this AI is safe can be compared to the list, e.g. "OK, so how does it solve outer alignment? What about mesa-optimizers? What about the malignity of the universal prior? I see here that your design involves X; according to the Official List, that puts it at risk of developing problems Y and Z..." (See this prototype list.)
Perhaps it would be good to have various important concepts and arguments re-written with an audience of skeptical and impatient AI researchers in mind, rather than the current audience of friends and LessWrong readers.
Thinking afresh, here's another idea: I have a sketch of a blog post titled "What Failure Feels Like." The idea is to portray a scenario of doom in general, abstract terms (like Paul's post does, as opposed to writing a specific, detailed story) but with a focus on how it feels to us AI-risk-reducers, rather than focusing on what the world looks like in general or what's going on inside the AIs. I decided it would be depressing and not valuable to write. However, maybe it would be valuable as a thing people could read to help emotionally prepare/steel themselves for the time when they "are confronted with the stark reality of how doomed we are." IDK.
I guess overall my favorite idea is to just periodically spend time thinking about what you'd do if you found out that takeoff was happening soon. E.g. "Deepmind announces turing-test system" or "We learn of convincing roadmap to AGI involving only 3 OOMs more compute" or "China unveils project to spend +7 OOMs on a single training run by 2030, with lesser training runs along the way" I think that the exercise of thinking about near-term scenarios and then imagining what we'd do in response will be beneficial even on long timelines, but certainly super beneficial on short timelines (even if, as is likely, none of the scenarios we imagine come to pass).
I speculate (based on personal glimpses, not based on any stable thing I can point to) that there's many small sets of people (say of size 2-4) who could greatly increase their total output given some preconditions, unknown to me, that unlock a sort of hivemind. Some of the preconditions include various kinds of trust, of common knowledge of shared goals, and of person-specific interface skill (like speaking each other's languages, common knowledge of tactics for resolving ambiguity, etc.).
[ETA: which, if true, would be good to have already set up before crunch time.]
One of the biggest considerations would be the process for activating "crunch time". In what situations should crunch time be declared? Who decides? How far out would we want to activate and would there be different levels? Are there any downsides of such a process including unwanted attention?
If these aren't discussed in advance, then I imagine that far too much of the available time could be taken up by whether to activate crunch time protocols or not.
PS. I actually proposed here that we might be able to get a superintelligence to solve most of the problem of embedded agency by itself. I'll try to write it up into a proper post soon.
There are probably a class of people for whom working on AI alignment is not worth it/optimal/their concern before crunch time, but becomes their main focus once crunch time is officially declared. Something akin to sleeper agents, if you will.
There should be a network ready to tap on these people's assets/skills when the signal is launched.
Future productivity tools would include text/workflow analysis. This goes better with more data, so you could record all you do, be it via screen recording, a time tracker program, or a keylogger. In particular, if what you do all day is think, write down your stream of consciousness if your thoughts look like a stream of words and your fingers can keep up. GPT-4 might well look back over your diary and tell you the paper you missed.
Is there an open-source lifelogging app, so privacy-conscious people's data isn't lost to the void?
My productivity appeared to go up by an order of magnitude a month ago when I started Vyvanse. Switching from playing video games all day to doing math all day went from a sense of "that ought to be possible somehow, can't the doctor just wave a wand" to effortless. (Although I still procrastinate on bureaucratic obligements.) Know your amphetamines, talk to your doctor. If we take folk wisdom into account and model a chance of polymorphing into a junkie on the streets or a hippie in the church, I'm not pulling Merlin's name on you to shut up and calculate, because I am not that wise, nor that comparatively advantaged to unilaterally command the community. But I am telling you to calculate whether you are in my previous position, then listen to the more conservative of your calculations and your gut.
If the crunch happens, wouldn't you profit from setting it up like a company? I guess if AI Alignment ends up a crisis that humanity only deals with last minute, but then with full effort, the funding will be there.
Get most of the people in the same office (relocate if necessary), have a company structure that is proven to work (hierarchies where necessary but not too much), get momentum going so you motivate each other to work longer and more productively.
Another consideration is that a lot of people that weren't doing AI Safety but were operating in a similar space might join the effort if the crunch happens, like Biontech developing the Covid vaccine although they were working on cancer before that.
This includes all big tech companies, so in case of a crunch the space of people doing Alignment Work might look very different.