Hell yeah!

This matches my internal experience that caused me to bring a ton of resources into existence in the alignment ecosystem (with various collaborators):

• aisafety.info - Man, there really should be a single point of access that lets people self-onboard into the effort. (Helped massively by Rob Miles's volunteer community, soon to launch a paid distillation fellowship)
• aisafety.training - Maybe we should have a unified place with all the training programs and conferences so people can find what to apply to? (AI Safety Support had a great database that just needed a frontend)
• aisafety.world - Let's make a map of everything in AI existential safety so people know what orgs, blogs, funding sources, resources, etc exist, in a nice sharable format. (Hamish did the coding, Superlinear funded it)
• ea.domains - Wow, there sure are a lot of vital domains that could get grabbed by squatters. Let's step in and save them for good orgs and projects.
• aisafety.community - There's no up-to-date list of online communities. This is an obvious missing resource.
• Rob Miles videos are too rare, almost entirely bottlenecked on the research and scriptwriting process. So I built some infrastructure which allows volunteers to collaborate as teams on scripts for him, being tested now.
• Ryan Kidd said there should be a nice professional site which lists all the orgs in a format which helps people leaving SERI MATS decide where to apply. aisafety.careers is my answer, though it's not quite ready yet. Volunteers wanted to help write up descriptions for orgs in the Google Docs we have auto-syncing with the site!
• Nonlinear wanted a prize platform, and that seemed pretty useful as a way to usefully use the firehose of money while FTXFF was still a thing, so I built Superlinear.
• There are a lot of obvious low-hanging fruit here. I need more hands. Let's make a monthly call and project database so I can easily pitch these to all the people who want to help save the world and don't know what to do. A bunch of great devs joined!
• and 6+ more major projects as well as a ton of minor ones, but that's enough to list here.

I do worry I might be neglecting my actual highest EV thing though, which is my moonshot formal alignment proposal (low chance of the research direction working out, but much more direct if it does). Fixing the alignment ecosystem is just so obviously helpful though, and has nice feedback loops.

I like this post, with one exception: I don't think putting out fires feels like putting out fires. I think it feels like you're utterly confused, and instead of nodding your head when you explain the confusion & people try to resolve it but you don't understand them, continuing to actively notice & chase the confusion no matter how much people decrease your status due to you not being able to understand what they're saying. It feels far more similar to going to school wearing a clown suit than heroically putting out obvious-to-you fires.

This was a big driver behind my vegan nutrition project: I could not believe people were strongly pushing drastic diet changes without any concern for nutrition, when the mitigations were costless morally and almost so resource-wise. 

For a concrete example, consider Devansh. Devansh came to me last year and said something to the effect of, “Hey, wait, it sounds like you think Eliezer does a sort of alignment-idea-generation that nobody else does, and he’s limited here by his unusually low stamina, but I can think of a bunch of medical tests that you haven’t run, are you an idiot or something?” And I was like, “Yes, definitely, please run them, do you need money”.

What sort of tests might these be, can you say? Eliezer is certainly not the only one with “low stamina” problems, and if there are medical tests to run that he wouldn’t already have had done, I’d like to know about them!

Like, keep your eye out. For sure, keep your eye out.

I think this is related to my relative optimism about people spending time on approaches to alignment that are clearly not adequate on their own. It's not that I'm particularly bullish on the alignment schemes themselves, it's that don't think I'd realized until reading this post that I had been assuming we all understood that we don't know wtf we're doing so the most important thing is that we all keep an eye out for more promising threads (or ways to support the people following those threads, or places where everyone's dropping the ball on being prepared for a miracle, or whatever). Is this... not what's happening?

Curated. 

I don't have as strong intuitions about this as So8res  does, but I do think this is a useful heuristic. The post feels both useful epistemically, and motivationally. I liked reading comments from other people describing plans they embarked on because it seemed like "holy shit, is nobody doing this?"

I do kinda wish I had more than vague intuitions backing this up. Ironically, while I'd be interested in someone studying "What motivations tend to drive the largest effect sizes on humanity? How do you control for survivorship bias? Is So8res right about this being a useful prompt?"... it neither gives me a strong sense of "geez, everyone is idiotically dropping the ball on this, I believe in my heart this is the best thing" nor seem really like the top result of a measured, careful spreadsheet of possible goals. 

(But, you're reading this and thinking either "man, people are idiotically dropping the ball not having done a rigorous analysis of this" or "man I think So8res is wrong that you need to believe in your goals for them to be particularly useful, and my careful spreadsheet of goals says that measuring this effect is the best use of my time", um, I'm interested in what you find)

For a concrete example, consider Devansh. Devansh came to me last year and said something to the effect of, “Hey, wait, it sounds like you think Eliezer does a sort of alignment-idea-generation that nobody else does, and he's limited here by his unusually low stamina, but I can think of a bunch of medical tests that you haven't run, are you an idiot or something?" And I was like, "Yes, definitely, please run them, do you need money".

I've always wondered about things in this general area. Higher levels of action that improve the productivity of alignment researchers (well not just researchers, anyone in the field) seems like a very promising avenue to explore.

For example, I know that for me personally, "dealing with dinner" often takes way longer than I hope, consumes a lot of my time, and makes me less productive. That's a problem that could easily be solved with money (which I'm working towards). Do alignment researchers also face that problem? If so it seems worth solving.

Continuing that thought, some people find cooking to be relaxing and restorative but what about things like cleaning, paperwork, and taxes? Most people find that to be somewhat stressful, right? And reducing stress helps with productivity, right? So maybe some sort of personal assistant a la The 4-Hour Work Week for alignment researchers would make sense.

And for medical stuff, some sort of white glove membership like what Tim Urban describes + resurrecting something like MetaMed to be available as a service for higher-impact people like Eliezer also sounds like it'd make sense.

Or basically anything else that can improve productivity. I was gonna say "at a +ROI" or something, but I feel like it almost always will be. Improved productivity is so valuable, and things like personal assistants are relatively so cheap. It reminds me of something I heard once about rich businesspeople needing private yachts: if the yacht leads to just one more closed deal at the margin then it paid for itself and so is easily worth it. Maybe alignment researchers should be a little more "greedy" in that way.

A different way to improve productivity would be through better pedagogy. Something I always think back to is that in dath ilan "One hour of instruction on a widely-used subject got the same kind of attention that an hour of prime-time TV gets on Earth". I don't get the sense that AI safety material is anywhere close to that level. Bringing it to that point would mean that researchers -- senior, junior, prospective -- would have an easier time going through the material, which would improve their productivity.

I'm not sure how impactful it would be to attract new researchers vs empowering existing ones, but if attracting new researchers is something that would be helpful I suspect that career guidance sorts of things would really yield a lot of new researchers.

Well, I had "smart SWE at Google who is interested in doing alignment research" in mind here. Another angle is recruiting top mathematicians and academics like Terry Tao. I know that's been discussed before and perhaps pursued lightly, but I don't get the sense that it's been pursued heavily. Being able to recruit people like Terry seems incredibly high impact though. At the very least it seems worth exploring the playbooks of people in related fields like executive recruiters and look for anything actionable.

Probably more though. If you try to recruit an individual like Terry there's an X% chance of having a Y% impact. OTOH, if you come across a technique regarding such recruitment more generally, then it's an X% chance of finding a technique that has a Y% chance of working on Z different people. Multiplying by Z seems kinda big, and so learning how to "do recruitment" seems pretty worthwhile.

A lot of this stuff requires money. Probably a lot of it. But that's a very tractable problem, I think. And maybe establishing that ~$X would yield ~Y% more progress would inspire donors. Is that something that has been discovered before in other fields?

Or maybe funding is something that is already in abundance? I recall hearing that this is the case and that the limitation is ideas. That never made sense to me though. I see a lot of things like those white glove medical memberships that seem obviously worthwhile. Are all the alignment researchers in NYC already members of The Lanby for $325/month? Do they have someone to clean their apartments? If not and if funding truly is abundant, then I "feel shocked that everyone's dropping the ball".

I feel shocked that so little effort is being put into human genetic enhancement, relative to its potential. Everyone here seems focused on AI!

https://twitter.com/carmenleelau/status/1593354133146402816 is another recent formulation of ~the same idea.

Maybe I don't know what I'm talking about and obviously we've tried this already.

I've heard Eliezer mention that the ability to understand AI risk is linked to Security Mindset.
Security Mindset is basically: you can think like a hacker, of exploits, how to abuse rules etc. So you can defend against hacks & exploits. You don't stop at basic "looks safe to me!" 

There are a lot of examples of this Security/Hacker Mindset in HPMOR. When Harry learns of rates between magical coins vs his known prices for gold, silver, etc, he instantly thinks of a scheme to trade between the magical and the muggle world to make infinite money.

Eliezer also said that Security Mindset is something you either got or not. 
I remember thinking: that can't be true! 

Are we bottlenecking AI alignment on not having enough people with Eliezer-level Security Mindset, and saying "Oh well, it can't be taught!"?!
(That's where I've had the "people are dropping the ball" feeling. But maybe I just don't know enough.)

Two things seem obvious to me:
- Couldn't one devise a Security Mindset test, and get the high scorers to work on alignment? 
(So even if we can't teach it, we get more people who have it)(I assume it was a similar process to find Superforecasters).
- Have we already tried really hard to teach Security Mindset, so that we're sure it can't be taught?
Presumably, Eliezer did try, and concluded it wasn't teachable?

I won't be the one doing this, since I'm unclear on whether I'm Security gifted myself (I think a little, and I think more than I used to, but I'm too low g to play high level games).