LESSWRONG
LW

AI Alignment FieldbuildingExistential riskAI
Frontpage

69

Resources I send to AI researchers about AI safety

by Vael Gates
14th Jun 2022
1 min read
12

69

AI Alignment FieldbuildingExistential riskAI
Frontpage

69

Resources I send to AI researchers about AI safety
15Adam Zerner
5Vael Gates
10Vael Gates
3Adam Zerner
1TAG
2Adam Zerner
1TAG
6lc
5KatWoods
2Vael Gates
4plex
3Vael Gates
New Comment
12 comments, sorted by
top scoring
Click to highlight new comments since: Today at 4:47 AM
[-]Adam Zerner3y150

I notice that Eliezer and MIRI are missing. Why is this? Low prestige amongst the academic community? Harsh writing style?

I don't mean to open a can of worms or anything. It just seems engaging with reality and not shying away from it.

Reply
[-]Vael Gates3y50

A great point, thanks! I've just edited the "There's also a growing community working on AI alignment" section to include MIRI, and also edited some of the academics' names and links.

I don't think it makes sense for me to list Eliezer's name in the part of that section where I'm listing names, since I'm only listing some subset of academics who (vaguely gesturing at a cluster) are sort of actively publishing in academia, mostly tenure track and actively recruiting students, and interested in academic field-building. I'm not currently listing names of researchers in industry or non-profits (e.g. I don't list Paul Christiano, or Chris Olah), though that might be a thing to do. 

Note that I didn't choose this list of names very carefully, so I'm happy to take suggestions! This doc came about because I had an email draft that I was haphazardly adding things to as I talked to researchers and needed to promptly send them resources, getting gradually refined when I spotted issues. I thus consider it a work-in-progress and appreciate suggestions. 

Reply
[-]Vael Gates3y100

With respect to the fact that I don't immediately point people at LessWrong or the Alignment Forum (I actually only very rarely include the "Rationalist" section in the email-- not unless I've decided to bring it up in person, and they've reacted positively), there's different philosophies on AI alignment field-building. One of the active disagreements right now is how much we want new people coming into AI alignment to be the type of person who enjoy LessWrong, or whether it's good to be targeting a broader audience. 

I'm personally currently of the opinion that we should be targeting a broader audience, where there's a place for people who want to work in academia or industry separate from the main Rationalist sphere, and the people who are drawn towards the Rationalists will find their way there either on their own (I find people tend to do this pretty easily when they start Googling), or with my nudging if they seem to be that kind of person. 

I don't think this is much "shying away from reality" -- it feels more like engaging with it, trying to figure out if and how we want AI alignment research to grow, and how to best make that happen given the different types of people with different motivations involved.

Reply
[-]Adam Zerner3y30

I'm personally currently of the opinion that we should be targeting a broader audience

Is the implication that, in order to target a broader audience, you think it would be wise to avoid mentions of LessWrong? Is that because you fear such mentions would turn them off?

If so, that seems like an important thing to take note of. Such a perception seems like a bad thing that we should try to fix. On the other hand, it is also possible that it is a net positive because it keeps the community from being "diluted".

I don't think this is much "shying away from reality"

I didn't mean to imply that you personally were. What I meant when I used that phrase is that this feels like a touchy subject that I myself wanted to flinch away from, but I don't actually think I should flinch away from.

Reply
[-]TAG3y10

There's a mention of the rationalist community.

Reply
[-]Adam Zerner3y20

True, but despite that fact, it still feels like Eliezer and MIRI are purposefully left out.

Reply
[-]TAG3y10

How it feels depends on how prominence you them to have.

Reply
[-]lc3y*60

Don't sleep on this stuff Vael Gates keeps putting out. They're doing the lord's work.

Reply
[-]KatWoods3y50

Love this! Added it to our list of AI safety curricula, reading lists, and courses.

Thanks for sharing this.  

Reply
[-]Vael Gates3y20

Thanks for doing that Kat!

Reply
[-]plex3y40

Amazing! Would you be happy for some of the content here to be used as a basis for Stampy answers?

Reply
[-]Vael Gates3y30

Sure! This isn't novel content; the vast majority of it is drawn from existing lists, so it's not even particularly mine. I think just make sure the things within are referenced correctly, and you should be good to go!

Reply
Crossposted to the EA Forum. Click to view.
Moderation Log
Curated and popular this week
12Comments
worth

If you're interested in seeing my up-to-date recommendations, please see the Arkose Resource Center!

For the purpose of reducing the number of pages where I keep updated recommendations, I'm now retiring this post. However, you're welcome to look at the 2023 web archive version.

Mentioned in
6Papers to start getting into NLP-focused alignment research