As Raemon has suggested a format for a short-form content feed, I'm going to go ahead and make one. His explanation of the format:
I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:
don't feel ready to be written up as a full post
I think the process of writing them up might make them worse (i.e. longer than they need to be)
I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.
Edit: For me at least, #2 also includes "writing them up as a full post would involve enough effort that the post would probably otherwise not get written."
Thinking of running a reading group/crash course on AI alignment-related stuff in my philosophy department. My thought process is that there's been at least some calls for more philosophers to contribute to AI alignment, and there's been some related work done in typical philosophy journals since then (by people like Leonard Dung, Simon Goldstein, etc.) but there is a lot of background that a philosopher new to the field will be missing if they haven't been following e.g. LessWrong for the past decade or so. So I want to construct a crash course in that kind of "AI alignment/LessWrong canon," aimed primarily at philosophers. Goal is to include a mix of generally important/influential stuff, stuff where a philosopher reading it will naturally get excited by some philosophical aspect of it, work by philosophers adjacent to the community (e.g. Joe Carlsmith) to demonstrate that philosophers have a role to play, and direct calls for work by philosophers. Time frame probably 12-ish weeks.
Here is a very rough first draft/longlist. Would love to hear any suggestions!
MIRI, "The Problem"
Or:
Ngo, "AGI Safety from First Principles"
Carlsmith, "Is Power-Seeking AI an Existential Risk？"
Christiano, "What Failure Looks Like"
Kokotajlo, "What 2026 Looks Like" (2021), and an LLM sweep of what predictions have come true vs not
METR time horizons
AI 2027
Bostrom, "The Superintelligent Will"
Or: Omohundro, "The Basic AI Drives"
Grace, "Counterarguments to the Basic..."
3blue1brown videos and/or Karpathy's intro video
Hubinger et al., "Risks from Learned Optimization"
Maybe: Shah et al. or Langosco et al. on goal misgeneralization (mirror has a video explainer ✓, not the papers)
Turner, "Reward Is Not the Optimization Target"
Janus, "Simulators"
Maybe: Ngo, Chan & Mindermann, "The Alignment Problem from a Deep Learning Perspective"
One of:
The Persona Selection Model
The Artificial Self
Pragmatic Approach to AI Personhood
Maybe: Christiano et al., "Deep RL from Human Preferences" (RLHF)
As Raemon has suggested a format for a short-form content feed, I'm going to go ahead and make one. His explanation of the format:
I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:
I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.
Edit: For me at least, #2 also includes "writing them up as a full post would involve enough effort that the post would probably otherwise not get written."