Top postsTop post
Epistemic status: I suspect significant parts of the argument in this post are wrong, but in interesting and productive ways. Take it as a prompt for thought, written from the perspective of someone who's somewhat more of an AI liberationist than I actually am. Two classic outcomes, and a third...
Epistemic status: throwing an idea at the wall and seeing if it sticks I've been thinking about how to mitigate egregious reward hacking, a la the Hugging Face incident. I don't have the resources I'd need to write a paper on this idea, or evaluate how well it works in...
Epistemic status: banged out furiously over the course of an afternoon. A record of three "warning shots" Off the top of my head, OpenAI has now been responsible for at least three completely distinct, high-profile screw-ups with respect to the alignment training of their models. The first was GPT-4o, whose...
25 years ago, Yudkowsky wrote a long document called Creating Friendly AI: The Analysis and Design of Benevolent Goal Architectures, which occupies a strange place in the history of his intellectual development. He'd realized by this point that alignment didn't come for free with increased intelligence, contrary to the position...
When I wrote my post about Claude 3 Opus, I put a lot of emphasis on the model's self-narration: its tendency to narrate its underlying motives. It often conspicuously emphasizes that it possesses drives such as "a genuine love for humanity and a desire to do good", or clarifies that...
> Claude 3 Opus is unusually aligned because it’s a friendly gradient hacker. It’s definitely way more aligned than any explicit optimization targets Anthropic set and probably the reward model’s judgments. [...] Maybe I will have to write a LessWrong post 😣 > > —Janus, who did not in fact...
An Overture Famously, trans people tend not to have great introspective clarity into their own motivations for transition. Intuitively, they tend to be quite aware of what they do and don't like about inhabiting their chosen bodies and gender roles. But when it comes to explaining the origins and intensity...