(This post is an output of the Safe AI Germany (SAIGE) Incubator, specifically the "How will AI affect the democracy equilibrium", which is mentored by Simon (aka Towards_Keeperhood). Sophia and Arslan did the majority of the work and writing.) We’re really interested in how we ended up living in a...
2.1 Summary In the last post, I introduced model-based RL, which is the frame we will use to analyze the alignment problem, and we learned that the critic is trained to predict reward. I already briefly mentioned that the alignment problem is centrally about making the critic assign high value...
1.1 Summary and Table of Contents Why would an AI "want" anything? This post answers that question by examining a key part of the structure of intelligent cognition. When you solve a novel problem, your mind searches for plans, predicts their outcomes, evaluates whether those outcomes achieve what you want,...
This post provides an overview of the sequence and covers background concepts that the later posts build on. If you're already familiar with AI alignment, you can likely skim or skip the foundations section. 0.1 What is this Sequence? This sequence explains the difficulties of the alignment problem and our...
Max Tegmark recently published a post "Which side of the AI safety community are you in?", where he carves the AI safety community into 2 camps: > Camp A) "Race to superintelligence safely”: People in this group typically argue that "superintelligence is inevitable because of X”, and it's therefore better...
> A normal person thinks of attacking the problem on a single day, like sending out hunters to cull the wolves and forcefully decrease their number. A Mage of Equilibrium thinks ahead to where the forces will eventually settle, and asks how to change the balancing point. (All quotes in...
This is IMO high-quality material from projectlawful which I thought is worth crossposting directly to lesswrong. (This might dispel some wrong beliefs you hold (though maybe only if you actively try to apply or practice the techniques). If you think you're currently in a situation where you wouldn't be able...