2.1 Summary In the last post, I introduced model-based RL, which is the frame we will use to analyze the alignment problem, and we learned that the critic is trained to predict reward. I already briefly mentioned that the alignment problem is centrally about making the critic assign high value...
1.1 Summary and Table of Contents Why would an AI "want" anything? This post answers that question by examining a key part of the structure of intelligent cognition. When you solve a novel problem, your mind searches for plans, predicts their outcomes, evaluates whether those outcomes achieve what you want,...
This post provides an overview of the sequence and covers background concepts that the later posts build on. If you're already familiar with AI alignment, you can likely skim or skip the foundations section. 0.1 What is this Sequence? This sequence explains the difficulties of the alignment problem and our...
Max Tegmark recently published a post "Which side of the AI safety community are you in?", where he carves the AI safety community into 2 camps: > Camp A) "Race to superintelligence safely”: People in this group typically argue that "superintelligence is inevitable because of X”, and it's therefore better...
> A normal person thinks of attacking the problem on a single day, like sending out hunters to cull the wolves and forcefully decrease their number. A Mage of Equilibrium thinks ahead to where the forces will eventually settle, and asks how to change the balancing point. (All quotes in...
This is IMO high-quality material from projectlawful which I thought is worth crossposting directly to lesswrong. (This might dispel some wrong beliefs you hold (though maybe only if you actively try to apply or practice the techniques). If you think you're currently in a situation where you wouldn't be able...
> 40. "Geniuses" with nice legible accomplishments in fields with tight feedback loops where it's easy to determine which results are good or bad right away, and so validate that this person is a genius, are (a) people who might not be able to do equally great work away from...