I think an important concept for sensibly modeling a part of the world is what I call a "macroagent", which abstracts over the similarity cluster of teams, companies, AI scaffolds, countries, the market, and the scientific community. In this post, I will lay out the basic ontology for modeling macroagents....
This is a distillation of the AI2040 alignment roadmap (with some parts also drawing from "How do we (more) safely defer to AIs?"). All ideas expressed are by Ryan and Thomas[1]; I only created the diagrams and substantially changed the structure to make it more accessible.[2][3] Note that the original...
(This post is an output of the Safe AI Germany (SAIGE) Incubator, specifically the "How will AI affect the democracy equilibrium", which is mentored by Simon (aka Towards_Keeperhood). Sophia and Arslan did the majority of the work and writing.) We’re really interested in how we ended up living in a...
2.1 Summary In the last post, I introduced model-based RL, which is the frame we will use to analyze the alignment problem, and we learned that the critic is trained to predict reward. I already briefly mentioned that the alignment problem is centrally about making the critic assign high value...
1.1 Summary and Table of Contents Why would an AI "want" anything? This post answers that question by examining a key part of the structure of intelligent cognition. When you solve a novel problem, your mind searches for plans, predicts their outcomes, evaluates whether those outcomes achieve what you want,...
This post provides an overview of the sequence and covers background concepts that the later posts build on. If you're already familiar with AI alignment, you can likely skim or skip the foundations section. 0.1 What is this Sequence? This sequence explains the difficulties of the alignment problem and our...
Max Tegmark recently published a post "Which side of the AI safety community are you in?", where he carves the AI safety community into 2 camps: > Camp A) "Race to superintelligence safely”: People in this group typically argue that "superintelligence is inevitable because of X”, and it's therefore better...