I have seen a lot of confusion recently surrounding exactly how outer and inner alignment should be defined and I want to try and provide my attempt at a clarification.

Here's my diagram of how I think the various concepts should fit together:

The idea of this diagram is that the arrows are implications—that is, for any problem in the diagram, if its direct subproblems are solved, then it should be solved as well (though not necessarily vice versa). Thus, we get:

$\begin{matrix} inner alignment & \to objective robustness outer alignment \land objective robustness & \to intent alignment intent alignment \land capability robustness & \to alignment \end{matrix}$

And here are all my definitions of the relevant terms which I think produce those implications:

(Impact) Alignment: An agent is impact aligned (with humans) if it doesn't take actions that we would judge to be bad/problematic/dangerous/catastrophic.

Intent Alignment: An agent is intent aligned if the optimal policy for its behavioral objective^[1] is impact aligned with humans.

Outer Alignment: An objective function $r$ is outer aligned if all models that perform optimally on $r$ in the limit of perfect training and infinite data are intent aligned.^[2]

Robustness: An agent is robust if it performs well on the base objective it was trained under even in deployment/off-distribution.^[3]

Objective Robustness: An agent is objective robust if the optimal policy for its behavioral objective is impact aligned with the base objective it was trained under.

Capability Robustness: An agent is capability robust if it performs well on its behavioral objective even in deployment/off-distribution.

Inner Alignment: A mesa-optimizer is inner aligned if the optimal policy for its mesa-objective is impact aligned with the base objective it was trained under.

And an explanation of each of the diagram's implications:

$inner alignment \to objective robustness$ : If a model is a mesa-optimizer, then its behavioral objective should match its mesa-objective, which means if it's mesa-objective is aligned with the base, then it's behavioral objective should be too.

$outer alignment \land objective robustness \to intent alignment$ : Outer alignment ensures that the base objective is measuring what we actually care about and objective robustness ensures that the model's behavioral objective is aligned with that base objective. Thus, putting them together, we get that the model's behavioral objective must be al...