I don't like the scheming vs reward-hacking dichotomy. It doesn't seem like a collectively exhaustive description of possible alignment failures in theory, and I haven't seen sufficiently convincing arguments for why they cover almost all misalignment failures^[1] in practice. (Which isn't to say such arguments don't exist but at least they haven't been made easily accessible to a moderately plugged-in non-technical researcher in this space like myself).

Sometimes when people say "X is a false dichotomy" they mean the truth is in the middle but I want to be clear I'm not saying that^[2]. I currently think there are probably large chunks of the catastrophic misalignment probability space that don't look like some admixture of either reward hacking or scheming.

I don't have amazing examples of what the "secret third thing" might be but I still find the structural shape of the dichotomy suspect. OOD generalization especially after reflection seems like a plausible candidate but I'm not wedded to it.

1. ^{^}

   I don't have a crisp definition of "misalignment failure" but maybe I mean something like "reasons that AI systems might agentically kill us for reasons other than misuse." So I won't count clear human misuse like agents design a bioweapon for a terrorist. I also won't count killing us by accident (as long as it's an accident and not an "accident") like an AI managing a nuclear plant fucked up some calculations.

2. ^{^}

   This might be true too but not the focus of this shortform.