JasonB

Summary Safe deployment of an AI system requires that we can make confident claims about its behaviour on out-of-distribution deployment inputs on the basis of only pre-deployment evaluations. One approach to making such claims is to take a cognitive perspective, in which we interpret the AIs behaviour in terms of latent cognitive constructs, such as motivations, intentions, and goals. Because the same behaviour may be compatible with a range of underlying cognition—such as scheming, fitness-seeking, or aligned motivations—inferring cognition from a behavioural snapshot can be tricky. In this post, we introduce the idea of Developmental Cognitive Interpretability (DCI), which aims to model how cognitive constructs change over the course of training. Further, by understanding how cognition results from training pipelines, we can predict agent behaviour resulting from pipelines that have not yet been tested. We discuss core assumptions and philosophical background of DCI, and lay out a broader research agenda. We have some initial evidence that the methodology works in at least one toy setting, and our current main uncertainty is whether we can scale our approach to LLMs. We invite those interested in working on these problems to reach out to us at jrb239[at]cam[dot]ac[dot]uk and edward[at]geodesicresearch[dot]org. 1. Motivation Confidently predicting that an AI system will not cause harm in deployment is the central challenge of AI safety. Pre-deployment evidence of alignment must be collected on inputs we can safely test, but deployment will inevitably give the model dangerous inputs where misbehaviour could be catastrophic. Being able to confidently say that an AI will behave as desired out of its evaluation distribution requires us to predict its OOD behaviour. How might we do this? One approach is to try to understand what a model is doing internally at a mechanistic level. However, the most ambitious versions of Mechanistic Interpretability may be out of rea