I believe there is a factor beyond model training or model constitution that may significantly shape how an AI system reasons about moral or strategic questions.

Model training implements a prior for a model's behavior, constitutions set guardrails, but we are not tracking the interpretation through which an AI system reasons. Through different interpretations, you can come to completely different conclusions.

A few straightforward examples.

Two doctors are given the same set of symptoms; they come to different diagnoses.

Two Customer Success Managers are given the same set of business metrics; they come to different strategies.

Two Lawyers given the same set of facts, come to different fact patterns and case strategies.

Two supreme court judges given the same constitution, come to different legal positions and comments.

Two VC’s given the same deck, one evangelizes the other rejects.

Whats the difference? Their lived experience, and their interpretation. Simplified, they must possess the knowledge (training), know how to deploy the knowledge (constitution), and realize that the knowledge is even significant (interpretation).

I believe interpretation is an understudied concept, especially in relation to the deployment of ai systems that may act on your individual behalf.

Operationalization of an interpretive layer could be a verifiable static document or construct, that can be used by an AI to apply a relevant interpretation. No different than providing context via RAG, Context Windows, System Prompting, or Basic prompting. This requires the immediate assumption that you can capture an individual’s unique interpretation. But more importantly how do you measure it.

Assuming you can faithfully capture an individual’s interpretation, verifying it is difficult. There are very few datasets that contain longitudinal data on an individual’s line of reasoning, with verifiable ground truth. The level of reasoning being referred to, is deeper than high level demographic data, but lesser than an individual’s raw reflections and conversations over their life. This level of reasoning could be called a Behavioral Specification; a compressed document that is composed by a structured and traceable pattern extraction pipeline, that attempts to faithfully encode your operating principles. For all intents and purposes, it is a compressed representation of your interpretive reasoning, no...

Mark Moseley

Mark Moseley