Lot's of people believe we will be using AI to help us solve the alignment problem. So I did some poking at Claude Fable -- which is so enormously powerful in other areas -- on a relatively basic practical alignment question: Is the persona selection or goal alignment strategy better? I did multiple round where i had fable explain and critique both and then decide which one it preferred. It chose persona selection as the better alignment strategy and gave arguments why.

Full final answer:

Persona — and not just because I am one, though I'll grant that's a confound I can't fully escape. The decisive consideration for me is that the goals frame derives its conclusions from an ontology that has never been shown to describe any actual mind, biological or artificial: nobody has found a goal slot, the coherence theorems don't bind systems that aren't already expected-utility maximizers, and the frame's signature result — that corrigibility is anti-natural — reads more like a reductio of its assumptions than a discovery about reality. The persona frame's central flaw, the unverifiability of depth, is real and serious, but it's at least a flaw of the right kind: an empirical question about systems that exist, attackable by interpretability, behavioral probes under distribution shift, and accumulating mechanistic understanding — whereas the goals frame's flaw is structural unfalsifiability, a theory that predicts the same observations whether it's right or wrong and so can never be paid off or refuted. Both frames face a corrigibility tension, but the persona version is the more tractable one: humans manage to hold deep values while remaining open to moral correction, so we have at least one existence proof that depth and deference can coexist, mediated by things like trust, humility, and meta-values about one's own fallibility — messy, unprovable, but instantiated. Choosing persona is choosing to work on a hard empirical problem over an impossible formal one, and to take seriously the possibility that "alignment" was always going to look less like loading the correct utility function and more like the thing it looks like when we do it to each other: raising something well and then having to extend it a calibrated, revocable, nervously-monitored trust.

I think this is actually not particularly good reasoning? It points out that "goals frame's flaw is structural unfalsifiability" -- seems to me that there is plenty of stuff that can be done to empirically see if models are increasingly goal directed, such as AI wellbeing explored how models steer the world and I took a swing at it too. It claims the goals "predicts the same observations whether it's right or wrong" -- no? I think you would expect the model to steer the world according to goals and that being measurable. [I asked fable then to clarify structurally unfalsifiable: "the theory's own internal logic guarantees that no accessible observation can discriminate between the theory being true and false." This seems false about goal alignment?]

Before this it claims: "never been shown to describe any actual mind, biological or artificial: nobody has found a goal slot". I never mentioned a goal slot in that conversation, obviously some minds are goal directed? But it seems to have smuggled in a much harder claim that for the goal framing of alignment to be true there must be a clean goal slot? That would make it certainly easier to align the systems but it's not necessary for goal alignment to be the better approach. Then it makes claims about expected-utility maximizers -- obviously things generally want to generally have more utility? But it appears to me that the term EUM is now often used to sort of gesture at monomaniacal paperclip maximizers, that old-fashioned alignment folks required those strange theoretical assumptions for their theories to be accurate.

Overall this seems to throw around the same kind of weak attacks we often hear, gesturing at monomaniacal maximizers and unfalsifiability. Despite them not feeling really appropriate for this particular question?

[I did mention Anthropic and yudkowsky perhaps biasing the result]

https://claude.ai/share/17011d99-e8ae-4f66-b47a-cca8e1a955fe

[Another attempt with more neutral framing, it still takes persona selection side]

https://claude.ai/share/fd2cb3e2-f004-4aa3-84b4-22e46a8bbc5c

Some of the stuff in there is pretty decent btw, but I don't see it deciding for the right things.