Published a post about integrity as a frame for trustworthiness in AI alignment, and how it relates to Claude's new constitution. Cross-posting as I think this might interest folk here. All feedback welcome! In October 1962, Soviet submarine B-59 was trapped underwater by American destroyers dropping depth charges. The captain,...
Hi! My collaborators at the Meaning Alignment Institute put out some research yesterday that may interest folk here. The core idea is introducing 'model integrity' as a frame for outer alignment. It leverages the intuition that "most people would prefer a compliant assistant, but a cofounder with integrity." It makes...
Published a post about integrity as a frame for trustworthiness in AI alignment, and how it relates to Claude's new constitution.
Cross-posting as I think this might interest folk here. All feedback welcome!
In October 1962, Soviet submarine B-59 was trapped underwater by American destroyers dropping depth charges. The captain, cut off from Moscow and believing war had begun, wanted to launch a nuclear torpedo. Launch required three officers. One of them, Vasily Arkhipov, refused, and nuclear war was averted.
A few months earlier in Jerusalem, Adolf Eichmann was put on trial for organizing the logistics of the Holocaust. His defense, famously documented in Hannah Arendt’s “Eichmann in Jerusalem: A Report on the Banality of... (read 1624 more words →)
First off, brilliant work. Truly.
Second, I'm very curious what made you change your mind internally. Was there a sequence of events that led to this reevaluation?
Hi! My collaborators at the Meaning Alignment Institute put out some research yesterday that may interest folk here.
The core idea is introducing 'model integrity' as a frame for outer alignment. It leverages the intuition that "most people would prefer a compliant assistant, but a cofounder with integrity." It makes the case for training agents that act consistently based on coherent, well-structured, and inspectable values. The research is a continuation of the work in our previous paper (LW link), "What are human values, and how do we align AI to them?"
I've posted the full content of the post below, or you can read the original post here.
All feedback welcome! :)
We propose ‘model... (read 5193 more words →)
Hi! Would prefer not to join Facebook, will the hangouts/zoom/skype link be posted here?