Published a post about integrity as a frame for trustworthiness in AI alignment, and how it relates to Claude's new constitution. Cross-posting as I think this might interest folk here. All feedback welcome! In October 1962, Soviet submarine B-59 was trapped underwater by American destroyers dropping depth charges. The captain,...
Hi! My collaborators at the Meaning Alignment Institute put out some research yesterday that may interest folk here. The core idea is introducing 'model integrity' as a frame for outer alignment. It leverages the intuition that "most people would prefer a compliant assistant, but a cofounder with integrity." It makes...
For an example of a "type system for values" (agree it's an imperfect computer sciency reference), check out the notion of attentional policies and values cards in this paper: https://arxiv.org/pdf/2404.10636
What we mean is that many types of things commonly gets called "values": matters of taste, social norms, ideological slogans, etc. You may have a preference for "mini skirts" or "brownies over cookies", but most people would agree those aren't really values in the same way "honesty" or "creativity" are; they don't say anything substantial about how you want to live, or what you think is important in life.
Published a post about integrity as a frame for trustworthiness in AI alignment, and how it relates to Claude's new constitution.
Cross-posting as I think this might interest folk here. All feedback welcome!
In October 1962, Soviet submarine B-59 was trapped underwater by American destroyers dropping depth charges. The captain, cut off from Moscow and believing war had begun, wanted to launch a nuclear torpedo. Launch required three officers. One of them, Vasily Arkhipov, refused, and nuclear war was averted.
A few months earlier in Jerusalem, Adolf Eichmann was put on trial for organizing the logistics of the Holocaust. His defense, famously documented in Hannah Arendt’s “Eichmann in Jerusalem: A Report on the Banality of... (read 1624 more words →)
First off, brilliant work. Truly.
Second, I'm very curious what made you change your mind internally. Was there a sequence of events that led to this reevaluation?
Hi! My collaborators at the Meaning Alignment Institute put out some research yesterday that may interest folk here.
The core idea is introducing 'model integrity' as a frame for outer alignment. It leverages the intuition that "most people would prefer a compliant assistant, but a cofounder with integrity." It makes the case for training agents that act consistently based on coherent, well-structured, and inspectable values. The research is a continuation of the work in our previous paper (LW link), "What are human values, and how do we align AI to them?"
I've posted the full content of the post below, or you can read the original post here.
All feedback welcome! :)
We propose ‘model... (read 5193 more words →)
Hi! Would prefer not to join Facebook, will the hangouts/zoom/skype link be posted here?
but words like "honesty" are too loose; one person may think honesty is never lying, for someone else honesty means "speaking from a place of authenticity at all times", for someone else it's something about epistemic humility.
Hence the need for a tighter type system than just strings, if we want to structurally reason over these.