moridinamael's Shortform

moridinamael

This is a special post for quick takes by moridinamael. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I've thought for some time that we should be at least trying to use AI to do our alignment homework for us. To that end, I took an original corrigibility concept of mine and ran it through a few dozen iterations of refinement, improvement and expansion with all the frontier models.

The core idea is essentially a corrigibility-first decision theory. Here is the link, if you want to look at it: https://github.com/moridinamael/moadt I also put the core documents in a Notebook LM project so that people can "chat with the document" directly: https://notebooklm.google.com/notebook/e7dd1eb8-01d1-4fcb-8c72-427a15049790

The GitHub repo contains a ~200 page detailed description with worked examples, a shorter core argument document, and a working Python library for building and evaluating your own test problems.

Having done all this, I am not sure what the best way is to present this material. Putting in a GitHub repo makes it available, but nobody knows it exists. I leaned toward writing a summary Less Wrong about the effort, but there is such a strong immune reaction to "AI slop" these days that I don't relish the thought of the response I would likely receive.

So I'm posting this "quick take" to see if anybody has any better ideas for how I could move forward with this.

4StanislavKrym9mo

@Max Harms How similar is the idea in the document to solving the problems of CAST?

Thanks for tagging me. I took a look, and am glad for Matt's efforts in trying clever, new approaches.

My main take is that this operates on a pretty different level than CAST, and I would personally be hesitant to say it produces corrigibility. (In Eliezer-lingo I would say "it doesn't engage with the hard problem".) I'd be more inclined to say it produces an agent that is extremely deferent. (My sense, by contrast, is that truly corrigible agents proactively surface important facts to their principal, which is not something I see coming from MOADT.) This ... (read more)

4habryka9mo

Yes, I mean that's why I would want the chat to be somehow available on-site (or maybe for there to be some way to connect a local LLM to the context through us), and for the author to be able to read the exchanges people have with the model (and of course to still have a comment section and voting right below it and stuff).