I think I have an interesting new research direction: Aligning Committees. Make ensembles of agents and have them act together as only one agent in the world with some protocol for how they combine preferences. Main Motivating Construction: Given a target consequence T, we construct a committee out of 3...
Work supported by a Manifund Grant titled Alignment is hard. While many people have made the claim that the alignment problem is hard in an engineering sense, this paper makes the argument that the alignment problem is impossible in at least one case in a theoretical computer science sense. The...
I think I have an interesting new research direction: Aligning Committees.
Make ensembles of agents and have them act together as only one agent in the world with some protocol for how they combine preferences.
Main Motivating Construction:
Given a target consequence T, we construct a committee out of 3 expected utility maximizing agents: Planner, Wanter, and Unwanter (which have different utility functions). Planner proposes a plan and then Wanter and Unwanter either signoff or veto said plan. A plan P which gets both to sign off is executed, otherwise the agent does the safe but useless null action
The Utility Functions
Then as the committee gets smarter and more powerful I believe this is a mild optimizer.
Questions:
1.... (read more)
Thanks for the feedback!
On your first point, correct the thing shown to be uncomputable is testing alignment. And yes, uncomputability is a worst case claim. Would it be clearer to call the paper an uncomputable alignment TEST as opposed to an uncomputable alignment PROBLEM? (Im considering editing the paper before submitting it to a journal)
Detering a few would be nice. More realistically, proofs in this vain could help convince regulators to ignore opaque box makers claims about detecting an agent's alignment.
Work supported by a Manifund Grant titled Alignment is hard.
While many people have made the claim that the alignment problem is hard in an engineering sense, this paper makes the argument that the alignment problem is impossible in at least one case in a theoretical computer science sense. The argument being formalized is that if we can't prove a program will loop forever, we can't prove an agent will care about us forever. More Formally, when the agent's environment can be modeled with discrete time, the agent's architecture is agentically-turing complete and the agent's code is immutable, testing the agent's alignment is CoRE-Hard if the alignment schema is Demon-having, angel-having, universally-betrayal-sensitive, perfect... (read more)
I understand this bounty is roughly closed, but the page is still listed as bounty active and my attempt took me only a couple hours.
Problem 2 is solvable when there are simple restrictions on the entries chosen.
Restriction 1: No diagonal entries need to be correct. Solution: Let C = AAT For each entry c_ij we construct the matrix with only c_ii, c_ij, c_ji , c_jj and all else zero. Clearly this matrix is constructible in O(n) work. So all entries can be constructed in this way. Note if c_ij and c_ji are requested, only construct one matrix for the pair. Repeating this for all m does O(nm) work. We then sum all such matricies to get... (read more)
New location of previous file on github: https://github.com/Alexhb61/Alignment/blob/main/committees/Control_by_committee.md
The main criticism I imagined of this piece was the disassembly risk.
By adding a latter step to this outer alignment approach (transforming a committee into a modified learning problem), I believe I have a much stronger angle on outer alignment.
I will continue to write about it here:
https://github.com/Alexhb61/Alignment/blob/main/committees/outer_alignment.md