On Idealized Agency
There's a story I've heard a few times in agent foundations, that involves starting with a very idealized theory of agency, like AIXI, IB, or UDT, in which we can achieve strong, provable safety properties,[1] and gradually making it more realistic, until it can eventually be implemented, or at least...
Oct 2110