To the extent we believe more advanced training and control techniques will lead to alignment of agents capable enough to strategically make successor agents -- and be able to solve inner alignment as a convergent instrumental goal -- we must also consider that inner alignment for successor systems can be...
To be clear from the outset: I don't believe Eliezer Yudkowsky would make any of the mistakes I describe in this post when playing 2-4-6-alike games. I've performed well in novel pattern-finding challenges, but not necessarily better than I imagine Yudkowsky would. I'm confident he correctly understands and would emphatically...
To the extent we believe more advanced training and control techniques will lead to alignment of agents capable enough to strategically make successor agents -- and be able to solve inner alignment as a convergent instrumental goal -- we must also consider that inner alignment for successor systems can be solved much easier than for humans, as the prior AIs can be embedded in the successor. The entire (likely much smaller) prior model can be run many times more than the successor model, to help MCTS whatever plans it's considering in the context of the goals of the designer model.
I've been thinking about which parts of AI 2027 are the weakest, and... (read 410 more words →)
To be clear from the outset: I don't believe Eliezer Yudkowsky would make any of the mistakes I describe in this post when playing 2-4-6-alike games. I've performed well in novel pattern-finding challenges, but not necessarily better than I imagine Yudkowsky would. I'm confident he correctly understands and would emphatically agree with everything I'm about to say. My aim here is to be pedantically precise about how this idea is presented to the public because I believe the fundamental difficulty people face when tackling the 2-4-6 game is both simple to describe correctly and valuable to consider.
Background
Let's review some required reading. In the 2-4-6 game, participants are asked to characterize a pattern... (read 488 more words →)
Indeed. I am tempted to edit to say every guess (instead of every answer), to resolve this ambiguity, but I suspect it's well understood as-is?