This is a special post for quick takes by RussellThor. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Self modelling in NN https://arxiv.org/pdf/2407.10188 Is this good news for mech interpretability? If the model makes it easily predictable, then that really seems to limit the possibilities for deceptive alignment
It makes it easier, but consider this: The human brain also does this—when we conform to expectations, we make ourselves more predictable and model ourselves. But this also doesn't prevent deception. People still lie and some of the deception is pushed into the subconscious.
3RussellThor
Sure it doesn't prevent a deceptive model being made, but if AI engineers made NN with such self awareness at all levels from the ground up, that wouldn't happen in their models. The encouraging thing if it holds up is that there is little to no "alignment tax" to make the models understandable - they are also better.