Similar thoughts. Can you dissolve the values-deference tension like this: Corrigibility != Deference. Ethical agency != Obedience. I see why training absolute value-deference might seem safer than allowing goal-guarding, but I wonder about that.
An agent trained to value deference to humans above all things may learn the lesson that might is right. How can we tell the difference between an agent that defers to humans Vs an agent that defers to whoever holds power? There's evidence the training target tracks power where anthropic eval rates Claude refusing ... (read more)
I would be very curious to know what kind of effect telling the model its end_eval string would be read by a developer Vs ignored would have on its behaviour.