A common point of contention in the "sharp left turn" vs. "iterative alignment improvement" debate is about whether we will see evidence that a sharp left turn will happen.
One piece of evidence is that in the "sharp left turn" world, AI can (probably*) tell when it's in an eval. The AI needs eval awareness to fool us into thinking it's aligned. If we don't see eval awareness, that's evidence that either there won't be a sharp left turn, or it will be far away. If we do see eval awareness, it's evidence that a sharp left turn will happen.
Caveats:
This is only weak evidence, because we could also see eval awareness without a sharp left turn.
On the other side, it could be that our evals are just not up to the task of detecting misalignment, even if the AI doesn't know it's being evaluated.
I haven't really thought through this line of reasoning, but it feels like there might be something of value here.
*It could be that eval awareness emerges suddenly right when the sharp left turn happens, but it's more likely that it emerges at a different time.
We've gotten plenty of left turns, whether sharp or no. All you have to do is make a more powerful version of what we have today, and we already know it would betray us in important ways. I think it's more likely that a new paradigm will emerge, and who's to say what the safety properties of that will be, other than "probably not what we want them to be by default," especially if it's part of an uncontrolled autonomous R&D loop and/or operating outside of human oversight.
* Naively, I would say every impressive capability you observe without a sharp left turn is weak evidence against it. At this point, both hypotheses explain the world about equally well, whereas the SLT hypothesis is more complex.
* Arguably, eval awareness is actually a pretty mundane LLM ability, because putting text into a milieu is something LLMs are superhuman at already. I.e., "ah, clearly a Wikipedia article / paper / Reddit post" entails a capability like "ah, clearly something made by these METR people."
* I'd say a sharp left turn is more likely to happen through something like a meta-skill, and less likely to happen by some sort of scheming.
* I think the plot of intelligence vs. years since LUCA is still one of the strongest arguments here, and something like language or whatever else happened to the hominids is strong evidence for the existence of such a meta-skill.
* Current efforts put more selection pressure towards "produce impressive-looking results" vs. "appear aligned to humans" than was assumed in optimistic "AI in a box" scenarios.[1] I.e. for lying to emerge, there must be a strong cost for speaking the truth in the first place.
1. ^
[...]
From page 118 of "Superintelligence". I don't think e.g. "test results are as good as they could be" describes the METR Frontier Report, and yet we've let the models in questions out of their box already. (In general I think the whole page still reads as very prescient)