“Alignment”, then, is no longer really a technical problem, at least not in the same sense as it was in 2020. It is largely a political problem
I wouldn't say this. I think that, like humans, Opus has perhaps learned some deeper and more abstract form of benevolence. We still don't know whether this breaks down, and if we accept these models as robustly aligned and follow through on the logical next step (deploy them everywhere), we would not be safe. We do have basic evals to determine how LLMs act when placed in extreme scenarios; we also can try to spot ... (read more)
How will they benchmark the models? You could say the deployment companies will do that, but what about sensitive capabilities? You don't want to hand a cyber-capable and misaligned model to the deployment company that wants to "move fast and break things," and how can you ensure you won't do that without some kind of testing? The government will do that, perhaps?
Then are they allowed to benchmark but not use it on research?... (read more)