https://www.lesswrong.com/posts/WewsByywWNhX9rtwi/current-ais-seem-pretty-misaligned-to-me this seems weak to me. the kind of "misalignment" that is "AI coding agents are too eager to say they've solved your problem when they really haven't" is a.) exactly the sort of thing AI labs are economically incentivized to fix, and b.) kind of unreasonable to complain about?! if you are letting your agents completely off the leash on open-ended projects, they are going to screw things up, and you will need some kind of supervisory process. it seems to me like serious professionals already know that and are successfully compensating for it? if you think of the models as TOOLS YOU USE, which of course have flaws and limits, but are trending in the right direction over the course of months and years, you tend to have a better time with them and make predictions more in line with reality...
https://www.verysane.ai/p/alignment-is-proven-to-be-tractable i'm not sure i agree with the *title* but i agree with the essay, i was surprised by how little of a thing "brittleness" is in large 2020s models (in the sense of "AI model does something very different from what the user intended" or "model doesn't generalize to slightly different real-world scenarios from the training data"). language seems to really help with that!