Much of the current literature about value alignment centers on purported reasons to expect that certain problems will require solution, or be difficult, or be more difficult than some people seem to expect. The subject of this page's approval rating is this practice, considered as a policy or methodology.
...most of value alignment theory, so try to pick 3 cases that illustrate the point in different ways. Pick from Context Change?
For: it's sometimes possible to strongly foresee a difficulty coming in a case where you've observed naive respondents to seem to think that no difficulty exists, and in cases where the development trajectory of the agent seems to imply a potential Treacherous Turn. if there's even one real Treacherous Turn out of all the cases that have been argued, then the point carries that past a certain point, you have to see the bullet coming before it actually hits you. the theoretical analysis suggests really strongly that blindly forging ahead 'experimentally' will be fatal. someone with such a strong commitment to experimentalism that they want to ignore this theoretical analysis... it's not clear what we can say to them, except maybe to appeal to the normative principle of not predictably destroying the world in cases where it seems like we could have done better.
Against: no real arguments against in the actual literature, but it would be surprising if somebody didn't claim that the foreseeable difficulties program was too pessimistic, or inevitably ungrounded from reality and productive only of bad ideas even when refuted, etcetera.
primary reply: look, dammit, people actually are way too optimistic about FAI, we have them on the record (find 3 prestigious examples) and it's hard to see how humanity could avoid walking directly into the whirling razor blades without better foresight of difficulty. one potential strategy is enough academic respect and consensus on enough really obvious foreseeable difficulties that the people claiming it will all be easy are actually asked to explain why the foreseeable difficulty consensus is wrong, and if they can't explain that well, they lose respect.
Will interact with the psychoanalysis of antitheorism.