I feel like outer alignment - in the sense of building training/learning systems that incentivize good behavior - is under-studied, because people think the analogue of "I'm a great communicator, people just don't understand me!"
"Outer alignment is easy, the AI just keeps finding perverse solutions!"
Hand-writing values, or hand-writing a reward function, isn't good outer alignment, because we don't know how to incentivize good AI behavior that way.
So maybe we can write down a bunch of correlates of human values in the present environment, and a way to leverage an AI's learned world-model to turn those correlates into guesses about good values, and heuristics that will lead an AI to generalize well, and a learning procedure that leverages human feedback to fix issues with the first three things. But trying to optimize over such a mish-mash will predictably lead to some pieces of the system being incentivized to subvert the whole. And again, doing a thing you know will incentivize bad behavior isn't good outer alignment.
Maybe it would solve problems with complicated value learning schemes if the AI understood what was going on and bought in. Not just in the sense of "has been trained to produce good-sounding text about its training process, while still being RLed to subvert it" but in the sense of having an abstract model of the training process and its "spirit," and using that model when calculating learning updates, analogous to how you might avoid myopic overfitting in a non-stationary environment by leveraging a model of how the environment is expected to change.