I think type-1 research is most useful, type-3 is second best, and type-2 is least useful. Here's why I think so.
All of your arguments seem to be about importance. Tractability and neglectedness matter too. (Unless your argument is that type 2 research will be of literally zero use for aligning superintelligent AI.) In particular, you seem dismissive of:
It's easier to learn all the prerequisites for type-2 research and to actually do it.
Even if you are perfectly altruistic and not limited by money, other people, reputation, etc this is a legitimate reason to focus on type 2 research from a perspective of minimizing x-risk.
At some point, humanity will create a superintelligent AI, unless we go extinct before. When that happens, we won't be making important decisions anymore. Instead, the AI will.
The AI will be making important decisions long before it becomes near-omnipotent, as you put it. In particular, it should be doing all the work of aligning future AI systems well before it is near-omnipotent.
(However, you still need a pretty strong guarantee about that AI system, such that when it aligns a future AI system, that future AI system remains aligned with us, so I think overall the intuition is right.)
Human-level AI might be alignable using hacky, empirical testing, engineering, and "good enough" alignment.
All alignment is "good enough" alignment, there is no such thing as "perfect" alignment except in idealized theory. All you get is more or less confidence in the AI system you're building. (I say this not to be pedantic, because I legitimately don't know what your threshold is for dismissing "hacky" alignment, or what you mean when you say it won't work on superintelligent AI systems, or what would count as "not hacky". I may or may not agree with you depending on the answer.)
Superintelligent AI is an extremely powerful optimization process. Hence, if it's unaligned even a little, it'll be catastrophic.
I agree with the intuition overall, but it isn't a particularly strong intuition. For example, any other human is at least a little unaligned with me, but there are at least some humans where I'd feel okay making them God (in the sense that in expectation I think the world would be better moment-to-moment by my values than it is today moment-to-moment; this could be an existential catastrophe because we don't control the future, but it isn't extinction).
I don't see why it'll be easier to work on the alignment of superintelligent AI in the future rather than now, so we'd better start now.
We'll have a better idea of how superintelligent AI will be built in the future.
There are too many confusing things about superintelligent AI alignment [...] Hence, deconfusion is very important.
I agree with the intuition in general.
Note though it's quite possible that some things we're confused about are also simply irrelevant to the thing we care about. (I would claim this of embedded agency with not much confidence.)
To clarify, when said "performs well", I did not mean "learns human values well", nor did I have any sort of scoring rule in mind. I intended to mean that the algorithm learns patterns which are actually present in the world - much like earlier when I talked about "the human-labelling-algorithm 'working correctly'".
Ah well. I'll probably argue with you more about this elsewhere, then :)
I want to work on technical AI alignment research. I am trying to choose an AI alignment area to get into. To do that, I want to understand the big picture of how to make AI go well through technical research. This post contains my views on this question. More precisely, I list types of AI safety research and their usefulness.
Probably, many things in this post are wrong. I want to become less wrong. The main reason why I am posting this is to get feedback. So, please
Three fuzzy clusters of AI alignment research
Their usefulness
I think type-1 research is most useful, type-3 is second best, and type-2 is least useful. Here's why I think so.
Many people seem to work on type-2 research. Probably, many of them have thought about it and decided that it's better. This a reason to think that I am wrong. However, I think there are other reasons people may choose to work on type-2 research, such as:
Also, I have a feeling that approximately in the last 3 years, a small paradigm shift has happened. People interested in AI alignment started talking less about superintelligence, singletons, AGI in the abstract, recursive self-improvement and fast takeoff. Instead, they talk more about neural networks, slow takeoff, and smaller, less weird, and less powerful AI. They might be right, and this is a reason to be slightly more enthusiastic about type-2 research. However, I still think that the old paradigm, perhaps minus fast takeoff, is more useful.