The lack of falsification criteria for AGI (unresearched rant)
Situation: Lots if people are talking about AGI, and AGI safety but nobody can point to one.
This is a Serious Problem, and a sign that you are confused.
Historically when these tests are passed the goalposts are shifted (Turning test was passed by fooling humans, which is incredibly subjective and relatively easy).
Solution:
A robust and scalable test of abstract cognitive ability.
A test that could be passed by a friendly AI in such a way as to communicate co-operative intent, without all the humans freaking out.
Would anyone be interested in such a test so that we can detect the subject of our study?
Becoming capable of building such a test is essentially the entire field of AI alignment. (yes, we don't have the ability to build such a test and that's bad, but the difficulty lives in the territory. MIRI's previously stated goal were specifically to become less confused)