This is a special post for quick takes by TurnTrout. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
When talking about "self-fulfilling misalignment", "hyperstition" is a fun name but not a good name which actually describes the concept to a new listener. (In this sense, the name has the same problem as "shard theory" --- cool but not descriptive unless you already know the idea.) As a matter of discourse health, I think people should use "self-fulfilling {misalignment, alignment, ...}" instead.
Hyperstition ≈ self-fulfilling prophecy, with the caveat that the word is somewhat connected to other ideas associated with Nick Land (technological accelerationism, Capital as AI, AI that retrocausally brings itself into existence...).
I think the problem of "may suggest a potentially suboptimal intervention" is less severe than "isn't descriptive." Plus, I think we're going to see "self-fulfilling alignment" be upsampled after the recent positive results. :)
2Jozdien
Yeah, that's fair. I guess my views on this are stronger because I think data filtering might be potentially negative rather than potentially sub-optimal.