I think that, sometimes people do the wrong thing because they underestimate their own power.
Examples: (in no particular order)
1. AI safety researchers making capabilities progress, thinking it doesn't matter because "it was so obvious" or "someone else where going to do it anyway".
Sidenote on the above. About a decade ago, when there where approximately no AI Safety training programs yet, the standard advice for aspiring AI Safety researchers, from within EA, was to go do an ML (capabilities) PhD, to gain skills. The reasoning was that there where so many people in capabilities anyway, that any contributions you made there would not matter.
1. Quote (from memory) from this video about the battle of the board https://youtu.be/_eYTkvZqbnQ?si=_GfUCXjJtpBgXH-j "She was looking to see where the wind was blowing and didn't realize she was the wind" (about the interim CEO just after Sam Alman was fired). She took Sam's side.
2. Some months ago I listened to Behind the Shock Machine: The Untold Story of the Notorious Milgram Psychology Experiments, about the Milgram Experiment (https://en.wikipedia.org/wiki/Milgram_experiment). There are a few reasons many people went all the way, but one re-occurring reason semes to have been, not realizing that stopping was an option.
3. Lot's of people in AI coms, thinking that they have to stay inside the Overton window, instead of trying to shift the Overton widow.
4. Possibly (but there could be other reasons people want to take these actions):
* * AI companies thinking they have to race
* * Some people thinking US has to race against China
* * All the powerful people who didn't stop Google's deal with the Department of War
Acknowledgment: I though of this because of Richard Ngo's talk and following discussion at Iliad, about all the ways AI safety people have screwed up and helped capabilities. I think this theory explains some of that, but not all.
