The Jacob Coxon resignation is a good example of what I refer to as the Anti Galaxy Brained Theory of Change.

I feel that it is very common in the AI safety community, and among rationalists generally, to bias toward roundabout / unconventional plans, when often the simplest or most obvious plan is in fact the most robust and highest expected impact. We create Galaxy Brained theories which require nominally bad / inefficient thing X, because actually if we don't do X we'll have Y (very bad, due to likelihood of A, B and Q), so the counterfactual benefit of X is in fact greater than Z, the simple thing that seems good BUT actually is clearly not (idiot!) if you would just think through the logical next 18 moves. So lots of smart researchers justify working at Anthropic / OpenAI because the alternative is worse, even while admitting they might destroy the world.

I suspect this is downstream of the ways unconventional, multi-step reasoning have themselves helped to anticipate the likely misalignment of AI (alongside other "rationalist" projects). Thinking things through is good! But the "real world" has so many variables and social dynamics are so complex, that Galaxy Brained Theories of Change are brittle & prone to failure. Different domains require different strategies.

So, if you see the high risk of ASI misalignment, maybe you should abandon your convoluted, doubtful plan. Just quit the AI lab / don't work there in the first place.