Avoiding what you suggested is why private conversations are an advantage. I think you misunderstood the essay, unless I'm misunderstanding your response.

> I think it's cultural and goes back to the rise of Christianity.

This seems testable with a cross-cultural analysis. Not just the pre-Christian Greek stories that Garrett mentioned, but Chinese, Japanese, Indian, and Middle Eastern cultures should have plenty of non-Christian stories.

This is pretty cool. As for Opus, could you just use it for "free" by running it in Claude Code and use your account's built-in usage limits.

Edit: That might also work for gemini-cli and 2.5 Pro.

This is a great story and the animation is also great. Good work everyone!

Oh yeah, I also find that annoying.

> If we were to put a number on how likely extinction is in the absence of an aggressive near-term policy response, MIRI’s research leadership would give one upward of 90%.

This is what I interpreted as implying p(doom) > 90%, but it's clearly a misreading to assume that someone advocating for "an aggressive near-term policy response" believes that it has a ~0% chance of happening.

I am in camp 2, but will try to refine my argument more before writing it down.

This is great. I recognize that this is almost certainly related to the book "If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All", which I have preordered, but as a standalone piece I feel estimating p(doom) > 90 and dismissing alignment-by-default without an argument is too aggressive.

Yeah, I don't think I read this when it came out, but I'm happy to read it now.

https://www.lesswrong.com/posts/yew6zFWAKG4AGs3Wk/foom-and-doom-1-brain-in-a-box-in-a-basement (and the sequel) seem highly related.

If you had a solution to alignment, building a Night-Watchman ASI would be decent, but that is a massive thing to assume. At the point where you could build this, it might be better to just build an ASI with the goal of maximizing flourishing.