"Actual LessWrong readers also sometimes ask me how I deal emotionally with the end of the world.

I suspect a more precise answer may not help.  But Raymond Arnold thinks I should say it, so I will say it.

I say again, I don't actually think my answer is going to help."

It's not a common trope, certainly, but if it is one, it's also one that Eliezer is happy to play out. (And there are lots of good tropes that people play out which they shouldn't avoid just because they are tropes - like falling in love, or being a good friend to others when they are sad, or being a conscientious ethical objector, or being someone who can let go of things while having fun, etc.)

Agree that it's not just about being dramatic / making the problem about you. But that was only one of the points Eliezer made about why people could fail at this in ways that are worth trying to fix. And in your case, yes, dealing with the excessive anxiety seems helpful.

Good question, good overview!

Minor note on the last point, which seems like a good idea, but human oversight failures take a number of forms. The proposed type of red-teaming probably catches a lot of them, but will focus on easy to operationalize / expected failure modes, and ignores the institutional incentives that will make oversight fail even when it could succeed, including unwillingness to respond due to liability concerns, slow response to correctly identified failures. (See our paper and poster at AIGOV 2026 at AAAI.)

Is this anxiety in the typical form of making it harder for you to do other things? Because yes, we all agree that it's very bad outcome, but a critical point of the post is that you might want to consider ways to not do the thing that makes your life worse and doesn't help.

In retrospect, the post holds up well - it's not a brilliant insight, but I've referred back to it, and per the comments, so have at least some others. 

I would love for there to be more attention to practical rationality techniques and useful strategies, not just on (critically important) object-level concerns, and hope that more work in that direction is encouraged.

Designing funding institutions that scale to handle 10x to 100x the number dollars, and also the number of "principals" (since I expect, as opposed to OP having a single Dustin, Anthropic will produce something like 50-100 folks with 10Ms-100Ms to donate)

Seems plausible that a decent part of Coefficient Giving's new strategy exactly supports this model.

I'd be especially interested in Angel investors funding early stage EA-aligned high risk moonshots that will need Series A funding in a year if successful - but it likely requires risk neutral, low regret funders, or people funding an entire portfolio, both of which are rare.

I agree that many of the worldviews being promoted are unrealistic - expecting companies in the current competitive race conditions would be a competitive disadvantage.

But I also think that there are worlds where Anthropic or OpenAI as companies cared enough to ensure that they can be trusted to keep their promises. And there are industries (financial auditing, many safety critical industries,) where this is already the case - where companies know that their reputation as careful and honest actors is critical to their success. In those industries, breaking the trust is a quick path to bankruptcy.

Clearly, the need for anything like that type of trustworthiness is not true in the AI industry. Moreover, coordinating a change in the status quo might be infeasible. So again, yes, this is an unrealistic standard.

However, I would argue that high-trust another viable equilibrium, one where key firms were viewed as trustworthy enough that anyone using less-trustworthy competitors would be seen as deeply irresponsible. Instead, we have a world stuck in the low-trust competition in AI, a world where everyone agrees that uploading sensitive material to an LLM is a breach of trust, and uploading patient information is a breach of confidentiality. The only reason to trust the firms is that they likely won't care or check, and certainly not that they can be trusted not to do so. And  they are right to say that the firms have not made themselves trustworthy enough for such uses - and that is part of the reason the firms are not trying to rigorously prove themselves trustworthy.

And if AI is going to control the future, as seems increasingly likely, I'm very frustrated that attempts to move towards actually being able to trust AI companies are, as you said, "based on unrealistic and naive world views."

Regardless of whether you think the company is net positive, or working for it is valuable, are you willing to explicitly disagree with the claim that as an entity, the company cannot be trusted to reliably fulfill all the safety and political claims which it makes, or has made? (Not as in inviolably never doing anything different despite changes, but in the same sense that you trust a person not to break a promise without. e.g., explaining to those it was made to about why it thinks the original promise isn't binding, or why the specific action isn't breaking their trust.)

I think that an explicit answer to this question would be more valuable than the reasonable caveats given.