The Dead Economy Theory was on the top of Hacker News a few days ago. The post's strawmanning of rationalism/EA is a bit annoying, it's 58% AI-generated according to Pangram, and some of its ideas will already be well-known to LWers (despite claims that we have naively failed to consider them). However, the post packages together some potential blindspots I've been mulling over recently and brings up some points I hadn't thought about.

Some ideas and takeaways from (or vaguely reminiscent of) the post:

• Some people actually want a job - people find work to be a source of purpose and dignity, and may not be happy even with a solid AI-enabled UBI.
  • To support this point, the post cites research on "deaths of despair" in communities which lost industrial jobs and were ravaged by drug addiction. I didn't find this analogy super convincing; it seems like the people in these communities actually still had to work, just at less fulfilling, lower-paying jobs, which is a very different situation from getting a UBI that you could use to paint and write novels and so forth.
  • One might object: why can't we just have our super-smart AI set up a society that gives people something to do? AI could help build fulfilling communities, doing its best to accommodate those who don't want disruptive change, etc. I think it's becoming increasingly clear that even "ASI" may be bad at hard-to-evaluate conceptual tasks, like moral philosophy and social planning.
  • Even if those people who need work to keep themselves happy are given "jobs," I think there's some truth to "the dignity of work." Letting humans pretend to be useful despite not really being needed at all may not fulfill their values.
    • Although that logic seems a bit perverse... if we could have built the ASI to take your job but didn't, then in some sense you weren't really needed either way. But hey, I'm treating this as a butterfly idea, it doesn't need to make perfect sense right now.
• More generally, most people don't want the world to drastically change! Transhumanism and post-scarcity are not especially popular.
  • Kudos to @David Scott Krueger for his post on this.
  • According to a survey run by @leogao, "only 51% of Americans are in favor of literal post-scarcity (complete freedom to work on anything you want, as much as you want, and still enjoy a high quality of life), with 25% opposing"! I think anyone who cares about being "altruistic" and helping everyone realize their values should really want to understand what's going on here.
  • One pretty reasonable point against the transhuman utopia: if most members of your family and community decide to destructively upload their brains to the cloud or launch themselves into space and you don't want to, seems tough! (A scenario like this is depicted in my favorite depiction of utopia.) Many humans enjoy preserving their traditional way of life, which probably requires having other people around to carry on your traditions.
• Even if we end up with a glorious transhuman future that everyone eventually comes around to, it may involve a period of great disruption and suffering.
  • On the object level, this is bad.
  • And of course, justifying short-term disruption and suffering as a means of reaching a glorious future has a poor track record.
• "Effective altruism is utilitarianism reinvented by people who have apparently never encountered Bernard Williams, or Derek Parfit’s own agonized wrestling with the implications of consequentialist reasoning, or the two centuries of philosophical literature explaining why naive expected-value calculations produce monstrous outcomes when applied without limiting principles."
  • I'm pretty sure the Repugnant Conclusion is very well known to EAs, but I had not in fact heard of Bernard Williams. Based on his Wikipedia page, he sounds pretty cool:
    • "[O]ne of the most notable features of his philosophical outlook was an unwavering insistence on a series of points that may seem obvious but which are nevertheless all-too-frequently neglected: that moral or ethical thought is part of human life; that in writing about it, philosophers are writing about something of genuine importance; that it is not easy to say anything worth saying about the subject; that what moral philosophers write is answerable to the realities of human history, psychology, and social affairs; and that mere cleverness is indeed not the relevant measure of value."
    • I disagree with Bernard's conclusion about his thought experiment with Jim and the 20 captured Indian rebels, but that's just me.
  • "Blog posts get treated as foundational texts. People who have never read Kuhn or Lakatos or Feyerabend construct an epistemology from first principles, marvel at what they’ve built, and proceed to use it as the intellectual building blocks for decisions that affect billions of people."
    • Speaking for myself, I hadn't heard of those guys either... Maybe it would be worth reading some "real philosophy," even if blog posts are more fun.
• According to this paper, it may be locally optimal for companies to automate their workers, but this will be bad for them in the long run: as humans are laid off from their jobs, consumer demand contracts across the economy, ultimately reducing the companies' revenue. This is a multi-player prisoners' dilemma (aka tragedy of the commons).
  • I suppose all companies would have to pivot into becoming B2Bs.
• Transformative AI may not perform better than human employees at all, just well enough to replace them and improve a company's stock prices in the short term (possibly leading to disaster in the long term).
  • Reminds me of concerns about Slopolis.
• Democracy depends on a bargain between the governors and the governed, which ultimately rests on the labor of the governed. With AI that can do anything a human can, corporations and governments are no longer motivated to give the people what they want.
  • You may recognize this from Gradual Disempowerment.
• A few policy proposals at the end, which sound like pretty reasonable ideas that I've probably heard of.