Except that there arguably exist technologies hated even by the humans who grew in their realm. For example, nuclear weapons.^[1] Or, according to a critic, social media. Suppose that AI establishes some kind of a future where the humans can't even usefully help each other or are so spoiled by, say, AI girlfriends or boyfriends that humans find it hard to relate to each other. If the humans don't become fine with it, then your case for the future and futuristic mores would break, but the case against futuristic mores would hold.   

1. ^{^}

   While nuclear weapons are hard to separate from nuclear power plants, thermonuclear fusion has yet to produce a peaceful application. 

The question of value drift is especially strange given that we have a "meta-intuition" that moral/social values evolving and changing is good in human history. BUT, at the same time, we know from historical precedent that we ourselves will not approve of the value changes. One might attempt to square the circle here by arguing that perhaps if we were, hypothetically, able to see and evaluate future changed values, that we would in reflective equilibrium accept these new values. Sadly, from what I can gather this is just not borne out by the social science: when it comes to questions of value drift, society advances by the deaths of the old-value-havers and the maturation of a next generation with "new" values.

I feel like this is sweeping a bit under the rug. First, there's a reason why there are people who label themselves politically as "conservatives" - some people do think that our current values are just fine and should, in fact, be preserved unchanged forever! Some even want to go back to previous values (however impractical and unfeasible that tends to be; usually what it happens is that you make up some new thing that is merely a bastardised modern caricature of the old values). As far as people who instead want the values to change go, they usually have an idea of a good direction for them to change - usually they're people who are far from the median of society and so they would like society to become more like them.

Of course push far enough in the future and all ideology might seem entirely incomprehensible to us. I don't really have a clean answer for what we should think of that, except that maybe it's a big discounting factor on longtermist thinking (after all, suppose that all humans from 500,000 years hence have agreed that slavery is fine and genociding aliens is desirable - should we feel particularly proud of ensuring there's more of those people around?).

While I agree at a basic level, this also seems like a motte-and-bailey.

There is clearly a vibe that all doomers have obviously always been wrong. The author is clearly trying to push back against that vibe. I too prefer arguing at 'motte' level, but vibes (baileys) matter, and pushing back against one should not require a long airtight argument that stands up to the stronger version of the claims being made. Even though I agree the stronger version would be better, that's true for both sides of any debate.

That's comparing apples to oranges.  There are doomers and doomers.  I don't think the "doomers" predicting the Rapture or some other apocalypse are the same thing as the "doomers" predicting the moral decline of society.  The two categories overlap in many people, but they are distinct, and I think it's misleading to conflate them.  (Which is kind of a critique of the premise of the article as a whole--I would put the AI doomers in the former category, but the article only gives examples from the latter.)

The existential risk doomers historically are usually crazy, and they've never been right yet (in the context of modern society anyway--I suppose if you were an apocalypse doomer in 1300s China saying that the Mongols were going to come and wipe out your entire society you were pretty spot on), but that doesn't mean they are always wrong or totally off base.  It's completely rational to be concerned about doom from a nuclear war, for example, even though it hasn't happened yet.  Whether AI risk is crazy "Y2K/Rapture" doom or feasible "nuclear war" doom is the real debate, and this article doesn't really contribute anything to it.

What this article does a good job of is illustrating how "moral decline" doomers as opposed to "apocalypse" doomers are often proved technically correct by history.  I think what both they and this article miss is that they often see events as causes of the so-called decline, when they're actually milestones in an already-existing trend.  Legalizing gay marriage didn't cause other "degenerate" sexual behavior to be more accepted in society--we legalized gay marriage because we had already been moving away from the Puritanical sexual mores of the past towards a more liberated attitude, and this was just one more milestone in that process.  Now that's not always true--the invention of the book, and later, the smartphone absolutely did cause a devaluing of the ability to memorize and recite knowledge.  And sometimes it's a little bit of both, where an event is both a symptom of an underlying trend, and also contributes to accelerating it.  But I really like how the article acknowledges that they could be right even if "doom" as we think of it today did not occur, because the values that were important to them were lost--

Probably the ancients would see our lives as greatly impoverished in many ways downstream of the innovations they warned against. We do not recite poetry as we once used to, sing together for entertainment, roam alone as children, or dance freely in the presence of the internet's all-seeing eyes. Less sympathetic would be ancient's sadness at our sexual deviances, casual blasphemy or so on. But those were their values.

We laugh at them for being prudish for how appalled they would be at our society with homosexuality, polyamory, weird fetishes, etc. all being more or less openly discussed and acceptable, but think what it would feel like to you if in the future you saw your society trending towards one where, say, pedophilia was becoming less of a taboo?  It doesn't matter if it's right or wrong, it's the visceral response that most people have to that idea that you need to understand.  That's what it feels like to be a culturally conservative doomer watching their society experience value drift.  People today like to think that our values are somehow backed up by reality in a way that isn't true of other past or present value systems, but guess what?  That's what it feels like to have a value system.  Everyone, everywhere, in all times and all places has believed that, and the human mind excels at no other task more than coming up with rationalizations for why your values are the right ones, and opposing values are wrong.

Overall I think this article is pretty insightful about the "moral decline" type of doomers, just completely unrelated to the question of AI existential risk that brought it up in the first place.

Doomers predicted that the Y2K bug would cause massive death and destruction. They were wrong.

This seems like a misleading example of doomers being wrong (agree denotationally, disagree connotationally), since I think it's plausible that Y2K was not a big deal (to such an extent that "most people think it was a myth, hoax, or urban legend") precisely because of the mitigation efforts stemmed by the doomsayers' predictions.

I suppose the downvotes are for the low-effort AI image, but I did chuckle at the joke.