"When you were partying, I studied the blade. When you were having premarital sex, I mastered the blockchain. While you wasted your days at the gym in pursuit of vanity, I cultivated inner strength. And now that the world is on fire and the barbarians at the gate, you have the audacity to come to me for help?"
I would encourage you to test your assumptions about others' capacity for interiority, fear, hope, paralysis, confusion, commitment, and so on, compared with your own. In your resignation - "humanity is a failure," "others don't care," "their ignorance, apathy, and meanness" - I think you make the mistake of putting the cart before the horse: you have separated yourself from others, derided them, and now (incorrectly, imo) complain they don't care about what you care about.
Are you sure this resignation, veiled as a knowing sorrow, is justified? Are you sure that your enemy is really 'other people'? Those 'other people' who vocally express widespread fear about AI capabilities, or oppose datacenter construction (however misguided)? That 'they' are simply 'doing nothing'? That your enemy might not instead be those in power, those with control, those who regulate - with whom you actually align yourself here?
Put simply, are you sure that you are not just projecting your own defeatism onto the world at large?
How are your actions different as a result of all this? Does worrying make some morally superior to less worried people if they both have no theory of change?
I now focus much less on the intended outcome (which seems unrealistic) and much more on the process (doing the right thing), and I think it is useful.
Sometimes, just sometimes, I thought that I was escaping the real world and real work under the excuse of caring about something more important, which wasn't, in fact, more important.
And yet, now that I live through the takeoff, no one cares.
I think that the society simply has yet to understand the extent of the AIs' capabilities. If a counterfactual world had a law of physics preventing the minds of unevolved origin from being capable, then the issues would've dissolved. Like, if delivering cargo to other stars requires about a hundred years, then the humans would be justified in not caring about intergalactic endowment because the colonists wouldn't be able to regularly communicate with the Earth and would quickly become independent.
A hundred years is nothing. The matter within a galaxy cluster remains gravitationally bound even after the more remote matter becomes inaccessible due to accelerating expansion of the universe in about 150 billion years. There's plenty of time to communicate within a galaxy cluster, especially in the long future after that happens. We have until then to colonize the other galaxy clusters within the reachable universe (assuming takeoff-capable aliens are sufficiently rare that some of these galaxy clusters would otherwise remain unintelligent). The Virgo Cluster (gravitationally bound with Earth) has 1.5 thousand galaxies, while there are about 4 billion reachable galaxies in total outside of it. Even the rest of the Virgo Supercluster is lost if not colonized in time.
Within a gravitationally bound galaxy cluster, there might be 1e34+ years until proton decay becomes a problem, or 1e100+ years until the larger black holes evaporate via Hawking radiation, depending on whether proton decay is a real thing, and if there's a way to keep compute going using an existing black hole when everything else is mostly gone, or maybe some other ways of engineering more serial compute out of the starting conditions.
if delivering cargo to other stars requires about a hundred years, then the humans would be justified in not caring about intergalactic endowment because the colonists wouldn't be able to regularly communicate with the Earth and would quickly become independent
Being independent doesn't imply indifference. Cargo is not communication. The future of humanity is not humans in the narrow sense. The moral status of unintelligent galaxy clusters is very different from that of the galaxy clusters that were given intelligent life.
Technological maturity (or abundant star-scale superintelligence [1] ) are out-of-distribution for the common sense about what human-scale difficulty of transportation implies or not. Some decontextualization and exploratory engineering become essential.
Edit: This particular thing is outside StanislavKrym's constraint of "only minds of evolved origin", but I think it makes a useful illustration for why the rest of the arguments go through even under the constraint, see the other comment. ↩︎
I very explicitly included the counterfactual assumption that there was "a law of physics preventing the minds of unevolved origin from being capable", causing the only capable minds to be descendants of humans, animals or aliens. Which is the layman's mindset, as I explained in my linked comment. As for the hundred of years, I mean that this is the duration of the journey to other stars, not the rate at which technology matures...
There's still some asymptote of technological maturity, even if all possible intelligent life is somehow human-level. And this wasn't your constraint, you didn't rule out more intelligent life, especially if a civilization is guiding its continued evolution over billions of years ("of unevolved origin" is ambiguous on this point). The arguments still apply, it's unlikely that superintelligence is necessary if you have billions of years to figure things out (see Figure 5 in the paper [1] I cited). Progress in engineering makes the project possible, progress in governance enables wider enduring coordination, progress in philosophy clarifies what specifically should be done. Even biological immortality (let alone biological re-instantiation from distributed backups) makes "100 years" lose their bite. Whole brain emulation (that just follows the evolved biology) keeps compute centrally relevant as a framing for the resources that matter in a sufficiently distant future (even if it doesn't ever become a path to uplifting, by stipulation).
I guess the statement about star-scale superintelligence at the end is sufficiently off-topic that I shouldn't have made it like that without more context, sorry about that (I've now added a footnote). I think star-scale superintelligence is a helpful anchor, to see by analogy the ways in which the arguments should still apply even with strictly human-level biological intelligence, let alone superhuman if biologically evolved intelligence (by gesturing at where technological maturity might be located, in contrast to what's obviously insufficiently ambitious as a prediction of far future engineering capabilities). So this anchor works a bit like exploratory engineering itself.
Armstrong, S., & Sandberg, A. (2013). Eternity in six hours: Intergalactic spreading of intelligent life and sharpening the Fermi paradox. Acta Astronautica, 89, 1-13. ↩︎
Of course, I would never expect the majority of people to do something useful at such a time. But yes, I expected them to do something, or at least to fear the gravity of the moment, with at least some implications for their lives.
Did you really? I'd love to hear a longer account of what you expected this average Joe to do...and in response to what moment, exactly? ChatGPT? DeepSeek? OpenClaw? Anthropic ARR hitting $40b? I'm just not sure there's been a Cuban Missle Crisis type of event to awaken this median person.
I don't know... I expected them at least to talk about it. I expected it to be a big political issue, probably.
I really think the lack of a "moment" is key. It's been closer to death by 1,000 papercuts...even on LW there's not a consensus, e.g. everyone screaming from the rooftops, "oh, this Mythos thing is the tipping point, look out!!!"
I've been feeling a different kind of banality. There's a way in which I can look at the world and the trends and go "holy shit, the future is here and things are getting better and weirder every day!" and then I can be so over Waymo's that I'm annoyed they aren't perfect drivers rather than being in awe that the car can drive itself at all. Or I'm at work and AI is building all this stuff for me, but I'm annoyed at how much work all this cheap code is making for me. Or how I can get an AI to generate any image I want, but it's not quite the image I want, and why doesn't the AI just understand me.
As it's said, no matter where you go, there you are, and once I get over getting to the place, it's still just me, being in a place, adapted to it and trying to live life the best I can.
I get your point, I'm also frustrated that very few people seem to be aware that the likelihood of something insane happening in the next 10 years seems quite high. Especially for me I'm very unused to being the most alarmed person in the room so it's easy to feel annoying or crazy for trying to talk about these things. I suppose other people with a past history of passionate causes are used to these feelings and to them I seemed like the one with my head stuck in the sand.
If it makes you feel better, if you (we) are right, there will come a point where the general population is gripped by the intensity of the moment. I can only suggest you make an effort to bring about that point as soon as possible by spreading the word. Perhaps I should do the same.
This reads like a religious person complaining about people's lack of faith.
I notice that you refer to people working on AGI as 'people' and the people who either disagree with you or are ignorant of AGI as 'humans'. That's some sneaky dehumanising language that you should probably check in with yourself on. Because that's the sort of thought patterns that allow you to justify horrible things happening to people.
I notice that you refer to people working on AGI as 'people' and the people who either disagree with you or are ignorant of AGI as 'humans'.
First, the post literally does not do this thing. It uses "people", "humans", and "my species" to refer to the same set; and repeatedly uses "people" in a way that explicitly includes those who disagree with the author; e.g. "However much I may disagree with the people working at AGI labs..." and "Of course, I would never expect the majority of people to do something useful at such a time."
The piece does shift to using "humans" towards the end, but not in a way that separates "people" from "humans" in the way you're claiming here.
That's some sneaky dehumanising language
Second, the accusation that calling people "humans" is dehumanizing, seems ... decidedly nonobvious. More so in the context of Less Wrong where "humans" is actually pretty common; the general discourse tends to be comfortable talking about human values, the future of humanity, etc.
that you should probably check in with yourself on.
Third, this comes across as obnoxious sneering, which is not a great thing to be doing here.
Back in the day, there was some point in dismissing and laughing at transhumanist-rationalist ways. One could make the case — correct or not, but at least reasonable-sounding — that it is useless or even actively harmful to spend months in singularity daydreaming, in thinking about the future of human civilization, of the local galaxy cluster, of the entire negentropy of this Universe. It was, arguably, better, more sober, more adult, for an intelligent, ambitious man to focus on financial consulting or on building better apps.
Of course, deep in my soul, I knew, or convinced myself that I knew, that society doesn't care about superclusters not because the time is not ripe for this problem, but because society is profoundly homeostatic in its beliefs and actions.
Back in the day, although I expected to see marvels of progress beyond my imagination in my lifetime, I never really expected to live through the takeoff. There was a part of me, to be clear, which harbored the fear that the things I care about are just not relevant — at least not yet relevant — that I am akin to a medieval peasant pondering the implications of nuclear proliferation. Sometimes, just sometimes, I thought that I was escaping the real world and real work under the excuse of caring about something more important, which wasn't, in fact, more important.
And yet, now that I live through the takeoff, no one cares.
However much I may disagree with the people working at AGI labs, at least I understand them. The desire to steer the future of reality is the most natural desire one can have, under my model. And if normal people do nothing instead of doing harmful things — not because they don't want to do harmful things, but because they envision doing nothing as the only conceivable option — is that more dignity for them, or actually less? Their choice to do nothing amidst the intelligence explosion is not a moral choice; it is not a choice at all. And if they had just slightly more agency, wouldn't they do things that are worse than nothing?
And yet, it is a humiliation to see my species lying catatonic as the shockwaves of the Singularity pass through.
Of course, I would never expect the majority of people to do something useful at such a time. But yes, I expected them to do something, or at least to fear the gravity of the moment, with at least some implications for their lives.
My relationship with normal people is… evolving interestingly in the face of AI doom. A bittersweet mixture: the realization of their fleeting and fragile value — of them being the only seed of fun and sentience in the Universe, the only spark, almost faded, bearing the potential to spread this fun and sentience everywhere — and of their ignorance, apathy, and meanness in front of cosmic-scale events. Humans are the best thing in the Universe, and that is our tragedy.
And so, depending on which axis I project onto, I am becoming simultaneously more philanthropic and more misanthropic. It is humanity I want to spend the rest of my life fighting for; humanity is worth dying for; and humanity is a failure.
Regardless, I am more and more alienated from normal people. I spend less and less time trying to convince them, or even engage with them. Their plans, life goals, and lifestyles never seemed particularly compelling to me, but now they look plainly insane. As time passes, I look at them less with desperation and more with curiosity — as at an intriguing new species, or a foreign culture.
Like Mad Max, I say: in this wasteland, it is hard to say who is more crazy — me or everyone else. And indeed, one of us must be crazy. Both options are terrifying.
One can always invent some plausible story for why others don't care. Not enough time, other priorities, long inferential distances, not smart enough, and so on. If I am charitable, it's not hard to forgive.
But now is exactly the time not to be charitable. If there is any single time to have high expectations of humans, it is now. If I am to ask humans, one single time, not to be insane, it is now.
Arendt's banality was the banality of the perpetrator: evil carried out by clerks, without hatred and without thought. The banality I live in is the banality of the bystander.
I love humans. I want to love humans. But I do not trust them.
I live in the banality of takeoff.