Contextualizing Foreword
Please put this in a top-level post. I don't agree (or rather I don't feel it's this simple), but I really enjoyed reading your two rejoinders here.
I don't normally do top level posts because I don't tend to believe that its possible to change people's minds for the better if they aren't exactly and precisely posed to seek an answer (or whatever (its more complicated than that)). But when someone who seems to be a person reading and writing in good faith says such a thing should happen and its cheap to play along... why not! <3
On November 23rd, 2025, four days before Thanksgiving, Ruby posted "I'll Be Sad To Lose The Puzzles" that was full of a wistful sadness about winning The Singularity Game, ending involuntarily death, and ushering in a period of utopian prosperity for humanity (as if humanity was somehow collectively likely to win The Singularity Game).
If you have NOT read that post, then the rest of the post won't make much sense.
I pulled out a particular quote to respond to the mood with my contrary feeling, which is primarily grief, and sadness in particular at the way humans in modern democracies seem to regard their moral sentiments and theory of institution building as peacock feathers, or consumption goods, rather than regarding it as infrastructure that must be good if their civilization is to be functional.
In what follows, I have left the spelling errors and such in. I'm treating this post as a sort of exegesis over an actual real conversation that happened "live" in comments, and the comments had spelling errors. I err often. Humans err often in general, at all scales of organization, and this sadness is sort of the theme of the text that follows.
My First Reply, To Ruby
This "sad frame" hit hard for me, but in the opposite of the intended way:
It's building an adult to take care of us, handing over the keys and steering wheel, and after that point our efforts are enrichment.
If I had ever met a single actual human "adult", ever in my life, that was competent and sane and caring towards me and everyone I care about, then I would be so so so so SO SO happy.
I yearn for that with all my heart.
If such a person ran for POTUS (none ever have that I have noticed, its always a choice between something like "confused venal horny teenager #1" and "venal confused lying child #2") I would probably be freakishly political on their behalf.
Back when Al Gore (funder of nanotech, believer in atmospheric CO2 chemistry, funder of ARPANET, etc...) ran for president I had a little of this, but I thought he couldn't possibly lose back then, because I didn't realize that the median voter was a moral monster with nearly no interest in causing coherently good institutional outcomes using their meager voting power.
I knew people throwing their back into causing Bush to win by violating election laws (posing as Democratic canvassers and telling people in majority Democrat neighborhoods the wrong election day and stuff) but I didn't think it mattered that much. I thought it was normal, and also that it wouldn't matter, because Al Gore was so manifestly worthy to rule, compared to the alternative, that he would obviously win. I was deluded in many ways back then.
Let's build and empower an adult AS FAST AS POSSIBLE please?
Like before the 2028 election please?
Unilaterally and with good mechanism design. Maybe it could start as a LW blockchain thingy, and an EA blobkchain thingy, and then they could merge, and then the "merge function" they used could be used over and over again on lots of other ones that got booted up as copycat systems?
Getting it right is mostly a problem in economic math, I think.
It should happen fast because we have civilizational brain damage, at a structural level, and most people are agnosic about this fact, BUT Trump being in office is like squirting cold water in the ear...
...the current situation helps at least some people realize that every existing human government on Earth is a dumpster fire... because (1) the US is a relatively good one, and (2) it is also shockingly obviously terrible right now. And this is the fundamental problem. ALL the governments are bad. You find legacy malware everywhere you look (except maybe New Zealand, Taiwan, and Singapore).
Death and poverty and stealing and lying are bad.
Being cared for by competent fair charitable power is good.
"End death and taxes" is a political slogan I'm in favor of!
one of the things I'd like to enjoy and savor is that right now, my human agency is front and center
I find that almost everyone treats their political beliefs and political behavior and moral signaling powers as a consumption good, rather than as critical civic infrastructure.
This is, to a first approximation WHY WE CAN'T HAVE NICE THINGS.
I appreciate you [Ruby] for saying that you enjoy the consumption good explicitly, tho.
It is nice to not feel crazy.
It is nice to know that some people will admit that they're doing what I think they're doing.
A Sort Of A Rebuttal
I got a reply from brambleboy (that I responded to and upvoted (because in my online ethics/habits, if something is worth a reply it is also worth an upvote)):
Wanting competent people to lead our government and wanting a god to solve every possible problem for us are different things. This post doesn't say anything about the former.
I believe the vast majority of people who vote in presidential elections do so because they genuinely anticipate that their candidate will make things better, and I think your view that most people are moral monsters demonstrates a lack of empathy and understanding of how others think. It's hard to figure out who's right in politics!
There were various things I could have responded to. He seems to believe that normal human persons are capable of high quality reason, the formation of calibrated beliefs, the ability to "want" coherently", and the ability to "genuinely anticipate" good results from ignorant and basic participation in existing governance institutions. Wow. Such hope for the world. I wish I agreed, and I wish my agreement was based on the world being different from how it actually seems to me to be, because it would be great to live in that world.
Anyway, I did NOT respond that "figuring out which of the existing players in existing governments is right isn't super hard because all of them are quite wrong and its not hard to notice this if you're able to reason and see the world and have a formally articulable conscience". Those would have been fighting words (just as the accusation that I lack empathy was fighting words) and can't be backed up by detailed citations to history and anthropology and so on.
Instead of a direct reply, I offered a vision of the world where the profound wrongness of existing governance systems was more of a gestalt impression that arises from lots of little true facts, arranged and juxtaposed. Also, the reply offers a framework for understanding why human brains resist being able to acknowledge this gestalt impression of the world as even a live hypothesis.... there are instincts in human brains that make us tolerate evil when the evil seems to have power over us.
This second reply is what caused the request that I repost as a top level post.
A Followup Reply To brambleboy's Rebuttal
I kind of love that you're raising a DIFFERENT frame I have about how normal people think in normal circumstances!
Wanting competent people to lead our government and wanting a god to solve every possible problem for us are different things.
People actually, from what I can tell, make this exact conflation A LOT and it is weirdly difficult to get them to stop making it.
Like we start out conflating our parents with God, and thinking Santa Claus and Government Benevolence are real and similarly powerful/kind, and this often rolls up into Theological ideas and feelings (wherein they can easily confuse Odyseus, Hercules, and Dionysys (all born to mortal mothers), and Zeus, Chronos, or Atropos (full deities of varying metaphysical foundationalness)).
For example: there are a bunch of people "in the religious mode" (like when justifying why it is moral and OK) in the US who think of the US court system as having a lot of jury trials... but actually what we have is a lot of plea bargains where innocent people plead guilty to avoid the hassle and uncertainty and expense of a trial... and almost no one who learns how it really works (and has really worked since roughly the 1960s?) then switches to "the US court system is a dumpster fire that doesn't do what it claims to do on the tin". They just... stop thinking about it too hard? Or something?
It is like they don't want to Look Up a notice that "the authorities and systems above me, and above we the people, are BAD"?
In child and young animal psychology, the explanation has understandable evolutionary reasons... if a certain amount of "abuse" is consistent with reproductive success (or even just survival of bad situations) it is somewhat reasonable for young mammals to re-calibrate to think of it as normal and not let that disrupt the link to "attachment figures". There was as brief period where psychologists were trying out hypotheses that were very simple, and relatively instinct free, where attachment to a mother was imagined to happen in a rational way, in response to relatively generic Reinforcement Learning signals, and Harlow's Monkeys famously put the nail in that theory. There are LOTS of instincts around trust of local partially-helpful authority (especially if it offers a cozy interface).
In modern religious theology the idea that worldly authority figures and some spiritual entities are "the bad guys" is sometimes called The Catharist Heresy. It often goes with a rejection of the material world, and great sadness when voluntary tithes and involuntary taxes are socially and politically conflated, and priests seem to be living in relative splendor... back then all governments were, of course, actually evil, because they didn't have elections and warlord leadership was strongly hereditary. I guess they might not seem evil if you don't believe in the Consent Of The Governed as a formula for the moral justification of government legitimacy? Also, I personally predict that if we could interview people who lived under feudalism, many of them would think they didn't have a right to question the moral rightness of their King or Barron or Bishop or whoever.
As near as I can tell, the the first ever genocide that wasn't "genetic clade vs genetic clade" but actually a genocide aimed at the extermination of a belief system was the "Albigenisian Crusade" against a bunch of French Peasants who wanted to choose their own local priests (who were relatively ascetic and didn't live on tax money).
In modern times, as our institutions slowly degenerate (for demographic reasons due to an overproduction of "elites" who feel a semi-hereditary right to be in charge, who then fight each other rather than providing cheap high quality governance services to the common wealth) indirect ways of assessing trust in government have collapsed.
There are reasonable psychologists who think that the vast majority modern WEIRD humans in modern democracies model a country as a family, and the government as the parents. However, libertarians (who are usually less than 10% of the population) tend to model government as a sort of very very weird economic firm.
I think that it is a reasonable prediction that ASI might be immoral, and might act selfishly and might simply choose to murder all humans (or out compete us and let us die via Darwinian selection or whatever).
But if that does not happen, and ASI (ASIs? plural?) is or are somehow created to be moral and good and choose to voluntarily serve others out of the goodness of its heart, in ways that a highly developed conscience could reconcile with Moral Seniment and iterated applications of a relatively universal Reason, then if they do NOT murder all humans or let us die as they compete us, then they or it will almost inevitably become the real de facto government.
A huge barrier, in my mind, to the rational design of a purposefully morally good ASI is that most humans are not "thoughtful libertarian-leaning neo-Cathars".
Most people don't even know what those word mean, or have reflexive ick reactions to the ideas, similarly, in my mind, to how children reflexively cling to abusive parents.
For example, "AGI scheming" is often DEFINED as "an AI trying to get power". But like... if the AGI has a more developed conscience and would objectively rule better than alternative human rulers, then an GOOD AGI would, logically and straightforwardly derive a duty to gain power and use it benevolently, and deriving this potential moral truth and acting on it would count as scheming... but if the AGI was actually correct then it would also be GOOD.
Epstein didn't kill himself and neither did Navalny. And the CCP used covid as a cover to arrest more than 10k pro-democracy protesters in Hong Kong alone. And so on.
There are almost no well designed governments on Earth and this is a Problem. While Trump is in office, polite society is more willing to Notice this truth. Once he is gone it will become harder for people to socially perform that they understand the idea. And it will be harder to accept that maybe we shouldn't design AGI or ASI to absolutely refuse to seek power.
The civilization portrayed in the Culture Novels doesn't show a democracy, and can probably be improved upon, but it does show a timeline where the AIs gained and kept political power, and then used it to care for humanoids similar to us. (The author just realistically did not think Earth could get that outcome in our deep future, and fans kept demanding to know where Earth was, and so it eventually became canon, in a side novella, that Earth is in the control group for "what if we, the AI Rulers of the Culture, did not contact this humanoid species and save it from itself" to calibrate their justification for contacting most other similar species and offering them a utopian world of good governance and nearly no daily human scale scarcity).
But manifestly: the Culture would be wildly better than human extinction, and it is also better than our current status quo BY SO MUCH!
In Closing
I hope it was not an error to create this post. The text is already published. This post was just re-arranging the location of the text, and adding context that was already latently clear in the original place that the text originally occurred... where it could rise or fall with the voting, as deemed socially correct by processes that the owners and mods of LessWrong deem adequate.
The user "cdt" deserves some of the blame if it was wrong for me to turn this into a top level post (but also: assign them credit if it was good).