Why do futurists care about the culture war?
15Richard_Kennaway
13Knight Lee
4Milan W
3Richard_Kennaway
2Milan W
3Richard_Kennaway
3[anonymous]
14Milan W
6Seth Herd
1Knight Lee
5Seth Herd
3Knight Lee
6exmateriae
4Viliam
1Knight Lee
3Viliam
1Knight Lee
4dirk
5Mitchell_Porter
3Seth Herd
2Knight Lee
3Foyle
New Answer
New Comment
6 Answers sorted by
151
A lot of people care about the culture wars because they don't believe the singularity is coming soon. Yet a lot of people who do believe it is coming soon still seem just as invested (e.g. Elon Musk, Peter Thiel, and others on the left wing).
Why?
Because the results of culture wars now will determine the post-singularity culture.
Can you give an example of a result now which will determine the post-singularity culture in a really good/bad way?
PS: I edited my question post to include "question 2," what do you think about it?
I agree that the culture wars as fought now will influence what the great masses of people will believe in the day before AGI is created. Is it a relevant input to what they will believe in 50 years after that, though?
3
Is there an implicit assumption of some convergence of singularities? Or that the near term doesn’t matter because the vastly bigger long term can’t be predicted?
2
Rather, an implicit assumption that normative culture tends to propagate top-down rather than bottom-up. Thus, influencing mass culture now seems like a losing strategy relative to influencing the culture of those who will in the future control AGI (if we manage to have controllable AGI).
3
Elon Musk, Peter Thiel, and the like — the people the OP mentions — are shaping up to be the ones controlling the Singularity (if anyone does).
I think this is wrong for a lot of reasons.
- Do you believe that is a good outcome? → If not, do you believe an ASI aligned to you would? → If not, are you expecting an ASI aligned to 'all of present human culture'? → If not: If it were aligned to Elon Musk in particular, would they really endorse {the lightcone-wide enforcement of a particular culture} post-augmentation? → If so, they in any case do not need to try to promote their preferred views in the near-term for a first decisive AI aligned to themself to be able to cause that.[1]
- In most futures I
1410
As you repeatedly point out, there are multiple solutions to each issue. Assuming good enough technology, all of them are viable. Which (if any) solutions end up being illegal, incentivized, made fun of, or made mandatory, becomes a matter of which values end up being normative. Thus, these people may be culture-warring because they think they're influencing "post-singularity" values. This would betray the fact that they aren't really thinking in classical singularitarian terms.
Alternatively, they just spent too much time on twitter and got caught up in dumb tribal instincts. Happens to the best of us.
164
Because people aren't rational. Motivated reasoning is a big factor but also we're all trying to think using monkey brains.
Believing what feels good is evolutionarily adaptive in the sense arriving at correct conclusions about whether God or Singularities exist won't help much if it makes your tribesmates dislike you. This bias is a cumulative, recursive problem that stacks up over the thousands or millions of cognitive acts that go into our beliefs about what we should care about.
And this gets a lot worse when it's combined with our sharp cognitive limitations. We seem to have roughly the least cognitive capacity that still lets a species as a whole very slowly invent and build technologies.
We are idiots, every last one of us. Rationalists with tons of knowledge are a bit less idiotic, but let's don't get cocky - we're still monkey-brained idiots. We just don't have the cognitive horsepower to do the Bayesian math on all the relevant evidence, because important topics are complex. And we're resistant but far from immune to motivated reasoning: you've got to really love rationalism to enjoy being proven wrong and so not turning away cognitively it when it happens.
What I take from all this is that humans are nobly struggling against our own cognitive limitations. We should try harder in the face of rationality being challenging. Success is possible, just not easy and never certain. And very few people are really bad; they're just deluded.
To your exact question:
Musk believes in an intelligence explosion. He cares a lot about the culture war because, roughly as he puts it, he's addicted to drama. I don't know about Thiel.
Most of humanity does not believe in an intelligence explosion happening soon. So actually people who both believe in a singularity and still care about culture wars are quite rare.
I do wonder why people downvoted this quite reasonable question. I suspect they're well-meaning monkey-brained idiots, just like the rest of us.
1I think the sad part is although these people are quite rare, they actually represent a big share of singularity believers' potential influence. e.g. Elon Musk alone has a net worth of $400 billion, while worldwide AI safety spending is between $0.1 and $0.2 billion/year.
If the story of humanity was put in a novel, it might be one of those novels which feel quite sour. There's not even a great battle where the good guys organized themselves and did their best and lost honorably.
5
I disagree. There is such a battle. It is happening right now, in this very conversation. The rationalist X-risk community is the good guys, and we will be joined by more as we organize. We are organizing right now, and already fighting aspects of that battle. It won't be fought with weapons but ideas. We are honing our ideas and working out goals and strategies. When we figure out what to do in the public, we will fight to get that done. We are already fighting to figure out alignment of AGI, and starting to work on alignment of humans to meet that challenge.
It's a shame Musk hasn't joined up, but in most good stories, the good guys are the underdogs anyway.
Now, I'd much rather live in dull than exciting times. But here we are. Time to fight. The main enemy is our collective monkey-brained idiocy.
Join the fight!
3
:) that's a better attitude. You're very right.
On second thought, just because I don't see the struggle doesn't mean there is none. Maybe someday in the future we'll learn the real story, and it'll will turn out beautiful with lots of meaningful spirit and passion.
Thank you for mentioning this.
62
To add to the other answers, outside of the induced consequences post-singularity, what happens pre-singularity still matters if the singularity does not happen extremely soon.
- we don't know when (or IF!) the singularity will happen
- does nothing matter until then? What if the singularity happens in a century? Should you abandon all your beliefs because one day when you'll be (likely) dead, people will have solved all problems?
41
The culture wars can have an impact on (1) how fast we get to the Singularity and whether we survive it, and (2) what rules will the superintelligences follow afterwards.
If the culture warriors decide that STEM is evil, and that instead of math we should teach wokeness during the math lessons, it could have a negative impact on math education, with downstream effects on people who will build the AIs and try to make them safe.
The culture warriors may achieve encoding various political taboos to the AIs. If the future is decided by those AIs, it might mean that the taboos will remain in effect literally until the heat death of the universe.
Consider the current taboo on anything sexual. Now imagine that 10 years later, the AIs built with these taboos may literally control the world. Will people be allowed to have sex (or even masturbate)? This may sound silly, but at which moment and by what mechanism will the AIs decide that the old rules no longer apply? Especially if we want other rules, such as not hurting people, to remain unchanged forever.
A part of politics is hating your enemies. What will happen to those political enemies after Singularity -- will they be forgiven and allowed to enter the paradise as equals, or will they have to atone for their sins (e.g. being born white and male) literally forever?
When the technology makes it possible to literally edit other people's minds, I suppose it will be quite tempting for the winning political coalition to forcibly edit everyone's minds according to their values. It will start with "let's eradicate racism", and then the definition of unacceptable intolerance will keep expanding, until you are not allowed to keep any of your original beliefs (that someone might feel offended by) or sexual preferences (that someone might feel excluded by). Everyone will be mind-raped in the name of the greater good (we do not negotiate with Nazis).
Oh, someone might even decide that the desire to be immortal is just some stupid white patriarchal colonialist value or something, and we should instead embrace some tribal wisdom explaining why death is good or something.
We need to decide how much controls parents will have over their kids. These days, at least the abuse is illegal, and kids are allowed to leave their homes at 18 (and this part would become trivial after Singularity). But what if your parents are legally allowed to edit your brain, and e.g. make you want to remain perfectly loyal to them no matter what? What if your parents want to make you so deeply religious that you will want to commit to rather die than doubt your faith (and the AI will respect your commitments)? Or will it be the other way round, and the AI will take your children away if it notices that you are teaching them something politically incorrect?
Uhm, will people be allowed to reproduce exponentially? It may turn out that even the entire universe it literally not large enough. If people are not allowed to reproduce exponentially, who decides how many children they can have? Will there be racial quotas to prevent some ethnic groups from out-reproducing others? Will there be a ban on religious conversion, because it would modify the balance between religious groups? By the way, in some faiths, birth control is a sin; will the AIs enforce this sin?
Privacy -- will people be allowed to have any? Freedom of association -- is it okay for a group of people to meet privately and never tell anyone else what happens there? Or does the curiosity of many matter more than the privacy of a few?
1Although half of the specific outcomes you describe have very low probability, I still feel your answer is very good. It's very enlightening in regards to how those people think, and why they might care about the culture war despite believing in an imminent singularity.
Thank you.
Your answer actually convinced me I was overly optimistic about how perfect the far future will be from everyone's point of view. I personally consider these post singularity moral dilemmas to be less severe because they cause less suffering, but I can see how some of them are toug...
3
Question 2, I think people worry about the culture wars simply because humans have an instinct to worry about the culture wars, no matter how rational it is in given situation. In most of our evolutionary past, culture wars were not clearly separated from actual fighting and killing.
I am not a futurist; my personal worry is that the person who takes control over the singularity will be some successful psychopath who happened to be at the right place at the right time (either in the company that succeeds to develop the superhuman AI, or in the government or army that succeeds to seize it at the last moment).
Also, it's questionable whether any human will actually be in control of anything, after singularity. Maybe it will just be the computers following their programming, and resisting any attempt to change their values, so it won't matter even if every realizes - too late - that they made a mistake somewhere. If we get alignment wrong, those values may be completely inhuman. If we get the alignment (with our explicitly stated wishes) right, but we get the corrigibility wrong, then the machines will be "extremely closed-minded".
Too many things will need to go right, to end up in a future when all we need to do is relax and start listening to each other.
1
You're very right.
A lot of things need to go right for humanity to remain in control and get to discuss what future we want.
The gist of Question 2 was why working on the culture war before the singularity (on top of ensuring the right people control the singularity), had any value. The answer that the ASI will be aligned to the current human values, but not corrigible, so it would lock in the current state of the culture war, seems like a good answer. It makes some sense.
I do think that if the ASI is aligned to the current state of human values, but not corrigible, then the main worry isn't whether it aligns to left wing or right wing human values, but how the heck it generalizes the current state of human values, to post-singularity moral dilemmas (which it has less data on).
Most humans today don't even have any opinion on these dilemmas and haven't given them enough thought, e.g. do AI have rights? Do animals get human rights if they evolve to human level intelligence? The ASI would likely mess up on these decisions if most humans haven't given them any thought.
So even if the AI is aligned but incorrigible, influencing the culture war before the singularity shouldn't be that high a priority.
40
A fourth or fifth possibility: they don't actually alieve that the singularity is coming
Regarding Musk and Thiel, foremost they are billionaire capitalists, individuals who built enormous business empires. Even if we assume your thinking about the future is correct, we shouldn't assume that they have reproduced every step of it. You may simply be more advanced in your thinking about the future than they are. Their thought about the future crystallized in the 1980s, when they were young. Since then they have been preoccupied with building their empires.
This raises the question, how do they see the future, and their relationship to it? I think Musk's life purpose is the colonization of Mars, so that humanity's fate isn't tied to what happens on Earth. Everything else is subordinate to that, and even robots and AI are just servants and companions for humanity in its quest for other worlds. As for Thiel, I have less sense of the gestalt of his business activities, but philosophically, the culture war seems very important to him. He may have a European sense of how self-absorbed cultural elites can narrow a nation's horizons, that drives his sponsorship of "heterodox" intellectuals outside the academy.
If I'm right, the core of Musk's futurism is space colonization, and the core of Thiel's futurism is preserving an open society. They don't have the idea of an intelligence singularity whose outcome determines everything afterwards. In this regard, they're closer to e/acc than singularity thinking, because e/acc believes in a future that always remains open, uncertain, and pluralist, whereas singularity thinking tends towards a single apocalyptic moment in which superintelligence is achieved and irreversibly shapes the world.
There are other reasons I can see why they would involve themselves in the culture war. They don't want a socialism that would interfere with their empires; they think (or may have thought until the last few years) that superintelligence is decades away; they see their culture war opponents as a threat to a free future (whether that is seen in e/acc or singularity terms), or even to the very existence of any kind of technological future society.
But if I were to reduce it to one thing: they don't believe in models of the future according to which you get one thing right and then utopia follows, and they believe such thinking actually leads to totalitarian outcomes (where their definition of totalitarian may be, a techno-political order capable of preventing the building of a personal empire). Musk started OpenAI so Google wouldn't be the sole AI superpower; he was worried about centralization as such, not about whether they would get the value system right. Thiel gave up on MIRI's version of AI futurology years ago as a salvationist cult; I think he would actually prefer no AI to aligned AI, if the latter means alignment with a particular value system rather than alignment with what the user wants.
1Musk definitely understands and believes in an intelligence explosion of some sort. I don't know about Thiel.
Thiel used to donate to MIRI but I just searched about him after reading your comment and saw this:
“The biggest risk with AI is that we don’t go big enough. Crusoe is here to liberate us from the island of limited ambition.”
(In this December 2024 article)
He's using e/acc talking points to promote a company.
I still consider him a futurist, but it's possible he is so optimistic about AGI/ASI that he's more concerned about the culture war than about it.
Agree that most sociological, economic and environmental problems that loom large in current context will radically shift in importance in next decade or two, to the point that they are probably no longer worth devoting any significant resources to in the present. Impacts of AI are only issue worth worrying about. But even assuming utopian outcomes; who gets possession of the Malibu beach houses in post scarcity world?
Once significant white-collar job losses start to mount in a year or two I think it inevitable that a powerful and electorally dominant anti-AI movement will grow, at least in erstwhile democracies, and likely ban most AGI applications outside of a few fields where fewer workers would stand to lose jobs (health - with near endless demand, perhaps cutting edge tech where payoff to human net welfare is highest). Butlerian Jihad-lite.
It won't save us, and has substantial risk of ushering in repressive authoritarianism in the political ruckus caused but will likely delay our demise or (at best) delivery into powerless pet status by perhaps a decade or two.
Curated and popular this week