LESSWRONG
LW

634
Richard_Ngo
19839Ω285416911130
Message
Dialogue
Subscribe

Formerly alignment and governance researcher at DeepMind and OpenAI. Now independent.

Sequences

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
6Richard Ngo's Shortform
Ω
5y
Ω
456
Obligated to Respond
Richard_Ngo19h100

"consistent with my position above I'd bet that in the longer term we'd do best to hit a button that ended all religions today, and then eat the costs and spend the decades/centuries required to build better things in their stead."

Would you have pressed this button at every other point throughout history too? If not, when's the earliest you would have pressed it?

Reply1
Richard Ngo's Shortform
Richard_Ngo4d30

Good question. One answer is that my reset mechanisms involve cultivating empathy, and replacing fear with positive motivation. If I notice myself being too unempathetic or too fear-driven, that's worrying.

But another answer is just that, unfortunately, the reality distortion fields are everywhere—and in many ways more prevalent in "mainstream" positions (as discussed in my post). Being more mainstream does get you "safety in numbers"—i.e. it's harder for you to catalyze big things, for better or worse. But the cost is that you end up in groupthink.

Reply1
Richard Ngo's Shortform
Richard_Ngo4d7-2

I like this comment.

For the sake of transparency, while in this post I'm mostly trying to identify a diagnosis, in the longer term I expect to try to do political advocacy as well. And it's reasonable to expect that people like me who are willing to break the taboo for the purposes of diagnosis will be more sympathetic to ethnonationalism in their advocacy than people who aren't. For example, I've previously argued on twitter that South Africa should have split into two roughly-ethnonationalist states in the 90s, instead of doing what they actually did.

However, I expect that the best ways of fixing western countries won't involve very much ethnonationalism by historical standards, because it's a very blunt tool. Also, I suspect that breaking the taboo now will actually lead to less ethnonationalism in the long term. For example, even a little bit more ethnonationalism would plausibly have made European immigration policies much less insane over the last few decades, which would then have prevented a lot of the political polarization we're seeing today.

Reply
Richard Ngo's Shortform
Richard_Ngo4d82

This is a thoughtful comment, I appreciate it, and I'll reply when I have more time (hopefully in a few days).

Reply1
Richard Ngo's Shortform
Richard_Ngo4d160

Thanks for the extensive comment. I'm not sure it's productive to debate this much on the object level. The main thing I want to highlight is that this is a very good example of how the taboo that I discussed above operates.

On most issues, people (and especially LWers) are generally open to thinking about the benefits and costs of each stance, since tradeoffs are real.

However, in the case of ethnonationalism, even discussing the taboo on it (without explicitly advocating for it) was enough to trigger a kind of zero-tolerance attitude in your comment.

This is all the more striking because the main historical opponent of ethnonationalist regimes was globalist communism, which also led to large-scale atrocities. Yet when people defend a "socialist" or "egalitarian" cluster of ideas, that doesn't lead to anywhere near this level of visceral response.

My main bid here is for readers to notice that there is a striking asymmetry in how we think about and discuss 20th century history, which is best explained via the thing I hypothesized above: a strong taboo on ethnonationalism in the wake of WW2, which has then distorted our ability to think about many other issues.

Reply
Richard Ngo's Shortform
Richard_Ngo4d*82

For the most obvious example, for the life of me I cannot understand how leaving the gold standard makes a culture less appreciative of any kind of moral virtue, unless you equate two very different senses of the word "value".

Might reply to the rest later but just to respond to what you call "the most obvious example": consider a company which has a difficult time evaluating how well its employees are performing (i.e. most of them). Some employees will work hard even when they won't directly be rewarded for that, because they consider it virtuous to do so. However, if you then add to their team a bunch of other people who are rewarded for slacking off, the hard-working employees may become demotivated and feel like they're chumps for even trying to be virtuous.

The extent to which modern governments hand out money causes a similar effect across western societies (edited: for example, if many people around you are receiving welfare, then working hard yourself is less motivating). They would not be as able to do this as much if their currencies were still on the gold standard, because it would be more obvious that they are insolvent.

Reply2
Richard Ngo's Shortform
Richard_Ngo4d21

I used to agree with your understanding but I am now more skeptical. For example, here's a story that says the opposite:

The poorer humans are, the more vulnerable each human is to the group consensus. People who disagreed with groups could in the past easily be assaulted by mobs, or harassed in a way that led them to literally starvation-level wealth. Nowadays, though, even victims of extreme 'cancel culture' don't face such risks, because society is wealthy enough that you can do things like move to a new city to avoid mobs, or get charities to feed and clothe you even if you lose your job.

Also it's much harder to design parasitic egregores now than it used to be, because our science is much better and so we know many more facts, which makes it harder for egregores to lie.

I'm not saying my story is true, but it does highlight that the load-bearing question is actually something like "how does the offense-defense balance against parasitic egregores scale with wealth?" Why don't we live in a world where wealth can buy a society defenses against such egregores?

Or maybe we do live in such a world, and we are just failing to buy those defenses. That seems like a really dumb situation to be in, but I think my post is broadly describing how it might arise.

Reply
Richard Ngo's Shortform
Richard_Ngo4d1-4

That's a mechanism by which I might overestimate the support for Hamas. But the thing I'm trying to explain is the overall alignment between leftists and Hamas, which is not just a twitter bubble thing (e.g. see university encampments).

More generally, leftists profess many values which are upheld the most by western civilization (e.g. support for sexual freedom, women's rights, anti-racism, etc). But then in conflicts they often side specifically against western civilization. This seems like a straightforward example of pessimization.

Reply
Richard Ngo's Shortform
Richard_Ngo5d20

Consequentialism and utility functions or policies could in principle be about virtues and integrity as about hamburgers, but hamburgers are more legible and easier to administer.

Here's one concrete way in which this isn't true: one common simplifying assumption in economics is that goods are homogeneous, and therefore that you're indifferent about who to buy from. However, virtuous behavior involves rewarding people you think are more virtuous (e.g. by preferentially buying things from them).

In other words, economics is about how agents interact with each other via exchanging goods and services, while virtues are about how agents interact with each other more generally.

Reply
Richard Ngo's Shortform
Richard_Ngo5d20

Sufficiently different versions of yourself are just logically uncorrelated with you and there is no game-theoretic reason to account for them.

Seems odd to make an absolute statement here. More different versions of yourself are less and less correlated, but there's still some correlation. And UDT should also be applicable to interactions with other people, who are typically different from you in a whole bunch of ways.

Reply
Load More
No wikitag contributions to display.
Twitter threads
Understanding systematization
Stories
Meta-rationality
Replacing fear
Shaping safer goals
AGI safety from first principles
152Underdog bias rules everything around me
22d
53
61On Pessimization
1mo
3
56Applying right-wing frames to AGI (geo)politics
2mo
25
35Well-foundedness as an organizing principle of healthy minds and societies
5mo
7
99Third-wave AI safety needs sociopolitical thinking
6mo
23
96Towards a scale-free theory of intelligent agency
Ω
6mo
Ω
44
92Elite Coordination via the Consensus of Power
6mo
15
252Trojan Sky
6mo
39
214Power Lies Trembling: a three-book review
5mo
29
243The Gentle Romance
7mo
46
Load More