Introduction from Ben

Zhukeepa is a LessWronger who I respect and whose views I'm interested in. In 2018 he wrote the first broadly successful explication of Paul Christiano's research ideas for AI alignment, has spent a lot of time interviewing people in AI about their perspectives, and written some more about neuroscience and agent foundations research. He came first in the 2012 US Math Olympiad, and formerly worked on a startup called AlphaSheets that raised many millions of dollars and then got acquihired by Google.

He has also gone around saying (in my opinion) pretty silly-sounding things like he believes in his steelman of the Second Coming of Christ. He also extols the virtues of various psychedelics, and has done a lot of circling and meditation. As a person who thinks most religions are pretty bad for the world and would like to see them die, and thinks many people trick themselves into false insights with spiritual and psychological practices like those Alex has explored, I was interested in knowing what this meant to him and why he was interested in it, and get a better sense of whether there's any value here or just distraction.

So we sat down for four 2-hour conversations over the course of four weeks, either written or transcribed, and have published them here as an extended LessWrong dialogue.

I think of this as being more of an interview about Zhukeepa's perspective, with me learning and poking at various parts of it. While I found it interesting throughout, this is a meandering conversation that many may prefer to skip unless they too are especially curious about Zhukeepa's perspective or have a particular interest in the topics discussed. You can skim through the table of contents on the left to get a sense of the discussion, and also read Zhu's introductory thoughts immediately below.

Introduction from Alex

Despite the warnings and admonishments against doing so, I’d decided 5 years ago to venture off to the Dangerous Foreign Land of Religion and Spirituality, after becoming convinced that something in that land was crucial for thinking clearly about AI alignment and AI coordination. Since embarking on that journey, I’ve picked up a lot of customs and perspectives that the locals here on LessWrong are highly suspicious of.

A few months ago, I caught up with my old friend Ben Pace on a walk, who expressed a lot of skepticism toward my views, but nevertheless remained kind, patient, respectful, and curious about understanding where I was coming from. He also seemed to have a lot of natural aptitude in making sense of my views. This gave me hope that:

With Ben’s help, I could clarify a lot of my views, and translate a lot of the core insights I’ve picked up into a LessWrong-compatible a ontology
By dialoguing with Ben, I could form gears-level models for – and learn how to make friends with – the parts of the LessWrong memeplex that think I’ve gone off the deep end

… which is what motivated me to begin an extended series of dialogues with him. Below are a couple of excerpts going into some points that I'm particularly glad surfaced over the course of this dialogue:

There's a thing where the good scientists all know that science as an institution is broken in a bunch of ways. For example, most published studies fail to replicate. But when the median person thinks about science, they don't think about the opinions of the good scientists who understand how broken the whole thing is. The experience I had being at the Parliament of World Religions was like finding the religious leaders who are the analogues of the good scientists, who were just like, "Yeah, most mainstream religion is totally broken. It sucks for exactly the reasons the skeptics says it sucks."

[...]

The proximal thing I'm trying to target is creating a mathematical Rosetta Stone for the world religions [...] I think it would be more like "here's the correct way to interpret these stories, in contrast to these other interpretations that a majority of people currently used to interpret them". Like, a lot of people interpret hell as like, "Oh yeah, it's a place where there's going to be lots of heat, that people will be stuck in for literally infinity years." That's just wrong. That's not making it into the Rosetta Stone.

[...]

I think my picture is basically that the Schelling coalition / the preference synthesis coalition is going to have minimal Moloch internally within it, and that they become powerful enough to determine how the world runs. This is kind of like killing Moloch, and the process by which this all happens is pretty much how I interpret religious prophecies about the end of the world.

I also managed to clarify a couple of my core beliefs over the course of this dialogue, that I'd like to summarize here:

Religious exclusivism (“my religion is the only way, and any other way is bad”), which is what most people associate with the word “religion”, still seems wrong to me. Religious pluralism (“my religion is one path out of many possible paths up the mountain of religious truth“), is the orientation toward religion that seems most interesting to me.
A solution to agent foundations is equivalent to a technical formulation of the religious metaphysics that I believe are convergent across the mystical traditions of the major world religions (Judaism, Christianity, Islam, Buddhism, Hinduism), à la Perennialism, and would in effect amount to a synthesis of the scientific worldview and the worldviews of each of the world religions. I furthermore think that Chris Langan may have already found such a technical formulation.
A steelman of Arbital, that’s able to facilitate a process that converges toward a synthesis of the beliefs and the preferences of its users – eventually extending to the whole world – is crucial to my conception of a positive AI singularity.
A solution to agent foundations is necessary for an un-gameable implementation of such a steelman of Arbital.
A robust solution to global coordination involves robustly defeating Moloch, via the establishment of a decentralized singleton powered by an un-gameable steelman of Arbital.
This process of defeating Moloch and establishing a decentralized singleton is consistent with pluralist interpretations of religious prophecies about the end of the world (like The Second Coming of Christ).

Conversation 1 — April 6th 2024

Alex and Ben had gone on a walk to discuss religion, and decided to continue the discussion over a LessWrong dialogue.

Alex's steelman of the Second Coming of Christ

Ben Pace

Let's get into it.

So, I'm not sure if your position is:

Classical religions were doing lots of good and meaningful and important things
Or the same as (1) but with the additional claim that they're the best sources of these meaningful and important things, and so we should all study/follow them

zhukeepa

Much more like (1) than like (2), although I would also want to emphasize the bad and mind-killing things in (1). (Not sure exactly what you mean by "classical religions".)

zhukeepa

Some topics I would be interested in exploring:

how AI plays into the (Distributed) Second Coming of Christ(-Consciousness) picture I'd started fleshing out during our walk
the connections between forgiveness and overcoming Moloch
more details around my epistemic status around religions and spirituality (especially the kinds of claims that they're commonly associated with)

Ben Pace

I think I don't understand (1) (am up for poking at it). I think (2) sounds interesting. They both seem more specific than (3) so I prefer them.

Ben Pace

Well, maybe I'll ask a few initial q's about 1?

zhukeepa

Yes, that sounds good

zhukeepa

I think it's a broader topic that includes (2) as a subcase anyway.

Ben Pace

I'm going to forget all the specific lore of these old religions, so remind me whenever it's relevant, but... I'll just open the wikipedia page on the second coming of Christ... the main thing I recall is something like "he returns, and then all wrongs are set right, and all people go to heaven or hell".

Ben Pace

zhukeepa

Right, so the first thing (which maybe you're already tracking) is that the thing I mean by "Second Coming" is not Christianity-specific, and is more like the common denominator behind the eschatological prophecies found across world religions (I think Islam has the Mahdi, Buddhism has the Maitreya, Hinduism has the Kalki, Judaism has the non-Jesus Messiah).

Ben Pace

Just adding some quotes from wikipedia for context:

The Second Coming (sometimes called the Second Advent or the Parousia) is the Christian belief that Jesus Christ will return to Earth after his ascension to Heaven (which is said to have occurred about two thousand years ago). The idea is based on messianic prophecies and is part of most Christian eschatologies. Other faiths have various interpretations of it.

and

Most English versions of the Nicene Creed include the following statements:^[^{citation needed}^]
...he ascended into heaven and is seated at the right hand of the Father. He will come again in his glory to judge the living and the dead, and his kingdom will have no end. ... We look for the resurrection of the dead, and the life of the world to come.

Ben Pace

In Islam, Jesus (or Isa; Arabic: عيسى ʿĪsā) is considered to be a Messenger of God and the masih (messiah) who was sent to guide banī isrā'īl (the Israelites) with a new scripture, the Injīl (Gospel).^[85] The belief in Jesus (and all other messengers of God) is required in Islam, and a requirement of being a Muslim. However, Muslims believe that Jesus was never crucified or resurrected, instead ascending directly into heaven. Additionally, they do not recognize Jesus as the Son of God, as they believe God has no equals, but rather that he was a prophet.

and

In the Quran, the second coming of Jesus is heralded in Sura Az-Zukhruf as a sign of the Day of Judgment.
And (Jesus) shall be a Sign (for the coming of) the Hour (of Judgment): therefore have no doubt about the (Hour), but follow ye Me: this is a Straight Way. 43:61

Ben Pace

(I have generally taken very little interest in religions and was not aware that Jesus appeared as a character in the Islamic holy books.)

(Makes it seem more like fanfic.)

zhukeepa

I am pretty sympathetic to religious myths being compared to fanfiction. (I think I will get some flak from religious leaders for this lol.)

A core aspect of this common denominator (as I interpret it) that I'm tracking: the true, good, meaningful, and important things that all the religions are trying to point at become common knowledge in the world, and the global power structures (e.g. the justice systems, defense systems) get restructed to operate in alignment with this thing that's pointed at, with a corollary that the world operates in a way that's actually just and peaceful (perhaps not literally 100% so, but true enough to an extent that would lead the global situation then to feel qualitatively different from the current one -- kind of like a "phase transition" in which Moloch is mostly vanquished).

Ben Pace

This is too vague, you should say more concrete things about what will change / what will be fixed.

zhukeepa

Sure, gimme a sec to copy/paste something I've written in a Google Doc:

Mass psychological healing
Mass recognition that insatiable materialistic desires (e.g. for money, fame, power, and status) ultimately ground out in desires for love and belonging
Mass recognition of the ethical and metaphysical truths shared across religions
Mass reassessments of history, including mass acknowledgments of historical injustices, according to the standards of these ethical truths (reminiscent of the Last Judgment)
Fading of attachments to tribal identities (like nation, race, and religion)
A new form of governance involving an upgraded form of democracy, in which individuals’ conflicting preferences are synthesized (as in the parable of the blind men and the elephant) to determine how society should be run, including how resources should be distributed
Phasing out of participation in dysfunctional systems (e.g. factory farming and environmental exploitation), and phasing in of participation in systems more aligned with life
Misguided spiritual leaders (“false prophets”) trying to claim unique spiritual authority, and genuine spiritual leaders (the vanguard of the Distributed Second Coming) who recognize and respect each others’ spiritual authority, collaborate very effectively, acknowledge and accept their own and each others’ personal shortcomings, and show compassion to the misguided spiritual leaders

zhukeepa

Idk if this is the sort of concreteness that you're looking for.

zhukeepa

I also want to caveat that the type signature of this sort of thing is less like "something that is preordained to definitely happen" and more like "a self-fulfilling prophecy that might happen if enough of humanity believes in it".

Ben Pace

These seem like 8 good things (probably), we could chat about how real/plausible/attainable they are.

I don't know why you're talking about religions. I think a lot of irreligious people could write down a list of big things like this that they'd like to solve.

Why is Alex interested in religions in the first place?

zhukeepa

Yeah, the bit about why we're talking about religions seems really important to hash out. Can you name some of your biggest confusions around that?

Ben Pace

Well, okay. I'll frame them as questions for you to answer that would help me understand your perspective.

Why are you reading stories about Muhammad? Why aren't you telling me stories about Lord of the Rings or of stories by Dostoyevsky or Heinlein? They seem like better stories in many ways.
Why are you pinning your dreams to a phrase connecting Christianity/Islam? These institutions are totally epistemologically borked and their stories are pretty boring to read. Why don't you just start afresh?
To be clear I find some of the meaning you find in these stories to be real, I don't object that they're empty/meaningless.
What are 3 things you've personally gotten from the religious perspective on things that you've not gotten elsewhere? I think some of the discussion we had of forgiveness was interesting, and I could be led to believe that you find it more tractable to get this discussion from religious groups than from, I dunno, literary book clubs (that sounds fairly plausible now that I write it). It seems plausible to me more examples of "spiritual growth" (ugh I don't like that phrase) you've gained from exploring the religious texts would be in-retrospect the thing that has the most impact upon me.

zhukeepa

I suppose I could also start with my motivation for bringing religion in to the picture, which has two main components:

the first is that I think certain kinds of philosophical insight need to be incorporated for these goals to get realized, and I think these philosophical insights are the underpinnings of each of the major world religions. While I don't necessarily think that any particular religion is likely to be a particularly good gateway for these insights to a randomly chosen person, I do think these insights can be found in the religions if you look in the right places and interpret them in the right ways.
the second is that I think it is in fact possible to meaningfully coordinate with certain religious people -- religious pluralists in particular -- who are up for "reclaiming religion" into something less obviously shitty/absurd, and jointly crafting a positive vision for the future that taps into powerful tropes that are already embedded in the collective psyches of most of the world population.

Ben Pace

I think these philosophical insights are the underpinnings of each of the major world religions

I want to poke at this but I expect you will say things that are too vague (for me), so I am not actually going to try to engage with this directly yet (I'd rather hack away at the edges more first).

Ben Pace

the second is that I think it is in fact possible to meaningfully coordinate with certain religious people -- religious pluralists in particular -- who are up for "reclaiming religion" into something less obviously shitty/absurd, and jointly crafting a positive vision for the future that taps into powerful tropes that are already embedded in the collective psyches of most of the world population.

I think I want to wait on this for now, seems down the road.

What has Alex gotten from religion?

zhukeepa

Looking at your four bullet points. Taking a moment to think about it...

OK, for context, I think my real crux for engaging a bunch with spiritual/religious exemplars is that I somehow started taking seriously the hypothesis that letting go of all my resistances, resentments, judgments, etc. (tanha is the technical Buddhist term) toward everything and anything in reality is (1) actually where my CEV points (2) actually an easier way for me to live. I can maybe go into how I started taking that hypothesis seriously. But living according to this hypothesis is... a pretty different way of living from how most people choose to live (it REALLY goes against what mainstream culture recommends, for example), and insofar as I cared about getting better at living my life in this way, I've found myself most interested in reading stories about the historical figures who (according to me) have done the best job at living in this way.

I've heard some rationalists talk about how they only want to read rationalist fiction upon discovering rationalist fiction, because it just feels so hard to relate to non-rationalist fiction. What's going with me here feels vaguely analogous -- not in the sense that I find non-religious/spiritual stories uninteresting, but insofar as I find myself obsessed with learning how to be a better person in a very particular way, where it seems like the only other people who care about being better in this way are super into religion/spirituality... well, those are the people whose stories I want to read about.

zhukeepa

I think I'm going to elaborate on bullet point 4 now (around things that I get from the religious perspective that I don't see mentioned much elsewhere), besides what the deal is with forgiveness / grace / turning the other cheek:

this is sort of a corollary to the bit about forgiveness we discussed on our walk, but holding myself to the standard of "What would Jesus do?" in all of my close relationships (family, romantic partner, good friends, close collaborators...) has totally changed my paradigm for how I approach relationship conflicts. In particular, I find that there are way more degrees of freedom for navigating relationship conflicts when I try to be utterly uncompromising about disendorsing any place I might feel judgmental/resentful, even if it's "technically fair" for me to feel those ways, and even if most of mainstream society would back me up on feeling those ways. I think this is still pretty vague and high-level -- I'm happy to elaborate further, but I'd like to move on to other points for now.
I think I'm unusually good at both (1) acknowledging my personal shortcomings, (2) forgiving myself for them, partly because of the high standards of forgiveness I hold myself to (3) calling out others' personal shortcomings in a forgiving way. (According to me, a common misunderstanding of the point of forgiveness is that you "should do it" because "that makes you a good person", whereas I think the real point is that forgiving others goes hand-in-hand with forgiving yourself. Verbatim from the Lord's prayer: "And forgive us our debts, as we also have forgiven our debtors.") Because I am unusually good at these things, I am also unusually good at earning the trust of people I encounter, from a broad array of backgrounds.
The religious perspective gives me a lot more nuance on the trope that "what people ultimately value is love". I think there's a sense in which most modern adults are pretty jaded about romance, and in some sense have layers of heartbreak that disconnect from the original innocent and open-hearted love they'd initially felt as children / young teens. I think there's something of a generalization of this, wherein babies and toddlers are sort of "in love with" their lives, their families, and to some extent all of reality, that they lose touch with in the course of repeatedly getting "heartbroken" as they discover that their naively optimistic perceptions of reality aren't calibrated. In one sense, I think the "goal" of the spiritual path is to live in a way that embodies both the innocence, lovingness, and open-heartedness of infancy/toddlerhood, while simultaneously embodying the nuance and discernment of adulthood; and one only comes to reach this place by recognizing that a majority of our judgments and resentments and entitlements are defense mechanisms to cover up underlying senses of vulnerability, from having our "hearts broken".
I've also been updating in the direction of "family values are correct values". There's one very abstract sense of this, in which there are decision-theoretic grounds for honoring the processes that made you, like your family / your culture / evolution as a whole -- by doing so, you're acausally "causing more of yourself to exist". There's another more tangible sense of this for me, which is related to the previous bullet -- the more I've been able to forgive my family, the more I've been able to authentically connect with them in the sorts of ways that I had as a child, which in turn has gotten me in touch with the innocence and open-heartedness that I'd embodied as a child, and the depth of the unconditional love that my family feels for me, to the point where I actively look forward to calling both my parents every week. It's kind of hard to verbalize the magnitude of what this has been like for me... around grokking that so much of who I'd thought I was, my values and my ambitions, were actually just defense mechanisms because I'd felt heartbroken by my family, and then reconnecting with my family's unconditional love, and realizing that the love I was looking for had actually been present all along. I'm aware I'm still being pretty vague and unspecific here, and I'm happy to elaborate later, but I'll leave it as is for now. I'll just mention that there's an Islamic saying "heaven lies at the feet of your mother" and I feel like I now grok the kind of truth it's trying to point at.

Ben Pace

the more I've been able to authentically connect with them in the sorts of ways that I had as a child, which in turn has gotten me in touch with the innocence and open-heartedness that I'd embodied as a child, and the depth of the unconditional love that my family feels for me

I'd say the relationship I have with my mother (who raised me alone) is the most intense relationship of my life. (I don't mean dramatic, I mean intensely felt.)

zhukeepa

Mm, I can imagine.

Ben Pace

I think I'm unusually good at both (1) acknowledging my personal shortcomings, (2) forgiving myself for them, partly because of the high standards of forgiveness I hold myself to (3) calling out in a forgiving way others' personal shortcomings.

Are you saying that this is a general property of you or one that you gained from engaging with religious stories and thinking?

zhukeepa

I think this is a general property of me that followed from me "taking seriously the hypothesis that letting go of all my resistances, resentments, judgments, etc. toward everything and anything in reality is (1) actually where my CEV points (2) actually an easier way for me to live." I think religious stories are much more interesting when you interpret them from this lens, and I don't really know how to find them that interesting if you don't have this lens.

zhukeepa

AFAIU, in theory, every Christian and every Muslim is supposed to take this hypothesis seriously, and live by it uncompromisingly. In practice I think this is not really the case, but I think the mythologized portrayals of Jesus and Muhammad are exemplars of people who do live by it uncompromisingly, which is why I find it interesting to read stories about them.

Ben Pace

I'd say the relationship I have with my mother (who raised me alone) is the most intense relationship of my life. (I don't mean dramatic, I mean intensely felt.)

Literally last text from her (an hour ago) reads:

To be clear, my love for you is stronger than the raging & not unpleasant winds (46.3km per hour) that currently rock this little island where I live.

zhukeepa

Literally last text from her (an hour ago) reads:
To be clear,my love for you is stronger than the raging & not unpleasant winds (46.3km per hour) that currently rock this little island where I live.

That really warms my heart to read <3

Ben Pace

It's such a good line. And yeah I really like my relationship with her.

zhukeepa

That also really warms my heart to hear <3

Ben Pace

(As I'm reflecting on what you wrote, it would help me to know if there's something you're wanting from me in response.)

zhukeepa

Got it. Nothing immediately jumps to mind -- I think you asked good and resaonable questions, and mostly I was just trying to answer them.

Resentment, forgiveness, and acceptance

Ben Pace

Can you tell me more about how you try to live up to "What would Jesus do?" in your relationships? I'd like to understand that a bit better.

Ben Pace

I try to do what's good and right. And sometimes I ask myself "How would person X respond?" and that gives me ideas I haven't thought of. But I don't know how Jesus would respond.

I feel like he would do something unrealistically good and it would work out unrealistically well (e.g. and then I said to my muggers "go back to your fathers and give them all your gold" and then they did and wept and praised me and were saved) but I don't think this will work out IRL.

zhukeepa

In practice, it looks like asking myself: "Am I judging them or resenting them in any way?" If I am, I do not count that as part of my endorsed behavior, and I will try to let go this judgment / resentment, and relate with them from that place.

Ben Pace

I try pretty hard to live without resentment, and to relate to people without judgment/prejudice. I don't know that I'd endorse "it's always the wrong call with literally no exceptions".

zhukeepa

That's a huge part of it, at least. I wouldn't say that's the totality. (And, in practice, for me asking myself "What would Jesus do?" and "What would Muhammad do?" and "What would the Buddha do?" don't ever return conflicting answers, but I do sometimes get clearer answers from one question than others.)

zhukeepa

I think I would endorse "it's always the wrong call with literally no exceptions", which I think is super counterintuitive!! In some sense, I think this is the crux of Jesus's crucifixion -- he was not resenting or judging his crucifiers, and instead praying for them to be forgiven.

Note that this is still consistent with setting boundaries, or otherwise saying that you're not OK with some things.

Ben Pace

My guess is that it would help (me, at least) if you could tell me a time where you used this approach, though it can be hard to pull up specific instances so it's nbd if it's more effort than it's worth rn.

zhukeepa

One particularly salient example was when I was navigating some poly drama that I won't go into. Let's just say there was a lot of judgment and resentment bubbling up in me, for reasons that third parties agreed were quite fair...

Another particularly salient example is in relating with my parents, who I had previously thought of as emotionally abusive, for which I'd seriously judged them and resented them. My current stance is "they were emotionally abusive by Western standards, but also they were pretty much normal by Chinese standards -- see e.g. Battle Hymn of the Tiger Mother by Amy Chua -- which in some sense doesn't mean their behavior was morally OK, but does sort of lead me to feel like it's weird to judge them for not having been different, and makes it even feel weird for me to label them abusive". A crucial prerequisite for connecting deeply with my parents was being able to let go of the "emotionally abusive" label -- which Western culture was totally backing me on -- and forgiving them for the mistakes that they'd had no idea they were making, because they were just doing their best to parent us based on what they'd implicitly learned from their culture and their parents, who in turn did the same things based on what they'd implicitly learned from their culture and their parents, etc.

Ben Pace

(Responding to your first example)

In these situations a common heuristic I use is "How do I act in this situation so as to keep open the possibility of a strong and close relationship in the future?"

This can include:

Not getting worked up over the current conflict, because it could easily be seen (by me) as relatively minor in the scheme of things later
Making sure to be up front and open with them about issues that I have, and stand up for myself, so that later on I'll be able to honestly move towards them in friendship / alliance
Not say things in private that would cause them to be unable to trust me (if they later learned what I said)

Nonetheless it's not never that a situation occurs where I'm like "I can see no concrete way forward to have a strong and close relationship in the future, and it's best to just dissociate / move away." I wish I could do better and suspect it's a skill issue. Even then I aim to harbor no resentment or ill-will.

Ben Pace

A crucial prereq for connecting deeply with my parents was being able to let go of the "emotionally abusive" label -- which Western culture was totally backing me on

Sounds like it was a pretty good move

zhukeepa

I also think "I can see no concrete way forward to have a strong and close relationship in the future, and it's best to just dissociate / move away" can sometimes be the right call. I think the crucial piece is aiming to harbor no resentment or ill-will. (It's plausible to me that you're already holding yourself to pretty similar standards to the ones I'm holding myself to, which is partly why I'd remarked that I thought you had a high spiritual IQ :P)

Ben Pace

I personally taboo 'abuse' and 'trauma' and a lot of other words that seem to me overloaded, and too load-bearing for people's ethical attitudes.

(I keep a list of such words that I aim to always taboo.)

zhukeepa

Sounds like it was a pretty good move.

Yes, it was an extremely rewarding one!

Ben Pace

(I feel like I'm making too many assumptions too quickly with this next sentence, feel free to say it doesn't make sense)

I assume the relationship with your parents/family wasn't "fixed", and didn't "become the relationship you had previously wanted", but it became a good relationship with the people who were actually there? Like, the best version of the relationship that was possible?

zhukeepa

That does resonate. I would sort of say that it got "fixed", but it definitely didn't turn into the relationship that I'd previously wanted, and did more become a good relationship with the people who were actually there (both me honoring who my parents actually are, and them honoring who I actually am).

zhukeepa

Our relationships certainly aren't perfect now, but I would say that the core resentments that I'd had since a teenager that I'd felt hopeless about ever resolving for most of my post-teenage years have in fact gotten resolved.

Ben Pace

Even then I aim to harbor no resentment or ill-will.

One more semi-related note on this: I generally have a low opinion of people who become actively antagonistic with people they were formerly close with. I recall reading Person A text my friend Person B asking them to repair some damage in their relationship, saying something like "well you don't want to unnecessarily make enemies" and I was kind of appalled that in their world the default thing that happens when friendships breakdown is that you have 'enemies' rather than 'there is no longer any ongoing relationship'.

Ben Pace

both me honoring who my parents actually are, and them honoring who I actually am

Sometimes people say this and I kind of get it but I'm not sure what it caches out to. What does it mean to "honor" who your parents actually are?

zhukeepa

I generally have a low opinion of people who become actively antagonistic with people they were formerly close with.

Maybe as a datapoint around how I hold myself to my religious / spiritual standards: upon reading this, the thoughts that came up for me were "gee, it must suck to be those people -- it must be lonely and scary for them to have those expectations around their friendships" and "I fucking hate that too, screw those people" and "hmm, I appear to harbor some resentment for them, and will probably feel like I am screwing up for doing so... I aim to forgive them, and also to forgive myself for this resentment". Lol

Ben Pace

I agree, it is probably scary to be those people. My guess is that their lives are more dramatic than mine though, which is kind of enjoyable from some perspectives.

(Also it kinda sucks to be around those people!)

zhukeepa

Sometimes people say this and I kind of get it but I'm not sure what it caches out to. What does it mean to "honor" who your parents actually are?

Hmm, something like, there's a way your parents Actually Are, In The Actual Territory, which was super frustrating to me as a child because they were supposed to be Perfect People (TM), not Who They Actually Are... and then being like, "wait, they were never supposed to have been Perfect People (TM) in the first place, that was just a childish fantasy, they're just normal people like myself, why don't I start interacting with them based on Who They Actually Are, rather than my projections of them?"

I think there's also a flip side around parents seeing their children for Who They Actually Are, In The Actual Territory, rather than as a vessel in which for them to project all their hopes and dreams and whatever.

Ben Pace

Well, I actually want to argue with this a bit.

There's a great thing you describe, which is seeing the substance instead of the symbol you believed. "Oh, these people aren't the personification of safety and love and home that I believed they were when I was 6, they're actually an English teacher and a plumber who occasionally read books and occasionally fight though love each other."

At the same time, sometimes people do make claim to be the symbol I see them as, and should be held accountable for failing at it.

...on reflection my guess is that the error typically goes this way consistently with parents, but I think sometimes people are like "I am a politician and will represent you" or "I am a journalist and will speak truth to power" or "I am a scientist and I will publish true results" and they totally suck and don't do that and then you find out and it's not quite right to go "Ah, well, I was holding them to my ideals of who a politician/journalist/scientist is, after all they're just people" and it does make sense to be like "you betrayed these ideals and you are in debt to me/society".

zhukeepa

According to me, in that case, the person that they actually are is someone who is failing to be a person they are claiming to be, who should be held accountable for that. (Part of my improved relationships with my parents has been around having more candid conversations with both of them around their shortcomings, as well as vice-versa.)

Ben Pace

Alright, our time's up, let's wrap up here.

Seeya next week :D

zhukeepa

Yup, seeya. This was great. Also happy to respond async before then if there are further things you'd like to ask.

Conversation 2 – April 13th 2024

At the next meeting we recorded audio and had it transcribed, and then lightly edited. This was 2 hours, and Ben was quite underslept.

Ben Pace

What do you want to talk about? What is on this chain train?

zhukeepa

I feel like picking up where we left off could be a good place to start.

Ben Pace

Yeah, I could open that and re-read, skim it, and jog my memory.

What Alex gets from religious stories

zhukeepa

I remembered you asking something like why should we be thinking of any of this in terms of religion? It seems like there are a bunch of nice things that you want that seem good and cool and stuff, but why bring religion into picture? And then we talked about some particular ways I was interpreting religion and how I was interpreting religious stories through that lens and why I found the religious stories interesting. I think that's pretty much where we left off last time.

Ben Pace

Yeah. What do I think? So I think there are two things.

One, you can get lots of things from different stories, lots of different sorts of characters and archetypes and virtues and vices. But something about these stories is optimized for communicating something like, I don't know, maximal goodness is probably wrong, but is really with the intention of storing the ideal virtues and so on.

And then the second thing is there is a whole sort of institution built around trying to find meaning in life and trying to expect the good out of these stories. And that institution is not built around other stories by a factor of a thousand or a hundred or something.

And so both of these things are attractive about these stories relative to other stories like Lord of the Rings or Heinlein.

That was a pretty quick gloss. I'm interested in how much that felt like it pointed that the things that you think versus how the details wrong or totally missed some things you think or you just rather say-

zhukeepa

I think there's a particular thesis for what the good is that I think I find in religious stories that I tend to not find much elsewhere. And when I do find them elsewhere, it's often the case that the authors think highly of religion in some particular way. I think I somewhat articulated this thesis for what the good is as endorsing living in a way where you let go of all judgments and resentments and resistances, something in that ballpark, which is not super precise and part of what we could try to do here is make that more precise but...

Ben Pace

I mean, I feel like I pretty regularly aspire to this sort of unattachment that the Buddhist stuff and talk about and I feel like came up in the story you were telling about, with our chat about Jesus.

zhukeepa

Sorry, can you say that last part again? It came up in the Jesus story too?

Ben Pace

I mean just him on the cross just being unwavering in communicate that 'I forgive you all in my heart'. And you're saying that's a level of detachment of unattachment to hatred or adversarialness or so forth that is surprising and rare and interesting.

But I don't think other stories are like 'no, all our characters are attached and we think it's good'. I think lots of other stories also have characters that are aspiring to not hold onto hatred and anger and bitterness. He's not the only such character in storydom.

zhukeepa

I agree that they're not, the prophets aren't the only characters who are trying to live up to those standards. I do think they're interesting as exemplars of people who are successfully living up to these standards, at least when you read them the right ways. I think a lot of fundamentalist Muslims do not read the story of Muhammad in the right way, for example.

Ben Pace

Right. It's plausible we should try to do some comparative literature. Perhaps you can take a story you like and I take a story and we spoke about what yours has got the good things in it and mine doesn't and then I defend it. But also it's possible you should say more about the things that you get, the nutrients you're finding in the religious stories. Maybe just tell me another story you haven't told me.

zhukeepa

I could. But I feel like there's a crux for our worldview somewhere, and I don't have the sense that that's where the crux is going to lie.

Ben Pace

Okay.

Orientation toward death

zhukeepa

I'm trying to go into my sense of where the crux actually lies.

I think a big part of it has to do around the orientation toward death, where as a special case of resisting nothing, death is something to not resist -- which, again, doesn't mean I think you should just roll over and not give a shit about life, but I think it does imply that the "death is the enemy that must be vanquished" perspective is sort of off-base somehow. I'm still pro-life extension research, and I'm still pro-medicine, but...

Ben Pace

I mean, I think most of history I should have been like:

"Okay, I woke up. I'm 18, I'm an adult, I'm going to die. Everyone's going to die. There's nothing I can really do about it. It's time to grieve that now, and then live in such a way that I focus on the positive things I can change in the world that will last and will have longer effects than my life. And not particularly be attached my own legacy or whatever."

That seems like the approach I would try to take.

There's dozens of things that I would give my life for that seem worth it. I'm not especially like 'Oh, but my life it's so important', and I think there are probably other characters of fiction who can live this way. I can think of one I think in Worth the Candle who doesn't give a damn about their own life, just following on of the tradition and goodness. So I think my guess is I think there's a lot of characters that aspire to be unattached to their fear of death.

zhukeepa

Yeah. First thought is, that sounds right to me. Second thought is I think the spiritual traditions and the mystical traditions of the religions aren't just about aspiring to have that orientation toward death, but actually doing the work to let go of a bunch of the fear.

Ben Pace

Yeah. I have fewer threads on how to do the work, so to speak, or have a harder time pointing to like 'Hey, you're a person who's scared of death, here's the seven things you do, and at the end of it you'll be freed'. I don't know how to say that to someone.

And maybe the religious people have got a better handle on that. Probably Jesus would say some things. He would say 'go into the town and give both your shoes to the first person you see' or 'go and tell your mother you love her'. You'll tell you seven things and at the end of it you'll be freed from your fear of death. And then the characters in the bible did it and then they were freed, and it was a great story. So maybe he's better at that than me.

zhukeepa

So Buddhist enlightenment is often described as dying before you die. Obtaining enlightenment and overcoming the fear of death by seeing how the fear of death is actually predicated on certain confusions, I think are kind of the same thing. And on my understanding, that's basically what Buddhism is about. And I think there are analogues of this in the mystical traditions of other religions.

Ben Pace

I mean, to be clear, I wouldn't be like "and I have achieved this". I'd be like, "I occasionally notice that it would be good to achieve and that I aspire to it and that I don't endorse not having achieved it". And I try not to allow myself to be the sort of person that acts afraid of death, but sure I could be doing better on it.

zhukeepa

Yeah, cool.

Ben Pace

I remember one time a friend of mine was telling me about a great meditation experience they had. I was asking them why they decided to have kids and they told me that they'd gone to a Buddhist retreat and had a meditation on death for half an hour. It was about walking consciously through the experience after you're dead, of what people do with your body, and putting your body into the ground. And as your body dissipates, I think feeling that in your body. I don't know, I can't remember exactly how the conversation went, but after that they said that somehow as a result of that, they were like, oh yeah, there's just a couple of things I really care about. And one of them was having kids. And that changed it for them. And then I think, she found the recording of the meditation on death, and we listened to it together.

And I think it was a good experience getting in touch with that world. I think I don't really live in that world where I will be dead very much. I don't really work for it or work around it or orient into it or think about it very concretely. And I think it was upsetting at the time. I think it made me want to connect out with people I'd not seen in a long time, I think. Yeah, I can't remember. I felt some different things after it that were helpful. I might just text her and see if I can get a link to it. Pretty interesting. [Yeah, here's the link to it.]

What useful things do old religious stories have to say about how to live life well today?

zhukeepa

Okay, I'm still struggling to find our worldview difference because I feel like it sure exists. It definitely feels like a smaller worldview difference than I have talking with a bunch of other LessWrongers -- related to my point about you having a high spiritual IQ -- but it still feels like there is some worldview difference somewhere. I'm going to take another stab at it.

Ben Pace

Well, I have multiple different thoughts.

The first one came to my mind is a bit rude, but I'll go ahead anyway. I don't know exactly what you've been up to in the last five years, but it doesn't seem like it's directly going to prevent us all from dying. Whereas I've read HPMOR and aspired to think about the most important things and work on them, and I've been trying to prevent extinction of humanity for the last five years as my primary goal, and building better epistemic infrastructure with that as a key motive. So I don't know that your stories have helped you more than my stories have helped me. Anyway, that was a rude thing I want to say.

Let me give relatively weaker objections just because I have them and it seems good to mention them. I feel like the stories were written a few thousand years ago and there are people around today who know way more about human nature. You just get to do a bunch of surveys. You'd ask questions like Aella does, like do you cheat on your partner? How many years have you been together? And you just get to see the trend after ~15 years when 30-40% of monogamous relationships start becoming unfaithful. Before then they could be like being faithful just works and it's good. And broadly had way less of the relevant data and understand. So probably their stories are pretty dumb.

I have so much more information about how humans work today. I also understand evolutionary psychology. The religious leaders might've said 'anger is a demon that is sent to hurt us'. But I think anger often has a good game theoretic justification and it's often worth letting yourself grow angry when someone screws you over, and acting on it. The religious stories probably had some good stuff in it, but surely it is the case is you could do better now if you tried.

zhukeepa

Better on the axis of understanding human CEV?

Ben Pace

No, better on the axis of how to live well as a human. I mean morally.

zhukeepa

Yeah. My first pass is that the specific concrete recommendations for how to live well, that the religions were built around, I think were solidly good for the time and cultures in which they arose, but then get overgeneralized. And in general if you're like, hey, it seems like this religion recommends this particular way of relating to food or sex or whatever, and this was the best way of doing it, I would probably not agree with the religion about that.

Ben Pace

I don't know which parts you want to save. I keep being like, well, not this bit. And you're like, of course not that bit. So I'm like, which bits?

zhukeepa

I mean, being good to your family I think is a pretty timeless bit.

Ben Pace

Hey, my mum should have cut her family out many years before she did. Some of them mistreated her and stole money from her.

zhukeepa

Right. I mean, Jesus also says to cut off your family sometimes. There's a lot of nuance in the whole thing. I think the thing where Muhammad said "heaven is at the feet of your mother" captures a pretty timeless human psychological thing. And it doesn't imply that-

Ben Pace

Almost until when the ems come, when we become ems, we'll probably make changes.

zhukeepa

Sure.

Ben Pace

But I agree that whilst we're still humans, it is pretty profound.

zhukeepa

Cool. I think there is stuff around sex that religions understand pretty well that get overgeneralized and overly rigidified in ways that I think are lame. But I think there is something "sacred" about sex that actually does get honored in religions that don't get talked about much in modern culture. In particular, I think there's something "sacred" around having children, and evolution gave us sex to have children, which is what the whole deal with sex was originally. But the associations between sex and the sacredness of having kids, and all the implications of those associations, are rarely emphasized in modern culture.

Ben Pace

Well also, we invented contraception and now the connection between the two is meaningfully severed in a bunch of pragmatic ways.

zhukeepa

Agreed. And I think a bunch of the cases against sex that were present at the time the religions got formed are much weaker now.

Ben Pace

All the diseases and the kid, the child-rearing...

zhukeepa

Yeah. I'm with you there. And still... if you're in a culture where 90% of the associations with sex are totally decoupled from having kids, I think that actually has bad consequences for the culture. I think there's something that's just pretty bad about hookup culture, in terms of people's long-term happiness and fulfillment, and certainly for having families.

That is the kind of thing that I think religions are more tuned into -- not that I think the super rigid recommendations from religions are necessarily better than hookup culture.

Ben Pace

Yeah, yeah.

zhukeepa

But zooming out a bit, I think it's useful to clarify that when I say religions have figured out how humans should live or something, I don't mean in the sense of like, yeah, the Jews have these laws, and these laws are human CEV. And the Quran says you should do this at such-and-such time, and blah blah. I'm also like, no, that's also --

Ben Pace

Yeah, yeah, yeah. I just want you to argue to me what's good about it, and tell me the thing that I'm missing.

zhukeepa

For some reason, the thing I'm now wanting to bring up is that I think the thing that's good about religions is also present in a lot of spiritually developed indigenous cultures and traditions. I don't think the good thing is uniquely found in religions, and I'm saying this mostly to clarify the type signature of the kinds of things I'm saying in favor of religion.

Ben Pace

Nonetheless, would the best place to look for it is in some of the stories about the religious prophets?

zhukeepa

I think the prophets are particularly good exemplars for a lot of what the good stuff is. And also, I think the good stuff is meaningfully present in numerous indigenous spiritual traditions as well, even before they were touched by any of the major world religions.

Ben Pace

I think I would like... I don't know. Maybe we should compare one of your prophets to the Comet King from Unsong who's sort of-

zhukeepa

I haven't read Unsong.

Ben Pace

Let me just check. I think I have this blog post. It's got some relevant quotes.

zhukeepa

I bet none of the Unsong characters would forgive their crucifiers while getting tortured to death.

Ben Pace

I'd be interested in you, maybe, I don't think it's quite again exactly the thing you want, but I think I would be interested in you reading some of the quotes from Unsong, about the Comet King. All right. Go on, ask me a question first.

Forgiving yourself, and devoting your life to something

zhukeepa

When we were on our walk, I mentioned something about how part of the deal with Christ is grokking that every aspect of you can actually be accepted and forgiven. Even the parts that you've buried so deep inside you because you're too afraid to acknowledge their existence, because it just feels like there's no way they could possibly be accepted or forgiven.

Ben Pace

By the way, I think one of the things that I find dissatisfying about all the religious exemplars is I feel like the people who are most into it are the people who do nothing else in their life, other than sit and forgive people or whatever, and that seems kind of utterly pointless. I think, probably, as you're trying to imagine people, pick people who actually do anything in the world using these tools.

zhukeepa

For what it's worth, I think Martin Luther King and Gandhi are examples of people who had done things in the world inspired by religious exemplars.

Ben Pace

I would agree with that. I more meant the Buddhist who just meditates their whole life, but-

zhukeepa

I see.

Ben Pace

Or even the priests, I guess. But maybe they do things, maybe I'm giving them a short shift. Sorry, I interrupted you. Please say that again, I was interested.

zhukeepa

I would say that part of the deal with Christ was grokking that every aspect of your being could be forgiven. And I remember you making a remark in the form, like, "Oh, I could imagine why someone might devote their whole life to this, if they understood this fully, or felt this fully."

Ben Pace

Yes.

zhukeepa

That feels like a pretty good inroad to worldview-bridging. That thing that someone might devote their whole life to gets at the core of what seems good to me about religion.

Ben Pace

I mean, it'd be very good if everyone did that. That would be great.

zhukeepa

Everyone did what?

Ben Pace

That would be a worthy thing. Everyone came to fully, actually believe in forgiving themselves, and that they could, and that others would, and that it was okay to forgive your own self, all the different parts of it. That would be very different than the current world, and it would be better than the current world, is my guess.

zhukeepa

Cool. Before you continue, can I just add another crucial piece of that picture in my head? Forgiving the self and forgiving others go hand in hand for Christianity, and in general.

Ben Pace

Yeah. It just seems like... I don't know, it feels like there'll be a lot less friction in doing anything good, in making things better. That's probably it. I don't really know how to describe it.

zhukeepa

Cool. This feels very juicy in my belly, by the way.

Ben Pace

What? Which bit? What? I don't know what...

zhukeepa

Oh, just the line of thinking you're going down, that we're exploring right now.

Ben Pace

Cool. I didn't quite understand why, but that's fine. Anyway.

zhukeepa

A world in which everyone is actually living in this way gets pretty close to the core of the thing I'm trying to gesture at when I say "Second Coming of Christ".

Changing the minds of all of humanity / steel-Arbital

Ben Pace

Yeah. I don't know. Go take one person and make them have this experience. Cool.

Coming to me, being like, "Here's the plan I'm going to make 8 billion people have this experience."

Eh, the point I'm making is too weak. I'll have to think of a better argument.

All right, so we've engaged with the steelman of this first. I don't know. I think also... if I was like, "How do I do this?" Step one would be to improve the epistemology, and to make sure that words mean things, and to build LessWrong.

I would still be like, "Look, I need to tell everyone what's available and communicate to them." You know what's messed up? Words don't mean things anymore, so when you say things, people go, "Ah, you're on the tribe of those people who say those words." I'm like, "No, no, no, I meant something important." And they go, "Oh, you're the tribe of people who insist that they're not playing language games—"

zhukeepa

Okay, this feels really juicy to me, because I largely agree, actually. My current conception of the good thing that happens with AI is pretty much a steelman of Arbital.

Ben Pace

Yeah, Arbital would be great. Maybe you should build it.

zhukeepa

I think... Have you heard of UpTrust? Jordan Allen's attempts to do something like Arbital?

Ben Pace

Someone mentioned that word to me yesterday, or today, and I remarked that it would be a funny 3rd axis of LW voting. Upvoting, agree-voting, and up-trusting? Anyway, that was a dumb joke. Someone mentioned UpTrust, but I didn't know what it was.

zhukeepa

Okay. I used to work with him on it. We had a co-founder split, and we're still friends, but talking with him about the idea did actually help shape my thinking around what a steelman of Arbital might look like. The steelman of what Jordan's trying to do, and the steelman of Arbital, pretty much point to the same thing in my head.

The thing that Jordan is tracking way more than I think the original Arbital idea was tracking, is that it's kind of misleading to say there is simply one objective truth, and everything that isn't that is just wrong. Different perspectives can have different kinds of truths, but that also doesn't mean that all of them are equally valid.

In other words, there's a question about how to think about truth in a way that honors perspectivalism, while also not devolving into relativism. And the way Jordan and I were thinking about this, was to have each filter bubble -- with their own standards of judgment for what's true and what's good -- to be fed the best content from the other filter bubbles by the standards from within each filter bubble, rather than the worst content, which is more like what we see with social media today.

And if this were to happen, the end result would basically be a process of coherence of the filter bubbles, and mutual understanding, and common knowledge actually being possible of what's true and what matters.

Ben Pace

I once again want to say that I've been trying something remotely at all in this problem area with LessWrong, and it has been very difficult, and not like, you know...

zhukeepa

I think a robust solution to this is as hard as the AI alignment problem. I think LessWrong is doing a very good job for a particular filter bubble.

Ben Pace

But even within it, I think there are a lot of people with very different standards, and I don't think we're doing a great job of... We're just like, "You're all in the same room, deal with it."

So, yes. So, it would be good if people's perspectives were made coherent, and their standards were made coherent, and they were able to collectively think within those perspectives, and then those perspectives were encouraged to interact with each other in a reciprocal and cooperative way, as opposed to a maximally antagonistic way. As such has happened on places like Reddit and Twitter and Facebook.

zhukeepa

Yes.

Ben Pace

Those all seem like laudable goals. We could... I forget how that's relevant.

zhukeepa

I was trying to say where I agreed around reason on the internet being possible at all, being pretty central to what I think of as the core issue.

Ben Pace

I mean, I think... I like believing in honest communication, as being able to be the key ingredient to... I'm trying to think of the right religious term. A savior or something, or... No, that's the Bible. Anyway, you know, things working out great.

zhukeepa

Yes, yes.

Ben Pace

And the thing you'll do is say to people "By the way, forgive yourselves. It works. Look, I did it. Here's the story of a guy who did it. You believe him." And then they'll go "Ah, wow, I could do that." Great. That's a nice idea. I would be up for trying telling people that one if we got to the place where people believed words meant things.

I also have other ideas, like, for God's sake, it's all prediction markets. Just legalize them, everything will go better. A few other things like that. But I don't know. I do think that the religious one-

zhukeepa

Can I just name the central place the religion stuff comes into this picture for me?

Ben Pace

Okay.

zhukeepa

Totally with you about believing in an honest communication. I think there's trickiness because the word "honest" is pretty loaded. I think the meta-context for the conversational container, in order for the whole thing to actually end up being robust, needs to be forgiving of what seems like dishonest communication.

Ben Pace

What did you just say? You said that it won't seem honest to everyone, so everyone needs to have a high level of forgiveness in those situations?

zhukeepa

That's not a bad summary.

Ben Pace

What is the thing? What's a better summary?

zhukeepa

In order for the communication context to stay robust, without just decaying or devolving, people need to forgive what they see as dishonest communication.

Ben Pace

They need to be able to.

zhukeepa

Under the nuanced version of forgiveness. Yes.

Ben Pace

Not expect to be taken advantage of, and to believe that the person is trying to act well, even if they sometimes really screw up and hurt someone? Maybe you disagree with that last clause?

zhukeepa

Nope, I don't disagree with that last clause.

Ben Pace

Yeah... something about this conversation's annoying. I can't figure out what it is.

zhukeepa

Let's try to figure out what it is.

Ben Pace

I don't know. It's good of you to offer, but I think I just at least want to note it to myself so I can remember it later. I don't know. Mostly, probably, just that we're moving through a lot of big, difficult, and, vague topics. Not vague, but topics that are hard to be precise about quite quickly, and that's not how I normally interact.

But that's probably okay... I don't know. I think I feel a bit worried that somehow I'm... I think I feel a bit like I'm not saying anything worthwhile or interesting, and you're probably disappointed in the conversation, but that doesn't seem to match up with the things you said.

zhukeepa

Yeah, that's not my experience of you. I usually find it helpful to talk with people who aren't in "Alex's crazyland", and see which things land and which things don't.

Ben Pace

Good. You have some dreams of how to save humanity. I don't know, I think I maybe get your things a bit... But actually do think that they would help a lot. Everyone's lives do seem kind of... Sorry, I'm not quite supposed to say that sentence, but I was going to say, everyone's lives do seem kind of meaningless, and they could just not be. That's not quite what I think, but I do feel like everyone could be a bit more full of magic if they wanted to be.

zhukeepa

Yeah, and I'll also reflect back that the thing that would be the "savior" being infrastructure for good discourse, I'm also super on the same wavelength with you about.

Ben Pace

So I'm sorry I keep bouncing off and being slow for some reason and maybe it's because the topic is emotional and difficult. I'm not exactly sure why. So anyway, I have some thoughts, but I don't know what's up anyway.

zhukeepa

That's fine.

How Alex thinks about changing the world

Ben Pace

So I feel like the worldview crux is less interesting than us talking pragmatically about how to supposedly cause your steelman of the Second Coming of Christ to come about. It seems like you have a thing that you want to do, and I like some parts of it and see enough of it that I...

It's more relevant to chat about how to do it, but I kind of feel like the structures we have for talking about plans are lame and will suck, but it's probably worth trying. And then I'll notice how they suck and then I can change what we're talking about.

zhukeepa

Yeah, that sounds good.

Ben Pace

Whenever effective altruists talk about plans for changing the world, I have a feeling like everything they say is a waste of time. That's not literally true, but it feels true and it kind of is true, anyway, and so I kind of feel like you and I don't have scripts for talking about how to change the world and so we're going to try and I'm going to be super triggered, but it's not quite true, but kind of by all the ways we talk. But we can try and then I can point out.

zhukeepa

I like this direction.

Ben Pace

Yes.

zhukeepa

Okay. First, I don't think of myself as trying to save the world. I think there are forces in the world that are leading it to try to kill itself, and there are forces in the world that are trying to save the world. And I see myself as trying to do my parts to differentially amplify the forces of the world that are leading it to save itself. But I'm not thinking of myself as a world savior. I'm thinking of myself as a guy doing his part.

Ben Pace

Yeah. That's good.

zhukeepa

And so I think what I want to then do is describe what I think it means for me to do my part, with the background context that, in my background models, the way the world gets saved is via enough people doing their parts that the world can actually come together and land in a totally new stable equilibrium.

There's the technical front and the coordination fronts, which maybe isn't a good decomposition of how I'm thinking about things. It's kind of a carryover from how I used to think about the AI problem. There's a technical AI alignment problem and there's the AI coordination problem.

The coordination front looks something like reconciling the world religions, and atheism, via a common shared vision of the future that we can all build toward together. And the reason why I mention messianic prophecies is because those are positive visions of the future that are already embedded in large swaths of the global population, which I think also have interpretations that are mutually compatible.

Ben Pace

Oh, I think that's a third reason to be interested in the stories of religion in so far as everyone's already heard of them to some extent.

zhukeepa

What were the first two reasons?

Ben Pace

There were written in order to portray people aspiring to maximum virtue and there are institutions built around trying to find meaning and goodness in these stories in particular, which there are not for Unsong and HPMOR and Lord of the Rings.

zhukeepa

I see.

Ben Pace

But the third one being everyone knows the stories, which is pretty... That might be the most important one.

zhukeepa

That feels pretty lumped with the second one in how I think about it.

Ben Pace

It doesn't to me, I think there are cults, there's probably fandom, maybe I can't think of a good example. It seems plausible to me that are sub-communities that are really interested in certain stories and finding meaning in them. But the rest of the world's not got it. So much less worth investing in. But anyway, not important or not very important.

Coordinating with the rest of the world sounds annoying

zhukeepa

I do think that if this kind of reconciliation happens, that would be a big deal in the world, and whatever's at the locus of this reconciliation would be an obvious Schelling direction for what humanity should do.

Ben Pace

You're reminding me of the rest of the world and I do find it annoying how... I'm going to say a bad version of the sentence. I do hate how stupid most people are. That's not quite right. That's not very-

zhukeepa

Something to forgive!

Ben Pace

What did you say?

zhukeepa

That's something I've been learning to forgive! How stupid most people are.

Ben Pace

Hehe. I think I haven't really looked at it. I think to forgive it, I first have to look right at it and I think I hadn't been looking at it for most time. I was like, "Oh, most people are probably fine or smart", or I just didn't think about them. And now I'm looking at them more and going, "Oh dear," and I'm seeing all the problems. I can only forgive after I have really come to understand it.

zhukeepa

Yeah, that sounds right.

Ben Pace

Otherwise, it's going to keep hurting me and then I'll get bitter again. I'd be like, what the hell? And then I won't expect it and that's painful. Anyway, just to be clear, I don't dislike people for being stupid, but it's very depressing and upsetting when I see how stupid they can be.

zhukeepa

It's very normal.

Ben Pace

Actually, sorry, it seems likely to me that most of the smart, nerdy-ish people experience this. I don't think I've chatted with them about it. I've been pretty bubbled myself. Just everyone around me is at least a standard deviation above average, if not two or three in terms of IQ and I forget about the rest of the world.

zhukeepa

Yeah.

Ben Pace

I forget that there's so many people that are standard deviation below average. I can't imagine it. It seems uncomfortable. I would not like it.

zhukeepa

I know what you mean. Are you having an experience of being like, wait, the median SAT math score score is 500—

Ben Pace

Oh, 800.

zhukeepa

The median SAT math score out of 200 to 800, is 500.

Ben Pace

Yeah.

zhukeepa

When I realized this, I felt shocked about my difficulties in empathizing with the median human.

Ben Pace

It must really suck to be so disconnected from very simple tools for understanding the world like quick multiplication and order of magnitude estimates.

zhukeepa

Yeah. I have also made some friends with some low IQ people, and that's also been good for my soul because I really liked them and I learned a lot from them. Like how to be more relaxed, and how to savor life.

Ben Pace

I'm sorry I cut you off.... You were in the middle of something and I lost the attention. You were saying "I want to assimilate the world religions and atheism into some better vision of the future for everyone."

zhukeepa

Something more like, produce an intellectual foundation that can unite or reconcile at least the world religions and science and articulate a positive vision of the future that's consistent with the existing positive visions of the future that are present in their eschatologies.

Ben Pace

This seems interesting to chat about for a while, how to do that. Before we do that or instead of doing that for now, I don't know. Anyway, you also were like there's a technical component. Do you want to just say a sentence about that?

zhukeepa

Yeah, which is actually coming up with the intellectual foundation.

And also figuring out how to build tools that actually enable the kind of robustly good discourse that could save the world. I think of this robustly good discourse as a meta-tool that will differentially empower the world-saving forces in the world over the Molochian, suicidal forces of the world, and I think they are very non-trivial technical challenges in building the infrastructure for this discourse.

Ben Pace

Yeah. I think, wait, I should say, I'm sorry, I'm tired, so I'm a bit quick to be annoyed or something in a way that's not about you or anything. Sorry, the second one again was you want to build better tools for good academic discourse and maybe you'll be able to talk about these ideas.

Oh, no, it was also you wanted to build the-

zhukeepa

The intellectual foundation for the coordination, which I think will also be the intellectual foundations for building the tools for the good discourse.

Ben Pace

Yeah. I don't buy that those two are that related.

zhukeepa

Yeah, there is a link there. I can make the argument for them being related.

Maybe religious pluralists are sane?

Ben Pace

No, I think the thing I'm more interested in, supposedly we should pause and come return to it in a week or something, is... The first thing you said was you would like to unite the religions and atheism and these sort of underlying stories and archetypes and virtues and so on.

zhukeepa

Yeah.

Ben Pace

I dunno, it kind of feels tasty to just try and do that together for a bit.

zhukeepa

Cool.

Ben Pace

Or it would be fun to do it or it would be meaningful to try.

Can you tell me something you've maybe been thinking about there about how to... any particular thread of the religious stories and atheism that you been thinking about how to reconcile?

zhukeepa

I would like to share that a big part of my current thinking around this came from attending the Parliament of World Religions, which is pretty much the main conference about reconciling religions.

Ben Pace

Where is it? When's it ran? Who runs it?

zhukeepa

I'm not sure who runs it. I went last August in Chicago. I think it's once every two years or so these days.

Ben Pace

How many people are you talking, a few hundred, a 1000?

zhukeepa

Like 10,000-ish I think.

Ben Pace

Oh, that's quite big. Oh, I forgot. This is religions. There are 8 billion people, so it's quite easy for 10,000 people to show up to a thing.

zhukeepa

Yeah. And all the big official events were vanilla and didn't feel like they were really saying anything. It felt like they were saying "yay! unity of religions!" in a way that seemed kind of lame and empty. But in the back rooms there were lots of religious leaders who I had really, really good conversations with about reconciling religions.

One thing I learned was the concepts of religious exclusivism versus religious inclusivism versus religious pluralism. Exclusivism is like, "my way is the only way and anything that isn't my way is bad", which is most Christianity and Islam. I think when most people hear "religion", they associate it with religious exclusivism, because that is in fact how most religious people think about their religions.

Religious pluralism is the diametric opposite of that, which says "my religion is one path out of many possible paths up the mountain of religious truth". I talked to a Catholic priest at the Parliament who'd said something like, "there was a girl I know who started out Christian, but had a really bad experience with Christianity, but then found something spiritually appealing in a Hindu temple, and I encouraged her to find God in Hinduism", or something like that.

Ben Pace

I feel like this is nice. I feel like my sense is the religions don't really endorse this. The Pope won't say this and the Imams won't say this.

zhukeepa

So I met imams and priests who said this at the Parliament. There was a room for Catholics, and the first thing I saw in their room was a pamphlet which said "Free Quran reading". And I grilled the people there about whether that was heretical, and they were just very straightforwardly like, "No, this is not heresy. This is not a mainline consensus view among Catholics, but our views here are actually strictly within Catholic doctrine."

Ben Pace

Sorry. By free Quran reading you mean come to a session and we'll read the Quran to you and teach you about it?

zhukeepa

I think so. Yeah. And it was right next to the Muslim room.

Ben Pace

I am confused by this. I feel like the institutions of the religions couldn't put up with this shit. I'm confused by it. All the books don't they say, "I shall have no other God before me but me," or whatever?

zhukeepa

Yeah. But the God might be the same God across all the religions. There's a thing where when Jesus is like, "I am the way and the truth and the life", some people interpret that as "the kind of consciousness that Jesus embodies, which some other people like the Buddha also embody, is the way, the truth, and the life". And others are like, "No, it's specifically Jesus himself, and exclusively so, and anyone who says otherwise is wrong."

Ben Pace

I know in so far as this is true, it's very strange and important, but I don't currently believe it, but it is.

zhukeepa

I can tell you some more facts in this direction that I can just send you links about.

Ben Pace

Yeah, that seems good.

zhukeepa

A few months ago, the Vatican released a statement calling Jesus and the Buddha great healers.

Ben Pace

What is happening? Are these people losing their edge?

zhukeepa

A lot of hardline or Catholics think so. They're like, "This Pope is the antipope."

Ben Pace

Didn't they want to kill all the Muslims and the Jews? Why are they saying that they're great healers?

zhukeepa

Because religion pluralism is a thing. The pulse I got from talking to some of the pluralists there -- it's definitely a biased perspective -- but the pulse I got was like, exclusivism is waning, it's losing popularity very fast, and pluralism is on the rise.

Ben Pace

No, I understand that. I just want to... Is it just the case that all the hardcore Catholics are like exclusivists and all the lukewarm Catholics are pluralists or other hardcore people who are also hardcore pluralists? Is it just a straight correlation with how strongly you're committed to your-

zhukeepa

The Pope seems pretty open to pluralist views, it seems.

Ben Pace

Look, it's very surprising and interesting. I don't quite know what I make of it yet.

zhukeepa

I want to give an articulation of what this really felt like for me.

Ben Pace

The conversation's really annoying.

zhukeepa

Don't worry about it.

Ben Pace

I don't want to, I don't know what's up with it? I was like, oh no, we're not still talking about God, are we? This has been going on forever and sucks.

zhukeepa

So I was going to give you an analogy. There's a thing where the good scientists all know that science as an institution is broken in a bunch of ways. For example, most published studies fail to replicate. Most published studies-

Ben Pace

Yeah. It's awful.

zhukeepa

Yeah. But when the median person thinks about science, they don't think about the opinions of the good scientists who understand how broken the whole thing is.

Ben Pace

I know, I know.

zhukeepa

Yeah. The experience I had being at the Parliament of World Religions was like finding the religious leaders who are the analogues of the good scientists, who were just like, "Yeah, most mainstream religion is totally broken. It sucks for exactly the reasons the skeptics says it sucks." And I was like, "Oh, okay." And then suddenly-

Ben Pace

That's a crazy analogy. That'd be pretty interesting if true.

zhukeepa

Yes.

I was telling one of the Catholic priests there about my experience of Jesus during an ayahuasca ceremony and he was just like, "I don't know what ayahuasca is, but the story you told sounds super legit and you are super lucky to have had that experience at such a young age. I've only had this experience after decades and decades of going deep into Catholicism and all the rites and rituals. All the doctrines of Catholicism are really about having that kind of experience. And you just had it directly." And another one who heard it was just like, "Whatever you're doing, Alex, keep doing it. It sounds like you're on the right track." And I was just like, "That feels great."

And I've also befriended an imam who's spent a decade living in Mecca. He is on the World Council of Muslims for Interfaith Relations, and he knows a bunch of the head imams in the main mosques in Saudi Arabia. And he'd heard about Nick Bostrom, and one time we were just talking about superintelligence and he was like, "Yeah, if an aligned superintelligence gets built that could be the messianic figure of Islamic prophecy, the Mahdi." He said that to me before I said anything of that sort with him. He is a very smart guy.

Ben Pace

Why do you say he is a very smart guy?

zhukeepa

Because he is. He has a high IQ.

Ben Pace

Right. Cool.

zhukeepa

So these are the kinds of people I have in mind when I'm thinking about religious leaders from the world of religion I want to be allied with, in terms of reconciling the religions.

Ben Pace

Yeah, it's a nice idea. As I say I feel very... Anyway, you want to reconcile them. I still, I'm not quite, what would it look like? What's a property of the world that would be different if you had succeeded in your goal? How do you know if you successfully reconciled them?

A mathematical synthesis of religious metaphysics?

zhukeepa

The proximal thing I'm trying to target is creating a mathematical Rosetta Stone for the world religions.

Ben Pace

Okay. I guess I thought you wanted to make a new religion that was better, but now it sounds like you're like, "No, we just should be able to work together."

zhukeepa

What this "new religion" would be is just taking all the good correct parts of all the old ones and making them precise, so I don't really think of it as a new religion.

Ben Pace

Sure. But you've got to write new books.

zhukeepa

Maybe. I think there are lots of people doing that already.

Ben Pace

I don't understand. I asked if you were saying "Come together imams and priests, quit your current jobs, we're going to make you priests of a new religion that's not got a bunch of metaphysical claims in the books that are false, and has stories that are actually appropriate for the current civilization, and so forth." It sounds like you're saying "No," so I just don't know what you would say "Yes" to.

zhukeepa

Let me say one concrete thing I want to say "Yes" to, which is being able to ground the apparently contradictory metaphysical claims across religions into a single mathematical framework.

Ben Pace

Look, a bunch of the metaphysical claims are there just to give people stuff to memorize and signal tribal allegiance. Not all are attempting to be deep truths about the human spirit.

zhukeepa

Yeah, I think that stuff would largely not end up in the math. But stuff like, is there reincarnation, or is there heaven/hell? I think these concepts are just under-specified, and I think most people's intuitive conceptions of these are wrong. I think they can be good metaphors for pointing at something deeper and more nuanced and complex, and the thing I want is a formalization of the thing that's more nuanced and complex.

Ben Pace

I don't know why you need anything formal.

zhukeepa

Because I think there's a there there, and the best way to talk about it is formally. When the religions talk about afterlife stuff, I think there's a there there.

Ben Pace

I understand that... I don't know, the formal thing feels like a red herring to me. Like, maybe Scott Garrabrant one day will come to us and be like, "Oh yeah, here's some math, how to think about the death of your agent and the values continuing in the world." I would go, "Thank you, Scott Garrabrant." But I didn't need it to be able to get the right attitude towards the world. It helps, but I didn't need it.

zhukeepa

I don't know that it's strictly necessary either, but I think it would help a lot in thinking through things more clearly (similarly to how FDT helps me think about decisions more clearly, for example), and in providing bridges between religions.

Ben Pace

But also, I am annoyed again. Is there heaven and hell, or is there reincarnation? These are very different. Concretely, they're very different stories of the world. They can't both be the case.

zhukeepa

And so my story is that if you ask, is an electron more like a particle, or is it more like a wave? The math kind of just answers this question. And if you don't have the math, if you just try to debate it in English, particle and wave seem like they imply pretty different views of what's there. But when we actually get precise about what's meant by wave or particle, it's like, "Oh, it's just both, in this weird way that's hard to understand when you don't have the math." And that's pretty much my sense of what the deal is with reincarnation vs. heaven/hell.

Ben Pace

So, let me point to something. Often I will make a decision that is locally costly in order to signal how I will behave in future counterfactual worlds. I will not take a bad deal, I will instead take no deal, so that people know they can't take advantage of me and force me to accept bad deals. It hurts to take no deal right now, but then in the future people will evaluate the counterfactuals and see they'll be better off giving me a fair deal than an unfair deal.

So you have these conversations about what would happen in worlds that are not this world, and that's why you're making the choices you are in this world. And it can sound kind of mystical or confusing, to be making choices based on what happens in worlds that don't exist in our physics and will never happen. And so sometimes you might tell people different made-up stories or approximations that sound a bit mystical in order to get them to make the right decision. You might tell someone that whenever they accept an unfair deal it hurts their mothers' heart in heaven, and they don't want to do that, so they shouldn't accept unfair deals, and they won't, and that will go well for them, but the reason is just that their counterparties do counterfactual analysis and notice that a bad deal means no deal.

That's not an amazing example but it gets the point across. There are lots of stories you might tell in order to approximate the math, and your religious stories might be like that. "What we tell everyone is that in the future, Christ will return and all judgments will be set right to wrong." Or we tell everyone, "You're going to live a life repeatedly, and each life, it'll be as good or as bad as you were in the last one." And this causes you to behave in a way that you would want to, on reflection, but it's just harder to give the actual reasons.

And then you might tell the people it is actually true, because people are too stupid to take nuanced specific stories. This is what I want you to be saying, and not bullshitting with statements like "Ah, yes, they're all true in their own way."

zhukeepa

I'm on board with there being different stories that aren't ultimately true, that are all trying to point to the same core truth thing, that the math can let us talk about precisely.

Ben Pace

I would be annoyed if, at the end, your Rosetta Stone says "They're all true, and here's how they're all reflecting some deeper truth or something." They're not all true. They're all false.

zhukeepa

I think it would be more like "here's the correct way to interpret these stories, in contrast to these other interpretations that a majority of people currently used to interpret them". Like, a lot of people interpret hell as like, "Oh yeah, it's a place where there's going to be lots of heat, that people will be stuck in for literally infinity years." That's just wrong. That's not making it into the Rosetta Stone.

Ben Pace

So if you worked on this reconciliation for the next year, or you'd made efforts to cause it to happen... I don't exactly know how you view your role in causing it to happen, or would be something you'd be satisfied with having happened over this next year, as a result of things you did. Or, what things could you do that you'd be satisfied with over the next year?

zhukeepa

For this one, for the nearer term, I'm picturing stuff like hosting retreats with AI alignment people and interfaith leaders.

Ben Pace

But it doesn't seem to me like it's got anything to do in the short term with alignment research, do you mean, I don't–

zhukeepa

Except to the extent that there are alignment researchers who think the right way to think about alignment involves the thing that's shared behind all the religions.

Ben Pace

I don't really get it. They don't seem directly connected to me. It seems like it would plausibly be good to reconcile all the religions, and give people truth and virtue and meaning to aspire to correctly with good stories. It seems to me like a different project than the AI alignment project.

zhukeepa

I think solving agent foundations is a lot like–

Ben Pace

Solving agent foundations would help with everything. I understand. But I still don't actually think that Scott Garrabrant talking to some imams is going to help him make a better sub-agent category theory.

zhukeepa

I do think Scott Garrabrant talking to me, who has talked to imams and Chris Langan, does help him.

Ben Pace

That may be true, but I make the relatively strong claim that you are not a central example of the imams and whoever the other guys are.

zhukeepa

Yeah. I do just also want to mention that my current model of Chris Langan has a solution to agent foundations in his head.

Ben Pace

Okay, sorry, he's not an imam, he's a weird, okay. I don't know enough about him. I didn't read that post, I'm sorry.

zhukeepa

The Rosetta Stone that I'm gesturing at, I think Chris has a version of that in his head. And so–

Ben Pace

So how old's the guy again?

zhukeepa

Early 70s.

Ben Pace

He's not fully dead yet.

zhukeepa

I hope he stays alive for as long as possible.

Ben Pace

Scott Sumner and Russ Roberts about 69. It's quite old. I didn't know how old they were. I know Russ Roberts stays sharp for a man of that age.

zhukeepa

Chris is definitely still very sharp.

Ben Pace

Where does he live in the world? What does he do with his time?

zhukeepa

Missouri? I'm not sure what he does with his time, but one of the things I'm also working on is helping more people understand what Chris is trying to say.

Ben Pace

Okay. Maybe we should start coming up for air. We're scheduled to wrap up in the next 10 minutes.

zhukeepa

Yeah.

Ben Pace

Anyway, I'd be interested in talking more about how to reconcile. It's currently more interesting to me, so I'd be up for that if you wanted to do more of that. If you wanted to do some of that.

zhukeepa

Yeah, I think also talking about the technical side, and the relevance for civilization having functional discourse. Functional discourse is also something I would like to flesh out more with you.

Ben Pace

Yeah, okay. Sounds good.

Conversation 3 — April 20th, 2024

We continued our dialogue over Zoom on April 20th 2024. The audio was also transcribed and lightly edited.

On the nature of evil

Religious prophets vs the Comet King on the problem of evil

Ben Pace

How are you doing?

zhukeepa

I'm good.

Before we pick any particular direction to go, let's enumerate possible directions.

Ben Pace

Before we do that, can you tell me what how you would compare the religious prophets and the Comet King? I want to know.

(Spoilers for UNSONG below.)

zhukeepa

Okay. I think the Comet King is baffled by the problem of evil and saw evil as something to destroy. I think he resists evil.

And I think part of what I found interesting was at the end of Unsong, there's this thing about how Thamiel actually has all along been doing God's will. And everything that looked like evil actually was done with noble intentions or something, that I found... it got me thinking that Scott Alexander might be wiser than the Comet King character he wrote.

But my general sense was that I felt like Scott and the Comet King were sort of grasping around. Scott was pointing in the right direction for how to grapple with the problem of evil, but there were some things he was missing that I think Jesus and Muhammad and the Buddha understood.

Ben Pace

Wait. So first, why did you think he was wiser than the Comet King character?

zhukeepa

Well, I think he was wiser than the Comet King, at least when the Comet King tried to destroy hell directly, in that at the end of Unsong, by the end... If I recall correctly, part of what happened was when all the minds were emerging, Thamiel revealed himself to be an instrument of God all along.

And something like, all the evil actions that were done under Thamiel's name in some sense had noble intentions behind them -- or, in other words, was in some sense innocent. And that feels like a key part of the problem of evil... I think the prophets get that, and I think the Comet King didn't get that, at least when he was trying to destroy hell.

Ben Pace

Yeah. I don't know. It seems a bit naive to me. I can see some noble intentions behind it, but I also can see a perspective where Scott is implicitly not getting that some people just want to be evil or something.

Evil is like cancer, maybe

zhukeepa

In my current model, I think evil is a lot like cancer.

So, Mike Levin has done some research where he figured out how to take cells that were slightly cancerous, and basically going off and doing their own thing separate from the rest of what the other cells in the organism were doing. And he found a way to reconnect the cancerous cell with the rest of the cells in the organism, and then the cell stopped being cancerous. (2:26:00 - 2:26:35)

Ben Pace

Sorry. You said that they could disconnect the cells from the other cancerous cells, and then reconnect and the cells are healed?

zhukeepa

Mike has done research on bioelectricity and organisms, which he basically recognizes as a software layer above the hardware level of DNA that modulates a lot of the activity within organisms. And what cancerous cells do is they sort of disconnect from the main bioelectric network of the organism. And he's managed to get pre-cancerous cells and slightly cancerous cells reconnected to the main bioelectric network, and in so doing, gotten them to stop becoming cancerous.

Ben Pace

That's interesting.

zhukeepa

I agree. I think this also has parallels to when people have traumas that they don't recognize as traumas and just think of as being who they are, which is my current model of where evil comes from. And that trauma that they're endorsing as who they are, rather than as a trauma, sort of takes on a life of its own, which seems structurally similar to a cancer. And the healing of that trauma, by reconsolidating the emotional memory via forging a connection through the mental mountain separating it from the rest of your psyche, seems structurally analogous to reconnecting the cancer to the rest of the organism's bioelectric network.

Ben Pace

I recently came up with a new frightening thought, or, one of the ways I can empathize with people who seem to be wicked and cruel.

Suppose someone hurts you. It does not get righted and no one really notices it, but you notice it and you're tracking it. From your perspective you think "I have learned, that if you're a person like me in situation A, the counterparty's response was to do X, and X hurts me, but the harm doesn't get repaid".

And then situation A' comes up where you're in the opposite position, a situation that seems to you structurally similar and your counterparty thinks "Well, I assume you'll be nice and friendly to me." But you're like, "Well, you don't get it. The thing that happens to you in situation A is that X happens to you. And no one cared about it when it happened to me. And so I'm going to behave the same, and I'll show you that this is what happens, so that you're not naive. If you would like, we could fight together to make sure the harm is accounted for across the world whenever situation A comes up, but that seems very ambitious, and in the meantime I'm going to continue to show you how this situation currently works, and so I'll take an action that hurts you and I will not apologize for it. This isn't me hurting you, it's me teaching you how the world works." But then from the other guy's perspective, you were just was randomly cruel and nasty to them." They're like, "Wow, that guy is just randomly cruel and nasty."

The reason I find this frightening is that, insofar as this is a common behavioral pattern, whenever a good norm is broken and not punished, the person hurt may start to propagate the norm breaking themselves, and the next people will too, and quickly norms may dissolve.

zhukeepa

Yeah, yeah. This is basically what I... the thing you described feels like a pretty central example of the kind of thing I'm pointing at with trauma that you endorse as just how things are or something, as opposed to viewing of the trauma.

Ben Pace

I guess for me, the thing that's weird about it is it comes from this noble kind of feeling. It comes from a desire to set things straight, where you're trying to track what's good and what's right and wrong, and every person and institution around you seems to not be tracking it. So around you, things seems to have gone wrong, and you're trying to set things right and teach people the rules of the game, but you're doing it in a fairly...

zhukeepa

Misguided way.

Ben Pace

...well, what's misguided about it? Somehow it's a bit zero-sum or something, but I can't figure out why. I don't understand it. I'd have to spend some more time to properly articulate it.

zhukeepa

You might also find people who are like you, and it may feel like you click with them, because you both share an internalized norm and can connect on that level and not get mad at each other about it.

Ben Pace

How does that relate to doing evil things or cruel things? Are you saying we all might agree to a certain cruel norm?

zhukeepa

I mean that there's something contextual about the nature of evaluating what's misguided. For example, I think a bunch of Chinese norms seem pretty antisocial by western standards, and I think some many western norms seem pretty antisocial by Chinese standards.

Ben Pace

But I think that's just a difference of protocols. If you interpret these actions in the other protocols, then you'll get the wrong results, but we pick these protocols and we think they're a good idea. And then there's–

zhukeepa

Yeah. I think also, part of what I'm saying is that judgments of evil are protocol-relative.

Ben Pace

You mean, I should not judge a Chinese person's actions the same way as I judge an American's person actions, even if they're the same actions because they have different background protocol assumptions. That was what I was hearing you say?

zhukeepa

More like the judgments of some person's actions as evil are relative to a particular protocol they're operating from. And there are some things that are pretty universally considered evil like murder, but I would just say that most human protocols count that as evil.

Ben Pace

Sure. Well, wrong and evil are different.

But yeah, another frame I have is... Jordan Peterson had this good point about bringing up kids where he was like, "You can bring up your kids poorly, and prepare them for the world poorly, by giving them an overly fine and easy environment." The point is to prepare them for the world that they're going to arrive in rather than to be like, "Everything is fine. Don't worry about a thing until you become 18." Insofar as there is something that will normally be punished at in the world, it is actively helpful to this new person to give them negative feedback for the thing, so that they form accurate expectations. Maybe not punish them as hard as the world would, but make sure that they anticipate that they will get a negative reward.

And similarly it reminds me a bit of the book series 'A Series of Unfortunate Events' that also has a similar philosophy about kids, which is not that you can protect them from the world, but you have to prepare them for it.

I feel like there's a similar impulse when someone is going around doing a lot of evil things, which is something like, "Well, this is how the world works and I'm teaching you. You keep going around being trusting and kind, and let me tell you, bad things are going to happen to you and I'm going to show you that bad things are going to happen to you as a result, so that you learn." And then the other person is like, "You just made the world worse. You are the cause of the world being worse." And they're like, "Well no, I think the world is worse and I'm just teaching you," as I said.

zhukeepa

Yeah, that resonates a lot.

Ben Pace

Yeah. And I think that connects with like, there's often a naivety that more evil people have towards good people or something. 'You're all counting on the world being nice and fairies and kittens, but definitely people are going to steal your stuff and rape you. So, you idiots. Don't judge me.' Anyway, I just care about this a lot because I've been trying to figure out what evil is.

Mistake vs conflict theory on cancer / evil

zhukeepa

So, I was drawing an analogy between evil and cancer.

Ben Pace

You were also saying you felt that somewhat like the Comet King didn't understand evil, as well as the prophets did.

zhukeepa

Yeah. There's sort of taking the conflict theory view of cancer, where the cancer is just in conflict with the rest of the organism and there's sort of a mistake theory view of it, where actually, the thing the cancer wants is to reintegrate with the rest of the organism.

Ben Pace

Utter idiot.

zhukeepa

But yeah, it doesn't know how to.

Ben Pace

Well, it 'doesn't know how to' as hard as you can fail at 'not knowing how to' — in that it kills the rest of the organism. It's such an idiot.

zhukeepa

Yeah. And I think the mistake theory view of evil, I think, is pretty central to all the deal about forgiveness. And yeah, I think the Comet King sort of had a conflict theory view of evil when he was trying to destroy hell.

Ben Pace

Can you give an example of how one of the prophets understood evil better?

zhukeepa

Yeah. I think Jesus on the cross being like, "Forgive them, for they know not what they do"... it's like, yeah, even the evil that results in me getting crucified, I'm still going to take the mistake theory view of it.

Ben Pace

I mean, the reason you would want to not do that is because it makes it so meaningless. It's just so annoying, you're being... It's like, "No, this person's evil and this is a big fight of good against evil and we need to kill them," versus "They're just screwed up and now I'm going to get killed and I'm going to get tortured and every good thing that could have happened isn't going to happen." It just feels more meaningless when it's a mistake rather than a fight.

zhukeepa

Okay. Yeah, I think there are more datapoints in steel-Islam, for how to navigate the fact that even though cancers might be mistakes, you sometimes maybe still need to kill them anyway.

Ben Pace

Yeah.

zhukeepa

The general vibe Muhammad took toward people acting toward him in evil ways was sort of trying to see through it, and connect with the person "behind the evil" who knew not what they did.

Ben Pace

It seems like an ambitious high variance strategy, but I hope it works out. I'm sure it works out sometimes.

zhukeepa

I mean, in the charitable accounts of his success, he did not always employ this for the people who were trying to kill him and his army.

Ben Pace

Yeah, yeah, yeah. But if he was talking to them, he would probably try to talk to the person behind them.

zhukeepa

Yeah. So one particular anecdote about Muhammad that for me, captures the heart of Islam for me actually, is a story where he was resting under a tree and one of the polytheists who hated him was like, "Oh, this is my chance to kill him." So he goes up to Muhammad and raises a sword over Muhammad's head and says, "Who will protect you now?" And then Muhammad with complete calm and confidence responds, "God will protect me." And this confidence shook the polytheist so much that he trembled in fear and dropped his sword.

Basically, what Muhammad is saying is, "My God is more powerful than your gods. And if you fuck with me, you're going to regret it, because my God is more powerful than yours." And he basically transmitted this vibe in his interaction with his would-be would be assailant. So the assailant dropped the sword, and then Muhammad picks up the sword, and then holds it over his head and says, "Who will protect you now?" And the polytheist says, "Nobody. This is it. I'm done for." And then Muhammad drops the sword and says, "Wrong, the same God will protect you too."

Ben Pace

Okay.

zhukeepa

And then he converts.

Ben Pace

...do you want to go out and try and convert some people together with a sword?

zhukeepa

Well, it would require total confidence that I would be protected from someone holding a sword above my head, which I don't have. There are stories of people being outnumbered three to one, but then scaring of all of their would-be assailants with their body language. I could not do that to someone threatening to murder me.

Ben Pace

No. I mean, maybe you could. I don't really know what would happen. Those are weird situations. But I didn't like that he said, "My God will." I feel like I wish he'd said... I don't know, this is a bad answer, but the concept of goodness will, which you and I share. I don't know if a person would do that.

zhukeepa

Well, they didn't share it. I think back then, God was the best language they had for talking about a lot of these things. The ontologies were just super, super different.

Ben Pace

What with?

zhukeepa

The ontologies of the people in Arabia around the year 600 are very different from ours.

Ben Pace

I didn't like that you said, "My God will." What were you connecting that to that we were talking about?

zhukeepa

The prophets seeing through evil.

Ben Pace

Oh. So how do you talk to someone through evil? I understand that's supposed to be an example, but it was a very low detail example. It was only one phrase. You have to find some part of them that's not... I was not convinced. I don't know.

zhukeepa

It sort of views them in a mistake theory way, where they're actually just mistaken about the actions they endorse doing and they just haven't recognized that yet.

Ben Pace

There's still like a, which part of them hasn't recognized it? There's some belief about what they're made up of in their mind or in their soul.

zhukeepa

The part of them that endorses acting in a way, in an evil way.

Ben Pace

Look, you can take me to some random evil person, I don't know, like a terrorist or something. And we can connect over parts of the world that are not about that part of the world. We could probably cook together and eat a meal and hang out in nature, and there would be some connection there. But how do you connect... I don't know how to... It's not quite the right phrase, but tempt them away from evil.

zhukeepa

They have to decide for themselves, but you can help facilitate that process, is sort of how I think about it.

Ben Pace

I don't quite get how to do it without knowing them very in detail.

Forgiving evil in ourselves

zhukeepa

Okay. I think the easiest sub-case to focus on is the evil in ourselves.

Ben Pace

All right.

zhukeepa

I think generally, anytime we're judging ourselves and thinking to ourselves, we deserve not to be forgiven for something, we're taking a conflict theory view of some evil part of ourselves, rather than a mistake theory view. And I think of the forgiveness and self-acceptance as a shift toward the mistake theory view of why we did the bad things that we've done.

Ben Pace

I wish this story had more detail to it. Do you have any other stories of talking to people, talking past the evil of them? I feel like I just want some more detail. Or of you talking past it to yourself I guess, is probably also good.

zhukeepa

Yeah. I've done things in romantic relationships that I considered unforgivable. They're pretty bad. I'm not going to go into them right now. And for a while, I just thought I was a bad person. And I basically came to realize that those parts of me don't just exist in a vacuum. They largely came from my mom having treated me in ways that were like that while I was a teenager.

And then I had a phase of blaming my mom for treating me that way for a while like, "Yeah, I'm a bad person, but that's because you're a bad person too, mom." But then I realized that she was just doing the best she could and she really had no intention to hurt me. And part of her doing the best she could was her mom treating her basically that same... She would tell me stories of her life as a kid in China, and her mom would be treating her in a similar kind of way.

And then there was a way in which I saw the patterns that we were enacting as impersonal, and just the way things were in some sense, and not deep reflections of worthiness of love or of punishment. And once I took that stance, I was able to shift the frame on myself towards "oh yeah, I'm a person who's capable of doing some pretty bad things because of his trauma, and I should be careful about that, and heal the trauma" and away from "I am a bad person who does bad things to people." I was also able to extend that same attitude toward my mom and her mom.

Ben Pace

What's the difference between... So sometimes you've wronged someone and you owe them some debt, and you don't really notice or care. Then you realize at some point you go, "Oh, dear. I wronged them. I owe them some debt. I can't fix this. I can't right this wrong." Not all things can just be undone. It's not how lots of things work. The world is just worse now. And you could make some effort to fix it or not to fix it, or to send a credible signal that you would fix it if you could.

I think that's part of what an apology is is to say, "I recognize that I'm in your debt and I would fix it if I could. I don't know how to, but I want to register that this is the state of things." And then once it's firmly in the accounting, the counterparty can choose to forgive the debt (or not). What's the difference between the thing I said, realizing that you would make it right if you could and the thing you said, which seems different, where it seemed like you already would've fixed it if you could have and then something changed where you forgave yourself?

zhukeepa

Sorry. What's the thing I would do if I could, like repay the debt?

Ben Pace

Or undo the hurt you had caused.

zhukeepa

For me, that's part of the process of forgiving myself. If I want to correct the mistake, that means living in alignment with being the kind of person who doesn't make that kind of mistake in the first place, and also tries to make amends with the people who he has made those mistakes with.

Ben Pace

But how much were you not that before, and then how much were you that after you changed your perspective on yourself?

zhukeepa

In this particular case, I think I was pretty willing to make restitutions, if that's the right way of saying it, pretty soon. But that didn't feel sufficient. The harder bit for me was like, "Oh, I have a part that's capable of doing this kind of evil and I don't trust myself to not do this kind of evil in the future."

I think it was only when I took an orientation of self-forgiveness that I was able to stop judging myself for the evil while still being realistic that I'm going to make these kinds of mistakes in the future, but letting that be okay while also actually taking the steps to be better at not making that mistake in the future. When I'm making myself wrong, it's a lot harder to find the root cause of why I made that mistake in the first place.

Ben Pace

Oh sorry, say that last bit again, what you did–

zhukeepa

When I'm making myself wrong for the evil, it's much harder to find out what the root cause of the evil is in the first place.

Ben Pace

When you're marking yourself as wrong or you're castigating yourself?

zhukeepa

Yeah, when I'm castigating myself for my wrongdoing.

Ben Pace

Insofar as you believe you can't change and then you just try and punish yourself every time.

zhukeepa

Right. When I allowed myself to take the stance that my evil knew not what it was doing, then there was room for getting curious for what... the evil knows not and what it does. Then there was room in me to get curious about what kinds of mistakes I was making that led me to take the evil actions in the first place.

Ben Pace

It sounds like the relevant thing was something like a belief that you could change?

zhukeepa

That feels like a core piece, but it feels downstream of allowing myself to even look at the evil in the first place rather than just thinking of it as something to exterminate

Ben Pace

I do hold, as a pretty high virtue, a virtue of being able to look at any part of reality. I think, as far as you are doing a bunch of all things, I do hold it as pretty high that you need to be able to look at that with a certain level of lightness and just be like, "Hey, I wonder what's happening over there. That's interesting. What happens if I do that? Oh, it does this." It seems important to me.

zhukeepa

Yeah, I think basically humans have trapped priors that make it very hard for them to look at evil for what it actually is, without trying to castigate it. I think that's a core thing.

Ben Pace

Definitely people have a difficult time with self-reflection and self-awareness.

Did you want to get something you wanted? I was getting things I wanted, but I think you wanted something slightly different.

Does mistake theory toward sociopaths really make sense?

zhukeepa

Cool. I mean, I'd wanted to enumerate possible topics to talk about. Now that we're on this thread, I think it's good to keep going down it. Yeah, so there are some things about the nature of evil that I think my steelmen of the religions get better than the Comet King.

Ben Pace

I think I agree. When you were like "forgive them, for they know not what they do", I didn't see that in the Comet King. I felt like he had something more of a fairly childish notion of ethics or morality.

zhukeepa

I would have called it standard rather than childish.

Ben Pace

Well, I feel like he couldn't empathize. I don't know if I was right. I don't know if this is right, but I feel like he couldn't empathize with evil people. And I always think being unable to empathize is not a virtue.

zhukeepa

Got it. I do want to make the caveat that I've only read the excerpts on Paul Christiano's blog post plus a couple of pages in the actual book and so I'm going off of-

Ben Pace

It's been a bunch of years since I read it, so I was also kind of going off of those, I'd have to re-read it to have a more confident stance.

zhukeepa

Okay, cool.

Ben Pace

I think the thing that's happening right now is I'm just annoyed by evil and would like to understand it and so I'm just flailing about in this conversation hoping that at the end of it I will.

zhukeepa

Yeah. What about it feels most annoying to you right now?

Ben Pace

Well, it's just that it's out there and I don't know how to engage with it or think about it and, I don't know, I thought I might be able to as a result of some of the things you said. I thought I had some threads but I don't know about what's annoying me about it? I know.

Feynman says you should always have a dozen big open problems in the back of your mind and whenever you hear a good technique you try and apply it to all of them.

zhukeepa

Yeah.

Ben Pace

Anyway, you said something that made me feel like "Oh, maybe I can finally understand it," I can't remember what you said. Because I just keep wandering around trying to because it's always metaphysically weird when you talk about good and evil. Everyone's like, "That's not a thing, is it?"

zhukeepa

Yes.

Ben Pace

It's not like an element, not like a carbon or something.

zhukeepa

I think part of the crux of it for me is taking a mistake theory rather than conflict theory view of evil, which doesn't imply that I'm not going to get into conflicts with evil. Sometimes I don't have any means at my disposal to communicate that there's a mistake. But also that leads me to take a pretty different orientation toward evil. It's easier for me to empathize. Rather than thinking of them as just fundamentally bad people, if I instead think of them as good people who made a mistake somewhere and are now in this hellhole relative to how I operate... as you were saying earlier, there's this way they're just being unkind to others that they just think is normal.

Ben Pace

It seems like a helpful fiction to imagine everyone's a good person just trying and failing. But I don't know if it's true.

zhukeepa

I mean I think it's true on the level that cancers are mistaken. It's like cancers are mistaken. Also, sometimes you should zap them with chemo because we don't know how to correct those mistakes yet. I feel like one concrete difference between the conflict versus mistake view of evil is that on the mistake view of evil, when you feel safe from the evil, there's room for you to turn the other cheek and then that helps plant the seed for the evil to realize its mistake.

Ben Pace

So, I was trying to emphathize with choosing to hurt people... and playing the role of evil in their lives... and I'm trying to figure out what mistake you might make to get there. But I think you can just be that the whole time. Can you not just terminally value hurting people?

zhukeepa

I think that's anti-natural.

Ben Pace

Why? There's people who like torturing animals and there's people around who we won't ever be able to fix and just go around hurting people all their days. There are sociopaths and–

zhukeepa

I don't know about whether we'll ever be able to fix them. We certainly don't have remotely the technology now to be able to help them.

I also think they tend to be pretty unkind to themselves and miserable in a particular way,

Ben Pace

But they also don't care about it as much. "Be kind to yourself," and they go, "No." I don't really know what they say. Perhaps they say, "That's not what I'm interested in. You'd like it if I was kind to myself, wouldn't you? Watch how unkind I can be to myself," he says cutting off his own finger. I don't know.

There's another way things can be. You can come to just pretty much learn the heuristic that hurting people is often a pretty good thing according to you or a thing to move towards. There's a bunch of reasons to think that, a bunch of ways that people can get to that heuristic, a bunch of conflict situations where they practice that heuristic and it goes well for them.

zhukeepa

Okay. I think a fairly concrete example for me is I've met a bunch of pathological liars who were raised in Eastern Europe, where my sense is that during the Soviet era, pathological lying was rampant, and it was an extremely common survival strategy.

Ben Pace

Yeah, that's messed up.

One part of me is disappointed that these kinds of people are able to exist at all. I'm like, "Wouldn't it be nicer if that strategy caused your civilization to crumble immediately?" And they're like, "Nah, the people lasted for 200 years" or something. I just like truth a lot, I wish it was obvious and easily the best strategy.

zhukeepa

Yeah, when I meet these people, my first thought is like, "Holy shit, I hate that you're lying to me and I can't trust you. Fuck you."

And the second thought is like, "Oh wait, you were raised in a culture where this was normalized and in some sense there's nothing personal about this."

And the third thought is like, "Holy shit, you're fucking lying all the time. That sounds fucking terrible. You can't trust your own beliefs, other people can't trust you and you can't trust other people because you think they're also all kind of lying to you."

Compared to how I live, that sounds like a hellhole. But they don't recognize that, and there's nothing you can do to force them to recognize that. They have to make the decision for themselves to stop lying and tell the truth. There are things you can do to help facilitate that decision, but ultimately it rests on them.

The mistake theory aspect comes from thinking of them as not fundamentally being liars at their core, but instead something they learned in the course of growing up.

Ben Pace

I feel like it's possible that the first person they were was a liar and they never were not that and you can be like, "Well, I can hope that you might change into a different person." But I don't know that they ever were.

zhukeepa

They weren't liars when they were babies. At some point-

Ben Pace

Maybe it's not that they got put into a rough culture and then they became a liar. Maybe they were just a liar from the start.

zhukeepa

I think liar from the start can be a thing as well. But even then, my story wouldn't be like, "Oh, it's because they were acculturated into that." It would be like, "Oh, you developed in a very unfortunate way". It's like if you have a birth defect that causes you to not have a limb, but instead it's a "birth defect" that gives you the false belief that it's better to live your life as a liar.

Ben Pace

But just as a question of are they a good person with bad parts clumped on top or are they just a bunch of good and bad parts and they could probably become a different person. You seem to be a bit essentialist about, "No, there's a good person there and they just have some issues," whereas I'm like, "Here's a person and here's some of their properties. We would like to change which person they are because this person utterly sucks."

zhukeepa

Mistake versus conflict theory are both valid self-fulfilling prophecies one could use to model someone, and I think I'm less being an essentialist about people being intrinsically good, and more insisting on mistake theory being a self-fulfilling prophecy that wins more, in that I think it gets you strictly more degrees of freedom in how you can interact with them.

Conflict theory kind of defaults to an "eye for an eye" view. Mistake theory leaves room for this when you're not in a position of strength, but also leaves room for you to turn the other cheek when you're in a position of strength, which can help them recognize their mistake.

Ben Pace

I want to rescue conflict theory from your criticisms.

zhukeepa

Okay.

Ben Pace

I'm not sure, but I think forgiveness is more relevant foe a conflict theorist than a mistake theorist. I think the goal should be to be a conflict theorist who can forgive your enemies rather than a mistake theorist who can forgive your enemies and it's much–

zhukeepa

Okay. I think of forgiveness as going from conflict theory to mistake theory.

Ben Pace

No, there's conflicts and you should be the sort of person who can forgive people you're in conflicts with, even if it doesn't fix anything and even if you're still in conflicts with them afterwards.

zhukeepa

That makes sense, but seems consistent with my views. I don't think mistake theory necessarily implies fixing anything (although it leaves more room for things to be fixed), nor do I think it implies not being in conflicts with them afterwards (although it does leave more room for conflicts to get resolved).

I can grant that some pathological liars didn't pick it up culturally, and kind of just have fucked up genetics. But I still prefer to view them as a person who would ultimately actually prefer truth but took a wrong turn at some point, than as someone who is just fundamentally in their core a pathological liar. I think, with the technologies available today, me viewing them as a mistake theorist does not lead to substantially different actions from me viewing them as a conflict theorist, except perhaps that I might be more attuned to the suffering they're going through for having such a deeply entrenched false belief about the best way to live their life.

Ben Pace

Was this coming down from a discussion of who understands evil better, Unsong or the prophets?

zhukeepa

Yeah. Then you were like, "I'm still annoyed about evil somewhere," and then I just started going to my mistake versus conflict theory models.

Ben Pace

I think you said that you think that the prophets had some better understanding of evil than the Comet King and that was why you found their stories better. Is that what you said?

zhukeepa

Those were one of the ways, yeah. I like them more than the Comet King.

Ben Pace

Or you find their stories more useful to learn from or whatever.

zhukeepa

Yeah.

Ben Pace

I also wanted to bring up HPMOR!Voldemort as an example of someone who's pretty far down the evil category, an evil sort-of-rationalist who is relatively quite self-aware about their evil nature. Harry has a reflective conversation with him toward the end of HPMOR and was scared by how confident Voldemort was that he would end up evil. Voldemort was like, "You're right. I will now start to consider strategies where I'm kind to people. That will be a better way of getting my goals sometimes." But Voldemort was also like, "At the end of it, I'll still be evil and kill people because it's just the way I want to be." I think that's something people can choose.

And I feel like that's just a thing in the world that you can get to... like, orthogonality and so forth. I think you can get to an entity that's just like, "Yeah, I decided I kept self modifying towards being evil and it kept being better every time and now this is where I'm at and I'm in an attractor state and any move away from this is locally not good and kind of anti-integrity."

zhukeepa

Yeah, I think it's anti-natural for such an entity to exist, and that's a crux for religion-according-to-Alex. In particular, I think such an entity would be mistaken so long as they hold such beliefs, although I don't think it's impossible for such entities to mistakenly hold on to such beliefs for as long as they exist.

Ben Pace

Yeah. So I kind of want to defend it. You can argue against it. I mean, what's your argument against that being-

zhukeepa

Well, would you classify the genetically defective liar as fundamentally evil?

Ben Pace

Look, I'm all for hope and believing in people. I just don't want to assume, in principle, everyone definitely can get better and everyone is definitely a good person. I think in practice hopefully, but I don't want to like... You'd be like, "What about this evil person?" You go, "Actually they're definitely fine, don't worry about it. They're definitely just a good person covered with some problems." I mean maybe it's true, but maybe humans just all are all good. But I don't know. I don't know most people.

I feel you're like, "No matter how far gone they are, they can always come back from it." And I want to be like, "You can get far gone enough that you can't come back from it," and this seems like a meaningful difference in our models of the world. And I'm not saying that any particular–

zhukeepa

Okay. I think people can be so far gone that, with the technologies available today, they're not going to–

Ben Pace

I'm saying, in principle, they can't come back from it. Like you could rearrange the atoms to get a good person, but the same person would no longer be there.

zhukeepa

So, if you rearrange the atoms of the cancer cell so that it stopped being cancerous and just want it to become a normal cell, would you say the cancer is still there?

Ben Pace

I don't view cancers as having a concept of personal identity.

zhukeepa

Well, the Buddhists make a point about how most of our concepts of personal identity are pretty confused, and I'm with them on that.

Ben Pace

Sure. I'm sure my concepts of personal identity are confused. That seems surely the case, given that I haven't thought about them... I've thought about them a fair bit. I don't know. I think I mostly identify with my virtues rather than my ego. Everyone keeps telling me this is wrong.

zhukeepa

It seems more correct to me to identify with your virtues than your ego.

Ben Pace

Well, then, I'm like, "And I don't really like myself." And then, they're like, "Well, you should." And I'm like, "But I'm not that good. I'm kind of weak in a bunch of ways except for some virtues, and those are the main things I care about."

zhukeepa

Just for the record, I do think there's more to who you are than your virtues. I think who you are includes both your virtues and your ego. But I'm not going to go into that right now. In the meantime, I propose we make a list of possible other topics to talk about.

Rescuing old religions vs competing with old religions

zhukeepa

Cool. All right. Let's talk about how you're sick and tired of me trying to rescue the old religions.

Ben Pace

I knew you'd pick that.

zhukeepa

It was one of the two things on my list.

I want to get closer to where you think we approach or think about religion differently.

Ben Pace

Right.

Well, they're just like the oldest, most epistemically, corrupt institution on the world, and they have so much power. And if you're like, "Wouldn't it be nice to rescue them?" I'd be like, "Wouldn't it be nice to burn them to the ground, and then, build something better?" And every time you're like, "What about rescuing them?" I'm like, "Look at all the horrors. They can't think or change their minds. They rape kids and have so much power over individual families' lives. People are like, 'Yes, let's just have them decide how my life works.'" I'm like, "This institution wouldn't stand up to Aella running five surveys on their members. We can't rescue this institution. We've got to start new institutions." I really think that they just can't live up to the level of individualism where you're just like, "What does everyone believe?" Because if everyone just says what they believe, you'd go, "Well, we don't have an institution anymore because we assumed everyone agreed on everything." But it turns out they all have their own opinions. Oh no.

Alex doesn't personally find institutionalized religion compelling

zhukeepa

For some reason, the thing I'm wanting to say in response to that is how I feel about the Baha'i Faith. Have you heard of the Baha'i Faith?

Ben Pace

Who is it? I've heard of it, but I don't know who it is.

zhukeepa

It's a new religion that started within the last 200 years that's basically... in some sense their doctrine is basically just "Here's how to interpret all the religions. Here's the core thing they have in common that we're going to express that to the best of our ability. All the religions are fundamentally trying to say the same thing, and we're just trying to be the religion that unites all the other religions." And their doctrine seemed super reasonable and legible. It wasn't like you have to believe these weird supernatural stories. Instead, it was like, "Yeah, here are reasonable ways you can interpret all the aspects of all these other religions." And their prophet's first principle, the highest principle for this religion, is the independent investigation of the truth. It seemed so reasonable that I'd even considered joining.

And then, I just did more research on this, and they were like, "Yeah, homosexuality is bad." And I'm like, "Well, where did they say that?" And then, they cited one of the prophet's original sayings, and one way you could interpret that was as homosexuality being bad. But in the original context, he was saying something like, "Yeah, you know the thing that our society does where men have sex with young boys? That thing is bad." And somehow that got generalized to all homosexuality is bad. And I'm like, "Okay, so my independent pursuit of truth leads me to not think homosexuality is bad, but apparently, if I join your religion, whose core tenants resonate with me and seem reasonable and non-dogmatic, I'm expected to come to eventually hold the belief that homosexuality is bad because of some great-grandson of the original prophet interpreting something the original prophet said in a particular way... ghat seems lame. I'm not going to join this religion."

Ben Pace

Hey, let me just read the Wikipedia page on Baha'i Faith for a moment. I just pulled it up.

zhukeepa

Sure. I'm going to get some water.

Ben Pace

That's funny. I want to meet some of these people that claim to be prophets. I like that they had a guy who was like, "There's another prophet coming soon." And then, another guy who was like, "And I am that prophet." And then, he was exiled and imprisoned. It seems like a pretty classic trope.

zhukeepa

I've read books about a lot of these people. They're interesting people.

Ben Pace

Rejects notions of racism, sex. Well, they do seem to have these false beliefs about God, but that's to be expected.

zhukeepa

I think there are ways you can interpret them that are not false.

Ben Pace

The Baha'i writing state that human beings have a rational soul. Sure. Isn't trying to sell itself to me in particular. Anyway. So, I don't like that, but that's fine. I can't quite tell what sort of religion it's like. Did they write all the books themselves, the Bab? Or is that an old book? No, it's from the 1820s or so on. Oh interesting, it was fastest growing religion between 1910 and 2010.

zhukeepa

I think if I were forced to choose a religion to join, where I would just like... I'm in a city and someone was like, "Yep, there's a community of every single world religion here, and you're forced to pick one to join," I think I would pick the Baha'i.

Ben Pace

I would see if I could make my own one, and if not, I'd probably pick one that didn't have any metaphysical beliefs in God.

I don't know. I hear this about the Unitarians, or the Buddhists. Or maybe some Jewish group who just clearly didn't believe in God.

Let's read the prohibitions. Backbiting and gossiping are prohibited and denounced. That's an issue. I like to gossip. I think it's good. Drinking and selling alcohol are forbidden. That's also an issue. I like drinking and selling alcohol. Okay. Participation in partisan politics is forbidden. That's kind of cool, but also kind of obviously dysfunctional if the entire population converts. Begging is forbidden as a profession. What? They must have had some problems with that when they lost all their money when they became religious. Okay, it's a new religion.

So, what were we talking about? I was like, "You need to respect individualism," and you're like, "This one is the best." And I'm like, "It doesn't seem different enough from the other religions for me to be excited about it."

zhukeepa

Sorry. I think that I brought this up as the most reasonable-sounding religion I could find, in some sense.

Ben Pace

Yeah. It's interesting.

zhukeepa

And it not passing my bar for wanting to join it.

Ben Pace

Yeah. Okay.

zhukeepa

I think I'm saying that as a level on which I feel like I'm agreeing with you about the inadequacies of religions, especially when institutionalized.

Mistake vs conflict theory toward existing religions

Ben Pace

You keep talking about uniting and interpreting the religions. And I'm like... I don't know. It is missing something like "We're in conflict with them". I'm like, "If the Christian Church comes by—"

zhukeepa

Oh. I think we're in conflict with the exclusivists, but not the pluralists.

Ben Pace

No! I think we're in conflict with the pluralists.

zhukeepa

Okay, that's a crux for me. I don't feel like I'm in conflict with the pluralists.

Ben Pace

I'm like, a bunch of Christianity and Islam does not have a good interpretation. It's just mistaken and dumb. And as far as people are trying to live it out, we should stop them. And as far as they're trapped in it, we should disempower this institution.

zhukeepa

Okay. I think a lot of pluralists do feel that way, although they have more of a mistake theory rather than conflict theory view of the kind of mistakes you're naming among the exclusivists.

Ben Pace

I think the correct relationship to the religions, even if you're getting the good stuff out of them, is to be an atheist, or to be like, "Cool, I'm starting my new religion. You guys can come and hang out sometimes, and we'll talk about how to live well in the world, but you guys are messing up really badly on loads of things". Sorry. You agree with that?

zhukeepa

The good pluralists I've talked to would agree with that. They would be like, "Yeah, it's a real mess out there. Things are pretty fucked. Lots of reform is needed. Atheist critiques of religion are largely correct."

Ben Pace

But I feel like you're arguing for a reformed Christianity, and I'm arguing for a destroyed Christianity.

zhukeepa

This feels like an instance of a mistake versus conflict theory thing actually.

Ben Pace

No, it doesn't. It's similar to you being like, "Let's fix the FDA." And I'm like, "Let's just end the FDA, and then, start any new regulations separately." Some institutions should die. Surely you agree with this. I don't think you think one should always reform the institutions. Sometimes, it's like, "Please end the institution. Everyone will be strictly better off as soon as it's over, and then, we can start working on maybe making some new institutions."

zhukeepa

I think Joel Spolsky wrote a blog post about how programmers who think that way about old legacy code bases, and they're just like, "Fuck that. We're just going to start anew," tend to get bitten in the ass because, for all the shortcomings of the old code base, there's a lot of adaptation to weird edge cases that you can only learn through contact with reality that you're going to lose if you just throw the whole thing out.

Ben Pace

No, I understand that sometimes, people are wrong in the other way.

zhukeepa

I'm also curious what you think are the most wrong metaphysical beliefs.

Ben Pace

Okay. We'll get to that in a minute.

There's a good point in the Moral Mazes sequence by Zvi. He talks about how institutions become corrupted and over time, where as everyone goodharts on gaining power within the institution, the very culture of it changes to reward power seeking rather than to reward actually object-level achieving the goals of the institution. And I think one of the things that he says is very good is that especially these big companies have competition such that a very mazy, culturally dysfunctional company can be outcompeted by a very healthy startup, or something like that. And that's something I believe, which is that competition is healthy and good and incentivizes against corruption.

zhukeepa

Yeah. Yes.

Ben Pace

And you just wake up one day and you'd be like, "Oh, my company is dead now because someone else made a better company than me." I've got no right to own the next company.

Religious pluralism coalition memetically outcompeting religious exclusivism

zhukeepa

I think Christian pluralism, Muslim pluralism, Buddhist pluralism, and "atheist pluralism" are all compatible with each other. And I want the mutually compatible thing to memetically outcompete the exclusivists' memes. I think this is just another articulation of the thing that I'm wanting, which is maybe consistent with destroying the exclusivist memes, and maybe also consistent with salvaging them.

Yeah. I want this thing to have memetic dominance. I think some of that dominance is going to come from converting exclusivists to pluralists, and some of that is just going to be from pushing them out and winning, or something. And maybe we have different views about what the relative fractions of those are, but that doesn't feel cruxy for the thing I care about here.

Ben Pace

With the companies, you're in some pretty competitive industry, and everything's a bit corrupt and mazy. And then, some new technology's invented that's much better than all the existing stuff, and someone makes a new startup and out competes all the other stuff. So, on one side, I agree, the person doesn't have to be like, "Grr, I hate these other companies. They're evil and need to be destroyed." They're just like, "I'm just going to do a good thing for the world." And the result is that the other companies just fade away. So, I can agree that I don't need to be going out to war with them or something. At the same time, the method isn't like, "How can I bring those companies together and all agree to use a new technology?" I'm just like, "I'm just going to go and make a better technology and out compete them."

I'm not trying to convert the companies, or reform the companies, or bring the companies together in spiritual healing. I'm just going to go and outcompete the companies, and this is better for the world. And maybe I'll read some books. I'll read Working Backwards by some of the high-level people at Amazon, about the core institutional tech that they have, like writing memos and starting with writing the public press release for the product before building anything. And there's stuff I can learn from them, and that seems good as well when I'm building my own company. But I'm not reforming Amazon when I'm outcompeting them. I'm stealing from Amazon, I'm studying them, and then I'm doing something better.

Ben Pace

I would also take this stance with regards to governments. My favorite way to improve governments is to let me please, please let me start a new country. I would much prefer this to trying to reform the FDA (and every other part of government). I think it would be faster and better. Unfortunately, the world is messed up and I'm not allowed to, but I do think it would probably be a better way of improving the state of government.

zhukeepa

Okay, I want to make an analogy with Muhammad. Muhammad first tried reforming Mecca by preaching to the Meccans, and it didn't work. They tried to kill him. Muhammad then managed to succeed in reforming Mecca by capturing it.

Ben Pace

With an army or what?

zhukeepa

With an army that the Meccans tried to kill many times. And then Mecca was reformed.

I think something analogous to that might have to happen, with Muhammad's army being analogous to the pluralist coalition, and Mecca analogous to the religious exclusivists.

Ben Pace

You'd like to become Pope?

zhukeepa

Would I what?

Ben Pace

Like to become Pope? I was teasing a bit.

zhukeepa

Probably not. There's a lot I respect about Catholicism, but I don't consider myself a Catholic.

Ben Pace

Sure.

zhukeepa

Right. So the Muhammad picture, I think, is actually pretty compatible with your picture. If Amazon is Mecca, then Muhammad's thing is a new startup.

Ben Pace

Well, it more sounded like he bought... No, no, no, no. Muhammad's thing is like Elon Musk and Twitter. He didn't outcompete Twitter. He showed up with all the money in the world and bought Twitter and then fired more than 50% of the staff. And now he runs Twitter, and it's a bit better (e.g. Community Notes).

zhukeepa

I think my main point is I think there's less difference in our visions than it might have initially seemed. In terms of what I'm trying to do, it's to figure out intellectual foundations for religions that let you translate between their core claims, and interpret their metaphysical claims in sensible ways. And getting a bunch of political clout around that, in favor of religious pluralism, seems pretty compatible with your picture of things?

Ben Pace

I don't think we are agreeing as much as you're saying we're agreeing. I think you're trying to-

zhukeepa

Okay. Where do you think are the concrete disagreements?

Ben Pace

I think you are saying your plan is to... I mean, again, your plan is more of a collaborative reform, whereas my proposal would be if you would like to fix religion would be to study the religions and learn what you can from them, and then start a better one.

zhukeepa

Sorry. I'm collaborating with the pluralists, not the exclusivists.

Part of my conception of the pluralists that I'm starting with here is that the pluralist's views are already pretty much mutually compatible. And there's also a thing where pluralists are already kind of shunned by the mainstream of the religions to a large extent. There's one pluralist Baptist that I know who was faculty at a Baptist university, and the students there led a prayer group for the salvation of his soul.

Ben Pace

He was a what? He was what?

zhukeepa

He is a Baptist. He is a pluralist Baptist. And he was a professor at a Baptist university, I think. The students there led a prayer group for the salvation of his soul because they thought he had gone totally off the deep end. And I'm like, "Yeah, that guy is the kind of pluralist I want to collaborate with." Not so much the students at the university praying for his soul.

Ben Pace

Reminds me about how some of us here feel about you!

So you want to take the pluralists, who are all the religious people who only care about the things the religions have got in common?

zhukeepa

More like who think that the things they have in common are more important than the things that are different between them.

Ben Pace

They're unique to their specific local brand?

zhukeepa

Yeah, that's right.

There's one way of interpreting the rites and rituals and dogmas of religion as the One True Way To Live, and if you don't live according to that, then fuck you. And there's another way where you can think of them as pointers for the masses of people to live in alignment with the real thing, but the real thing is what it's really about. And when you know what the real thing is, the rites and rituals and dogmas don't matter so much, and therefore it doesn't really matter if other people from other religions aren't doing the rites and rituals or adhering to your dogmas. That's a common pluralist perspective, I think.

Ben Pace

So you're doing this pluralist group, and with them, you are trying to figure out what the important things are that are common between all the religions. And then what's the next step?

zhukeepa

Have this coalition gain memetic dominance over the exclusivists' memes.

Why Alex doesn't want to start a new religion

Ben Pace

Okay. Why do you not want to start a new religion? I feel like in your shoes, that would be my next thought, and I don't know why it isn't yours.

zhukeepa

One thing is I don't want to be a prophet. Another thing is, what do you think are the substantive differences between starting a new religion and the kind of thing I'm describing?

Ben Pace

It's got a name, and it has some recommendations for how to live one's life that are very concrete and involved with things about having a rest day in the week and so on.

zhukeepa

I don't find that kind of thing very interesting.

Ben Pace

Religions also have a set of stories that become common knowledge amongst tens of millions of people.

zhukeepa

I think an anthology of the best of the existing stories would work well. I think the existing stories are extremely powerful. They're very Lindy.

Ben Pace

Look, stories are powerful and some institutions are there to tell stories about how to live life well. These are religions.

zhukeepa

Yeah. Sorry, I am trying to tell a story about how a positive singularity might happen, which is kind of a new story, where I'm invoking aspects of existing religions' conceptions of something like their version of a positive singularity.

Does the "Allah will protect me" story suck?

Ben Pace

Now, I do think your Muhammad story under the tree kind of sucks, and I could write a better version if I tried for a year.

zhukeepa

Strongly disagree there. But what do you think sucks about it?

Ben Pace

Just most of the time, that doesn't work, and it didn't tell you why it worked in this instance. Most of the time someone comes with your sword and you say, "Who's going to save you?" You say, "God." They go, "No," and then you die.

zhukeepa

Yeah. I think my crux for whether it's good is whether it's plausible to me that someone could actually be so charismatic that that could work.

Ben Pace

Charismatic can't be the right word.

zhukeepa

Yeah. I more mean "embody the source code for which that works". I've read stories of cult leaders who make eye contact and the people feel like their lives are changed from that eye contact experience.

But also, I agree that if there's no way for you to picture someone saying "God will protect me", and then the other person being persuaded by that, the story seems kind of lame. But I think if you have prior context, including from other anecdotal reports of modern cult leaders who have done analogous things, I think it seems less crazy as a story.

Jesus's empirical claim about evil

Ben Pace

All right. There was something else on my mind. I was thinking about the story of Jesus being forgiving as he is being killed.

zhukeepa

Yep.

Ben Pace

I'm trying to understand what was good about it. I mean, it's partly he's in a situation where it's very hard to forgive someone. But I think that's maybe not just it. It's also... It's arguing... He's implicitly saying it's correct to forgive that person or he is explicitly saying it, I guess. Forgive the people who are hurting him.

zhukeepa

I would say better rather than correct, but yeah.

Ben Pace

I guess the they know not what they do is an actual claim about not just I'm choosing to forgive them, but I believe... This is a fact about the world. I genuinely think that's an empirical claim that they don't understand.

zhukeepa

Yes.

Ben Pace

...they lack self-awareness of what they're doing.

zhukeepa

Yes. In contrast to the view you were putting forth earlier of some people just are evil or something.

Ben Pace

Or self-aware of their evil and hold onto it.

zhukeepa

Yeah. I think there's a way in which their self-awareness has a blind spot. Their self-awareness knows not what it does.

Ben Pace

Yeah, that's possible.

zhukeepa

It's like a pathological liar who is self-aware that they're a liar and they're just like, "Yep, this is the best thing for me to do and I endorse that." And I look at that, I'm like, "I see that. I think you're wrong anyway."

Ben Pace

Yeah. Although a part of me would want to be like, "He does know that he's lying right now. That's true." There's still a sense in which they don't understand. Anyway, I guess I was trying to think about it with other virtues or something. Like what would be a similar story to the Jesus story with the virtue of curiosity? And at first, I was just like, "What's the least adaptive situation for curiosity or the hardest time to be curious?" There's probably a situation in which curiosity is being beaten out of you by some government or something, or some situation where the world is very uninteresting and everyone is justifiably kind of bored by it

But I was trying to think about it for a while, and it wasn't just that it was disincentivized very harshly, but it was also that somehow it was still the correct virtue in that situation. It wasn't just that Jesus was doing a costly signaling of forgiving. It was also that he was saying, "Surprisingly, they actually don't know what they're doing." That's relevant. And similarly, if you were trying to be curious in a situation that was very unrewarding of being curious, a good story for curiosity would be one where it actually was the right thing to do, even though... I don't know. Even though it didn't save you personally, it still was the right sort of thing to do spiritually. But I don't know what that is. It's hard to write the story. Anyway. I'll think about it more.

Alex doesn't want to tell people how to live

zhukeepa

I don't want to start a religion because I don't want to tell people how to live. I don't like doing that.

Ben Pace

On the other hand, I think a lot of people don't know how to live and and would probably appreciate it.

zhukeepa

I am happy to make a blog about things that have helped me and a bunch of other people live their lives, but I wouldn't want to generalize it to the whole world. It's related to how for every piece of advice, there's equal and opposite counter-advice that some people need to hear more.

I'm pretty for writing out things I've learned on my spiritual journey, and for other people to do that, and for all that kind of stuff to come together on Steel-Arbital or something. But the thing that I'm wanting feels fundamentally decentralized rather than fundamentally centralized, while still remaining coherent, and telling people "you should live your life in XYZ way" feels too centralized to me.

Ben Pace

But... Sorry. There's a distinction between starting a new religion and being a prophet, although I understand that typically they're the same guy. But I think there are answers to questions like, what society-wide protocols would help people a lot? And I think a Sabbath is a pretty reasonable answer to that question. I don't think it's in-principle impossible for you to come up with good answers to these questions.

zhukeepa

I agree about that, but that also doesn't feel like the kind of question I'm interested in answering. For example, I think there's something good that comes from the ritual of praying five times a day at a set time as a culture. I don't think the world would be better off if everyone who's doing it just stops doing that, nor do I think it would be better if everyone in the world started doing that. And I'm saying this as a datapoint to illustrate why I don't want to be giving recommendations for how people should live their lives.

Ben Pace

I feel like a religion is aspiring to be a way of life for masses of people. I agree that one marginal person quitting their local religious traditions may hurt them, but I think if a whole city switches religious traditions... I mean, actually the first generation might also run into some issues or the first decade, but... I don't know. It depends on what time scale you're trying to invest in civilization at. But I think the five times a day one, it wouldn't surprise me if that one, on net, it would be better if everyone gave it up and picked up some other traditions instead.

I mean, I'm not certain. I haven't really thought about this one. I've not really chatted with people who do it. My guess is it's good to be able to focus on a single thing for more than five hours at a time. If you sleep for eight hours, then you've only got 16 hours, which means every three hours you've got to pray. I think you need to be able to focus on something for more than three hours is important. So my guess is that this one is not one that would survive contact with reality or that should not.

But I think once a week, most people taking a rest day makes more sense to me, though I'm not certain about it.

zhukeepa

Yeah. I want the pluralist representatives of each religion/atheism to be able to translate to the inclusivists who can then translate to the exclusivists. Where the exclusivists are like, "My way is the only way." And the inclusivists are like, "My way is the best way, but the other ways have good things, too." And the pluralists are like, "My way is one way out of many."

zhukeepa

I think I want to summarize my position around the thing I'm more interested in doing... Steel-Arbital is really at the heart of what I want, along with a vision for the future that pluralists from each world religion and "pluralist atheists" can all get behind, and then sort of have that vision percolate outward through the rest of the world through Steel-Arbital.

In closing

Ben Pace

All right. Nonetheless, I think it was important to get on the same page about, or to at least talk through, my issues with trying to rescue or reform religions. I think that was a part of the model that I was pretty suspicious of and would've felt bad about not talking through.

zhukeepa

Yeah. I'm glad we talked that through.

Ben Pace

I think relevantly, I brought up my criticisms and it didn't seem to obviously disagree with your plan. I think there's still other things we'll touch on, about what it looks like if you succeed at the thing you're trying to do.

zhukeepa

Yeah.

Ben Pace

But it was still good to chat through this.

zhukeepa

Yeah. Sweet.

Ben Pace

I'll see you next time. Thanks, Alex.

zhukeepa

Yep. Thank you, Ben. Goodbye.

Conversation 4 — April 28th 2024

We continued our dialogue over Zoom on April 20th 2024. The audio was also transcribed and lightly edited.

Intro

Ben Pace

I think I had a sense that there's a bunch of things you want to get from unifying the religious pluralists and the conceptions of good and so forth. And there's a bunch of challenges for how to enact this in the world such that everyone engages with them well. And there's a bunch of collective epistemology issues.

I think that you had some like leads for, "Here's how I want to go about doing it or here..." I don't quite know how far it was like, "Here is my plan for doing it versus here are some bits that would need to go well versus..." I don't know, I'm not exactly sure what your relationship. But it seemed like-

zhukeepa

Yeah, it's more like the latter, like here's the big picture for how things can go well. Or, if things go well, here's what that could look like. Here's a direction humanity could take that seems like it has some hope according to me.

Ben Pace

Did you want to add anything else to where your attention was going?

zhukeepa

I didn't really come up with anything else. I would just mention that what you'd said around steel-Arbital and steel-UpTrust point to the same thing in my head. I'll just call it steel-Arbital in this context.

Ben Pace

Yeah. All right. Do you want to start telling me a bit about either that or any other parts of the puzzle that you think you see? Or how to get those things to go well, or what would have to happen if things to go well?

Apparently: Agent Foundations = Religious Metaphysics = SteelArbital

zhukeepa

One thing is that I think:

the technical challenges involved in solving Agent Foundations
the technical challenges involved in finding mathematical foundations for metaphysics, in a way that can give accounts for the metaphysics shared by the mystical traditions of the major world religions
the technical challenges that we need to solve in order to build steel-Arbital

have very substantial overlap. Why I think this might be very non-obvious, and I think this is maybe an important thing to unpack before continuing too much further down.

Ben Pace

Yeah, I'm skeptical. Can you pick two of them and point how they're connected?

zhukeepa

Yeah. Let's pick the first two first, which is Agent Foundations and the religious metaphysics.

Ben Pace

So, Agent Foundations is asking questions like where are the agents? What is an agent? How can you tell if something's an agent? What makes something more or less an agent? How should an agent make decisions?

How should one generally make decisions? How should one form beliefs about the world? There's a bunch of bounded theoretical questions here that no one has answers to.

zhukeepa

Yes, that seems like a fair overview.

Ben Pace

The religious metaphysics is asking... I don't know what is that. "Here is a story for how the whole universe works (that's wrong)."

zhukeepa

What's wrong about it?

Ben Pace

You don't actually rebirth a lot of times. There aren't a bunch of devils who will burn you in hell for eternity. The soul is not a fundamental unit of physics.

I can see them as connecting to same questions, but I don't see them as like... I don't think the religious people are going to come up with any good math. I expect the Agent Foundations to answer some helpful questions about a place in the universe, not unlike, I don't know, Darwin's theory of evolution and helped answer some relevant questions in the universe. I wouldn't say that Darwin's theory of evolution was roughly the same thing as a religious metaphysics. I think they had very different methodologies for understanding the world and very different details as to how to apply them to understanding the world. So, it seems confusing to lump them. It's the same thing.

zhukeepa

That's a very helpful starting point.

Is Steel-Arbital harder than coordinating around not building AGI?

zhukeepa

For now, I'll continue by going into more detail around my vision for steel-Arbital.

You're a person. You've got conflicts with other people in terms of beliefs and actions. Fortunately, AIs give you the best versions of where they're coming from and show you ways you can coordinate with them, in addition to the ways that you're in conflict with them,. This results in gradual increases of mutual understanding and coordination, with the end result being mutual understanding and coordination on a global level. That's one piece.

The second piece is that I don't imagine this looking like everybody agreeing with each other. I more think that there will emerge / congeal a Schelling coalition of people who actually care about truth and doing good things, and understanding how to coordinate with the other people who are doing that.

And I imagine this coalition winning in the world, where part of winning involves Moloch not being an existential threat to humanity anymore, whether from AI or other things. That's the high level of what I'm picturing. And importantly, I think this Schelling coalition is going to involve a bunch of pluralist religious leaders and "pluralist atheists".

Ben Pace

This is all quite vague. What's Arbital got to do with this, or what UI design causes this that has not already happened? And why is it different from the current state of the world? Sorry. You might not have answers, but these are the sorts of questions I would like answers for.

zhukeepa

The tech that I specified was extremely vague. It's...

Ben Pace

I think I'm going to give up on trying to not be rude in this conversation because I think we're friends, and it's fine. And...

zhukeepa

Yeah, go for it. I said at the beginning that at a high level, this is what I want the tech to do. I think asking about what the UI should be is missing the point of what I'm trying to say.

I'll also give up not being rude. I think the default vision that AI people have of a positive singularity is repulsive to most of the world, and for good reason. What I'm trying to do is rescue the good parts and find a thing that's more compatible with what the rest of the world cares about.

Also, the positive vision that most religious people have for the end of the world is repulsive to the other people who are not part of their religion, and for good reason. And the thing I'm trying to do is rescue the good parts and put them together, paint a picture of what it looks like and a rough picture of what it might look like to get there.

There's a thing we want the tech to be doing. And I want to emphasize where the tech can help with peacemaking, instead of just scientific development and material abundance. I think scientific progress and material abundance are very cool and may end up being very helpful, but I think peacemaking is a more central component of my picture.

Ben Pace

I'm not especially arguing for specific vision of the future. I don't have a particular governance proposal in mind for post-utopia. If I wanted to get one, I'd probably go and read Bostrom's new book. And he'll probably have some interesting suggestions. I'm mostly against the "definitely lose control of the future" aspect where you just build a different species and then it kills you and moves on. I think that's the minimum future that I'm thinking we can all get behind. That's the response to one of these things.

zhukeepa

We are aligned on that. I think the next piece of the picture I'm trying to gesture at is getting clearer on what it actually looks like for tech to help get the world to not be vulnerable anymore.

Ben Pace

Yeah, I know. And you were saying you have dreams where everyone can come together and coordinate really well. But that seems vague and stronger than what I need. I just need everyone to agree to not build AGI until we solve the alignment problem. This is a far weaker problem than I would like to solve than all coordination between all humans.

zhukeepa

I think that's politically impossible. I think...

Ben Pace

But your thing is harder than that.

zhukeepa

I disagree. How are you going to beat Moloch? How are you going to stamp down Moloch? Bostrom once proposed mass surveillance, but I think that's not going to be politically feasible. I think any such approach would collapse because of internal politics, and it also won't have moral legitimacy with the rest of the world. That's why a "benevolent dictatorship" is not going to work.

Ben Pace

I guess I was going to make everyone feel a bit ashamed for doing it and low status and scared, similar to how lots of people aren't doing human genetic enhancement—

zhukeepa

That's definitely not going to work.

Ben Pace

—lots of people don't do human genetic enhancement just because they're like, "Everyone will think I'm shitty for doing it and be mean to me." That's a pretty strong force in human civilization.

zhukeepa

That is nothing near the strength of the forces of wanting military and economic advantage. So, it's not going to work.

Ben Pace

We can argue about whether my thing works. And I'm happy to defend a little bit more. I was making it up on the spot, but again, I was like, I guess I don't understand why your thing isn't a superset of my thing. And I'm like, "We will have to solve the specific problem."

And you were like, "Not only will we solve this problem, we will solve all problems about coordination." I don't understand how your thing isn't just like... mine isn't just... I'm just like we should solve this coordination problem. You're like, oh, well, the software will solve all coordination problems, and then we'll manage to easily solve this one.

zhukeepa

I don't think I'm trying to say "solve all coordination problems". I think I am trying to say that whatever approach for the coordination problem you have in mind runs into fundamental difficulties, especially around overcoming Moloch. If you think the problem is to beat Moloch, I'd be on board with you, but I'd ask you how you would beat Moloch without a benevolent dictator, because I don't think benevolent dictatorship is really a thing.

Ben Pace

What are you saying? I guess, still, I was like, "Be specific about what this Arbital looks like." And you were like, "That's not the point. The point is to have a decentralized process for coming up with true arguments that everyone trusts."

zhukeepa

That's not the full picture. That, I think, is going to be critical infrastructure for the Schelling coalition to have the moral authority to win in the world -- in other words, to have a "decentralized benevolent dictatorship", so to speak.

Ben Pace

By winning in the world, you mean build a global governance or dissolve national borders or something. Not like that's the definition of it, but that's a part of it.

zhukeepa

Something like global governance is closer to the mark of what I'm picturing, but in a way that's decentralized and continuously earning the trust of everyone it's governing. It would be through a style of governance that's very different from anything that currently exists.

Ben Pace

But I guess I may be hearing you maybe mistakenly. You're saying, "I'm not going to go out and build a global governance. I'm going to and go out and build technology for causing people to have trust in these arguments in general. And then they will figure out that they should have some global governance. And they will trust it. But the thing I'm doing is not launching a flag for global governance. It's for people being able to congeal arguments."

zhukeepa

And including for what should be done. I think it should also congeal values.

Ben Pace

Well, what? Values...

zhukeepa

And by congeal, I don't mean everyone should have the same values. I mean figure out where they can be reconciled and where it's too expensive to reconcile them.

Steel-Arbital as a preference synthesizer

zhukeepa

Have I given you my analogy of the elephant with a blind man?

Ben Pace

It's a standard analogy. Five guys...

zhukeepa

Yeah. So, the picture I have in mind is there's something like moral realism that's built into the steel-Arbital. I think you can't have a thing that does good discernment for reasoning without a solution to moral realism. I can go into that as a separate bit, but the TLDR is that the discernment of whether an argument is correct is value-loaded. And I don't think that means arguments are irreconcilable, because I believe something-like-moral-realism is true.

I think all our values come from somewhere and can be ultimately reconciled. We can go into that, but I'm not going to go into that right now. My main point is if you imagine an upgrade of democracy where people have conflicting views about how society should be run in service of the common good, but you think of these conflicting views as different parts of the elephant of how society actually should be run, the thing that I want from the technology is to be able to figure out that synthesis and communicate that synthesis to each person.

I am just now realizing that "convergence toward a synthesis of preferences" is just as important as "convergence toward a synthesis of beliefs" in my vision of steel-Arbital, and I want to apologize for not making that clearer earlier.

Ben Pace

Can you say slightly more detail about how you think the preference synthesizer thing is suposed to work?

zhukeepa

Well, yeah. An idealized version would be like a magic box that's able to take in a bunch of people with conflicting preferences about how they ought to coordinate (for example, how they should govern their society), figure out a synthesis of their preferences, and communicate this synthesis to each person in a way that's agreeable to them.

Ben Pace

And this is the project that my colleague, Jacob Lagerros, is currently working on. He's building an app called Octopus, where we all sit down and we all... The dumb thing it currently is for is what restaurant do you all want to go for dinner? And you all chat with Octopus. And you talk to her about preferences and so on. And at some point, it just tells you, you are all going to this place. And everyone was indeed down for it. He's trying to—

zhukeepa

Sweet.

Ben Pace

—make that work for other things. Anyway, maybe I shouldn't have interrupted you.

zhukeepa

I liked that.

Ben Pace

Okay. So, you want a preference synthesizer, or like a policy-outputter that everyone's down for?

zhukeepa

Yes, with a few caveats, one being that I think preference synthesis is going to be a process that unfolds over time, just like truth-seeking dialogue that bridges different worldviews. I think the idealization of a magic box that instantaneously spits out synthesized preferences might not be possible.

Another caveat is that people will only be happy with the synthesized preferences if they're willing to admit where they're wrong. If a young earth creationist tried to have their preferences synthesized with a bunch of reasonable scientists around what schools should teach in their local community, they might not be happy with the preference synthesizer telling them that the earth isn't 6000 years old.

Killing Moloch with a decentralized singleton running on Steel-Arbital

(What would you do with a trusted preference synthesizer?)

zhukeepa

The core hope I have is that the set of people who can actually coordinate with this preference synthesizing process can out-compete the kinds of people who wouldn't, in terms of getting to run the world.

Ben Pace

That's one of the words that commonly hides the most complexity. "Out-compete", a phrase which here means "starts new countries and wins wars and builds new religion". It's just such a deceptive little word...

zhukeepa

It's not necessarily starting a new country. There's one version of this where enough people inside existing governments are able to coordinate with this and then do reforms from the inside out. And nations still exist, but they're just all way friendlier with each other.

Ben Pace

Sure. I want new countries so bad. I think if we could have new countries, I might be like, "Finally, there's hope." Anyway. It's good that your plan is in fact quite ambitious.

zhukeepa

Again, I feel like it's less of plan and more like... there's one vision for defeating Moloch, which is benevolent dictatorship, and I'm proposing a different vision of preference synthesis.

Ben Pace

Well, I don't know what you want exactly, but I feel like you need to say something that leaves me with a reason to believe that there's clear gains to be had in preference synthesis. I think using machine-learning assistants to do it is of a different type signature than most other proposals. And that's somewhat tempting.

zhukeepa

Sorry. The clear gain is that it's a possible attractor state of the world in which the thing that's in power isn't a centralized dictatorship and isn't eaten up by Moloch.

Ben Pace

Why isn't it eaten up by Moloch?

zhukeepa

Part of my vision is that it outcompetes the people who aren't participating in this process. And the Molochian incentives that misalign with the common good would get noticed by this preference synthesis process.

Ben Pace

All right, but I'm not sure whether you're saying, "By the way, Ben, unfortunately, I think in order to save the world, we'd have to kill Moloch. We probably can't do that. So, we're all screwed," or whether you're like, "Ben, I think I have some details on how we can kill Moloch. And I think it can work out."

zhukeepa

It's more like the latter. Is your current position like, "We probably can't kill Moloch. So, we're probably all screwed"?

Ben Pace

[Long pause]

I think my current position is I haven't heard of a plan that would kill Moloch. I don't think I would say the words "We definitely can't kill Moloch," but I have no plan or I have no concrete bets. I couldn't tell whether you were saying, "Well, we will have to, even though I don't know how to," or whether you were saying "I have a good way of solving it".

zhukeepa

I think my picture is basically that the Schelling coalition / the preference synthesis coalition is going to have minimal Moloch internally within it, and that they become powerful enough to determine how the world runs. This is kind of like killing Moloch, and the process by which this all happens is pretty much how I interpret religious prophecies about the end of the world.

Ben Pace

So, is this a singleton but it's internally decentralized?

zhukeepa

Yes.

Ben Pace

Okay. Seems like a nice idea. No, we could try and talk more about, concretely, about preference synthesis and what would happen in the world if people were better at it. Maybe I'll just keep—

zhukeepa

That seems worthwhile to do. I think, insofar as there is skepticism behind your "would be nice," or something, I think I want to hear more about that. Or is it more just like, "I don't really know how to picture it so it's hard for me to comment on it."?

Ben Pace

I wouldn't say I had a very specific skepticism to share, which kind of sounds like I don't have a concrete enough vision of it to comment on. Yeah, I think that's probably where I'm at on it.

Steel-Arbital should reveal and forgive blind spots

zhukeepa

Yeah. I think the thing I'm wanting to say right now is a potentially very relevant detail in my conception of the preference synthesis process, which is that to the extent that individual people in there have deep blind spots that lead them to pursue things that are at odds with the common good, this process would reveal those blind spots while also offering the chance to forgive them if you're willing to accept it and change.

Ben Pace

Okay. I still can't tell when you say these things whether you're saying them as, "These are just the 17 aspirational things we've got to hit," versus you're like, "I think I can see how you can get these."

zhukeepa

Closer to the former, but it's more like "if the process is working at all, this is going to be a crucial component of it."

Ben Pace

Yeah.

zhukeepa

The reason it felt relevant for me to name is because I think in my implicit model of people's implicit models of conflicts, a lot of where conflicts ground out is other people just not getting how the thing they're doing is bad, and furthermore seems to be more interested in doubling down than caring about getting it... but also, everyone's kind of like this.

Ben Pace

Well, yes. I think this is most of the way that things go badly is that people don't understand the bad... We could go into detail on that and I think we would end up agreeing with it. I currently don't bet that it's literally all of it. I think sometimes people like to-

zhukeepa

I wouldn't either.

Ben Pace

Yeah. Just to follow up on that, I generally have noticed that when I'm in conversations with folks where I'm like, "I'm pretty sure the other person here has wronged someone," there's a pattern I've noticed where they seem unable... they specifically, they use the word bad. They're like, "Oh, that's bad," and they weren't able to concretely say what was bad about it. They just said that it was bad. They couldn't describe the detail of the cost that was imposed or whatever. So, I have a personal strong heuristic to taboo the word bad pretty heavily, to go, "Oh, yes, it'd be bad if that happens. Sorry, what I mean to say is, 'This specific cost would be imposed on these specific people,'" or whatever. I generally also get the sense that people have blind spots around the places where they inflict harms, but anyway, we could ground that, I think.

zhukeepa

Cool.

Ben Pace

Why was it relevant to you?

zhukeepa

A critical mass of people sort of letting go of core blind spots is just a concrete thing that I think would be part of this process, that I think is the sort of thing that most people don't think about very much as a possible way the future could go.

Religion as a tool for coordinating with the masses

Ben Pace

It seems hopeful. Seems kind of unrealistic, but maybe that's a bit on me for not really chatting with half the world whose IQ is below 100.

zhukeepa

I think the way you coordinate with those people is with celebrities or something.

Ben Pace

I really would prefer stories. Or, that seems wrong-headed to me, but I feel like you just said the way you help people in their marriages and the way you help people become better parents is via celebrities, or the way you help improve schools is via celebrities and I'm like, I think–

zhukeepa

The way you help people improve is by showing them people they trust who are doing the thing that would be an improvement over where they're at and the people that most normal people trust are celebrities. The people they look up to and think to themselves, "I want to be more like this person."

Ben Pace

There were two things. One, just because someone is successful in front of me, it doesn't mean you understand how to be successful. They're like, "My kid hurt me in this way and I forgave them for it."

"Ah, I guess I should always forgive my kids." It's not the rule. The rule is a much more complicated negotiation. Just because you can see someone nailing something doesn't mean you've now learned how to nail that thing. Secondly–

zhukeepa

Sorry. I'm saying this is the general avenue with which you can reach people like this. If there's celebrity-endorsed content that goes into the specifics of how you can actually do things better... the only thing I'm trying to say is that it is very tractable to communicate things to normal people.

Ben Pace

Hey, but you're into religions. Why didn't you say the great way to do it is to have a priest in every town who read some good stories about how to be virtuous?

zhukeepa

Because people find celebrities more compelling than religions.

Ben Pace

Well, I think in their free time, but I think fewer people would be willing to-

zhukeepa

For the devoutly religious, then yeah, framing in terms of the Bible would also work. The point is just to, for any given person, take whoever their moral authorities are, who are actually aligned with the coalition, and then filter it through that. Filter the message through whatever language the moral authorities would use.

Ben Pace

Look, if we had 50 years to build a new religion, I think we could do better than celebrities. I think we could have better books and stories and reading groups and local preachers and better websites and communication technology, like Google Docs but better, for discussing ethics and getting problems solved and having anonymous conversations with people. I think people should have more anonymous conversations about the problems in their lives with other people.

zhukeepa

For the kinds of normal people who generally seek religious advice from priests, the thing I would want the AI to do is to quote Bible references in a compelling way, and in particular in a way that is compelling relative to what they're experienced with.

Ben Pace

I'm saying if I had 50 years, we could write more stories and the AIs could talking through those stories in sort of personalized one-on-one dialogues when they needed it.

zhukeepa

That's part of my picture as well. It's just that if you're starting with the Bible as a moral code, I think there are pluralist interpretations of the Bible that are compatible with how to actually be a good person. I don't think the fundamentalists would have much of a role in the preference synthesis process in the beginning. They would more be like people in the outskirts who would get integrated later on, if ever. Zooming out a bit, I do feel that part of what's going on here is optimism around coordinating with normal people versus pessimism and hopelessness around that, or something like that.

Ben Pace

Right. It's not just quote-unquote "typical" people, but it's also the existing structures of the world, like governments and countries and regulations and trades. I think it's not hard to get a bunch of people in a room to agree to a superior option. I think it is much harder to in fact change legal structures or change the way companies work. That takes longer time and has a lot of annoying shear forces.

Steel-Arbital vs religious visions for a fixed world

zhukeepa

I agree with you, and for what it's worth, in my visions for how this would work, there would be a coherent decentralized process that's pretty small at first. But as existing power structures fall apart and dissolve under their own weight, that's when the Schelling Coalition actually expands in power. It's when the existing structures dissolve and people are needing alternatives that people will turn to the Schelling Coalition, as the Schelling alternative that's more nimble and stable and trustworthy than whatever they were leaning on before.

Ben Pace

I don't know how to do that, especially quickly or even very slowly, with nation states, or whatever you're supposed to call them, state actors. If I could be like, "I have a new country. You can just come to my country, and it's got a better legal system that I can start afresh with," then I could compete with it relatively quickly. I don't buy a story of, like, "Ah, yes, within 20 years, France and Canada's legal structures will just fall apart and they'll be looking for some other better one." I'm like, "Nah. They've lasted hundreds of years. They'll stick around. Probably they will just stagnate and persist. They will persist, rather."

By the way, this conversation isn't not fun.

It's kind of fun to just make up how to fix the whole world.

But to be clear, it's fun in the way that it's probably fun to write fanfic.

zhukeepa

It feels relevant that I kind of am picturing the Schelling Coalition as a meta... It's not like a new religion or a new country, but it is kind of like a meta-religion and a meta-country that has a coherent worldview instead of beliefs and norms about how to operate. Maybe there will be a president from the Schelling Coalition and that's one way a lot of change could happen.

Ben Pace

Sorry, there'll be a president? Okay.

zhukeepa

Yeah.

Ben Pace

Yeah, anyway, I'm getting a sense you're like, "My plan is to fix the world and here is a bunch of details for how to do it." Sorry, plan is a strong phrase. "I think we need to fix the world and here is a bunch of details about how to do it."

zhukeepa

More like a vision for a fixed world.

Ben Pace

Okay. "I have a vision for fixed world. Here's a bunch of details to it," and I'm not sure I buy any of the details.

zhukeepa

Yeah, like which details?

Ben Pace

"There will be a political coalition amongst all the players in the world and it'll be the best one and it'll beat everyone else. They will be able to solve preference problems. They will be able to synthesize all their preferences into a unique thing via some unspecified computational method."

zhukeepa

Hold on. The goodness of the coalition comes from the goodness of truth. There's this trope where good people are better at coordinating with each other than evil people, and I'm basically picturing this as like, yeah, the coalition of the good people, and in the epistemic domain, the coalition of people who are actually trying to figure out truth in a real way.

Ben Pace

Oh, all right. I feel like there's this thing I thought when I was reading through the previous dialogues where I was like, "Insofar as you think religions can save the world, I want to point out that we already had a period of history of religious dominance where they got really good shots at saving the world."

zhukeepa

Or they got, like-

Ben Pace

"Every smart person was religious and they had all the power in the world, and then they didn't save the world and they were kind of stupid, and it was good that they're in less power today because they're epistemically corrupt, and they hurt a lot of people." I think that trying that again doesn't sound to me like a winning strategy. I mean, you can be like, "We'll do it better," and I'm like, so-

zhukeepa

Trying what exactly again?

Ben Pace

I don't know, having a big power structure over the people's lives that is built around stories of virtue and goodness and that has the storytelling leaders in every town and is-

zhukeepa

Okay, all right. I object to the notion that they've already tried to save the world many times and failed. They all have prophecies that say, "We're just starting. The world will get saved in the end days when the truth of religion actually ends up dominating." And under exclusivist interpretations, this is kind of dumb, but under pluralist interpretations, they're all basically saying the same thing, according to me at least.

Ben Pace

Well, that's interesting. That seems like it's a standard attractor, then.

zhukeepa

Yes.

Ben Pace

Doesn't mean it's truth-tracking.

zhukeepa

That's right. I think whether the attractor gets realized is an empirical question that's up to the actions we take as a civilization. I don't think it's preordained to happen.

What religions say about starting new religions

zhukeepa

I want to hear more skepticism.

Ben Pace

Why do you want to hear more skepticism, Alex?

zhukeepa

As inroads to cruxes.

Ben Pace

Why do you want cruxes, Alex?

zhukeepa

Because I want to double crux with you.

Ben Pace

Why do you want to double crux with me, Alex?

zhukeepa

Because I appreciate this attempt at bridging worldviews. I appreciate this attempt to find where my informational gaps are, and also getting clarity on where there are weaknesses in the pictures that I have.

Ben Pace

Yeah, I still don't really get why you don't want to start a new religion.

zhukeepa

Because I respect the existing ones.

Ben Pace

You know there's, like, dozens?

zhukeepa

Yeah, I respect the ones that I've interfaced with.

Ben Pace

No, I know, but each one of them thinks the thing you're supposed to do is start a new religion.

zhukeepa

You know in the Quran it says that the true believers of Christianity are also going to get saved?

Ben Pace

Losing their edge, I tell you! Nevermind. Okay, I didn't know that.

zhukeepa

And the Jews, as well. I don't know what you mean about "losing their edge". This was in the Quran. I think Muhammad saw himself as-

Ben Pace

So, you make your new religion and it's better than the other ones, and then you'd be like, "Had I existed at the time of the Quran, the Quran would've said that the true believers of my religion would also go to heaven, or be saved." It's not an argument against starting a new religion.

Revisiting agent foundations vs religious metaphysics vs steel-Arbital

zhukeepa

Okay, I think the place I want to go for a sec... I want to go way back to the beginning, around... steel-Arbital (/ preference synthesis) versus agent foundation versus religious metaphysics. I think the ways in which those feel very similar feel very relevant to my picture.

I think physicalism is wrong. I think you can interpret souls as the source code that is sort of the object of FDT. That's not a material thing that ends when you die.

Ben Pace

Yes. Math also exists. Multiple truths are true, irrespective-

zhukeepa

There also isn't as clean a distinction between math and matter as one might commonly assume.

Ben Pace

I agree with these things.

zhukeepa

Yeah. I think the heart of religion and the heart of what decision theory says about how we should act are in alignment. I think for the right notion of identity, which is related more to your FDT source code, you can reasonably think in terms of reincarnation.

But it's also very easy to make dumb mistakes around reincarnation if you don't have the right metaphysical sophistication, which Buddhists heavily emphasize around how they talk about rebirth, for example.

Ben Pace

I'm not saying it's never a nice idea to think about it as though you were being reincarnated, but I think the parts of me that are in the other people are my virtues, but not a lot of my personal and individual characteristics, and it's certainly not my memories.

zhukeepa

I'm with you there.

Ben Pace

Of course you're with me!

zhukeepa

I think you're stereotyping religions as having wrong metaphysics, and I think that's true of naive interpretations of religion, but-

Ben Pace

Also, all the things they would've said 500 years ago, they would've just kept saying false things. "God directly made the people," as opposed to, "Evolution evolved the people." That's not a great phrasing, but you know what I mean.

zhukeepa

Yes. The people who say that still are pretty embarrassing.

Ben Pace

I'm not trying to be rude, but they would be like, "No, no. They were right the whole time, if you think about it."

I'm like, "No, they were wrong. The things they said were false." There was some useful stuff in what they said, but anyway, I don't know why-

zhukeepa

Yeah. They wrote about a lot of things and they made a bunch of major mistakes. You'd previously characterized religions as having obviously wrong metaphysics and I feel like there's something meaty there in our disagreement. When I was talking to Catholic monks two days ago, I thought to myself that their metaphysics seemed wrong. And so on that level I agree, but that's not the thing that I'm trying to talk about with religion here.

Ben Pace

All right. If you want disagreements and stuff, I can try and say more things like, "I think it would be reasonable to try and make a new religion, and there are interesting and useful things to learn from the previous ones about how to relate to death and child-rearing and good and evil, and you should go and take those from it, but I'm not on the page with you of reforming them. They seem obviously awful and horrendous and dysfunctional. Your attempts to reform these institutions seems obviously dumb." Sorry, I'm not sure I... whatever. But it seems like there is stuff that they have been able to talk about with regards to good and evil and child-rearing and death and so forth that is useful to get from it.

zhukeepa

I'm not trying to reform, I'm trying to work with what exists rather than against. I think in the ways that matter, the sort of Schelling Coalition I'm describing is, for the relevant intents and purposes, a new religion. In many ways it's going to be way more powerful than any of the past religions because it's got the power of technology. Rather than designing a standard set of new stories, the AIs in this can craft their own stories that are good, in terms of being persuasive and fruitful and ethical. In some ways, it is a new kind of nation or a new kind of religion.

Ben Pace

Sorry, did you follow up on the promise of why agent foundations are the same as preference synthesis is the same as the religious metaphysics?

zhukeepa

I was giving overlaps between agent foundations and religious metaphysics. I haven't spoken about preference synthesis yet, but basically I think you need a moral-realism-like-thing -- universal laws for coordination -- that the preference synthesizer would have to be based on. Otherwise, you can just have irreconcilable values that can't be synthesized.

I think these universal laws of coordination are at the heart of what religions are actually about, and require something resembling the metaphysics that I claim are shared in the mystical traditions of all the major world religions in order to properly formulate.

If we figure out agent foundations, I think we're also going to figure out this metaphysics, and these universal laws of coordination. And I think having a technical understanding of these universal laws of coordination are a necessary prerequisite for building trustworthy preference synthesis technology.

Cruxes around coordinating with the masses

Ben Pace

Why don't you ask me a question about my worldview?

zhukeepa

[Long pause]

Why are you so down on coordinating with the rest of the world?

Ben Pace

I mean, I believe in arguments and evidence and conceptual understanding and research and science and breakthroughs. I don't believe in social media and I guess I don't believe in most governments as a space to reliably have true things happen to you and not moral mazes.

Generally, I don't know, communication seems hard.

zhukeepa

What seems hard about it?

Different question, actually.

Ben Pace

Well, briefly, the answer is something like reality has a surprising amount of detail and when you try to get two people to look at the same bit of reality and to have a shared model of it. It's not in fact a simple task. I'm not saying it's hard. I'm saying it's not even simple to get on the same page about that part of reality because the reality itself is not simple.

What was your other question?

zhukeepa

How do you feel about how HPMOR Harry treated Ron Weasley?

Ben Pace

I think the real Harry Potter, the original Harry Potter would've been upset about it. I think it seems like a very natural way for things to go in-universe.

zhukeepa

I think he was an asshole.

Ben Pace

Which character?

zhukeepa

HPMOR Harry to Ron Weasley.

Ben Pace

I think you got to let that sort of thing go. I mean, you don't want to get into fights with everyone who's dismissive or doesn't get along with you. You're going to meet thousands of people in your life. Not that important.

zhukeepa

If this is your attitude, it makes sense why we might have different views about coordinating with the rest of the world.

Ben Pace

Why? I might be forgetting how he treated him, but I mostly remember him being fairly dismissive and uninterested and maybe once or twice a little bit rude. I don't remember anything especially egregious. Was there anything especially different than my recollection? In your recollection？

zhukeepa

Your recollection sounds fairly accurate. For what it's worth, I'm pretty up for people not wanting to hang out with someone or spend time with them or learn that much from them.

Ben Pace

They're not a married couple. They're not even... they're essentially strangers to each other.

zhukeepa

Yeah. I think he is rude to strangers. He was rude to that stranger. I'm not holding this... I'm not like "fuck Eliezer for that". I'm more like, well, obviously if you're someone who has a superiority complex around people like Ron Weasley, you're obviously not going to be able to think you can coordinate with the rest of the world. Because whatever dimensions along which you could coordinate, you're kind of just preemptively dismissing. You're like, I've got my ways of operating and my ways of coordination and if you can't meet that, then you're not worth my time.

Ben Pace

I'm tempted to argue in defense of the opposing position out of an instinctual disagreeableness that seems healthy, but insofar as it seems like strongly morally flavored to you, I think my main response is I haven't really thought about it very much and I don't have a strong personal take.

zhukeepa

I don't endorse any moralistic elements of what I'm saying here, and to the extent I'm being moralistic I don't endorse it. I'm just trying to say, from a matter-of-fact standpoint, that if this is your attitude toward normal people, you probably won't consider it tractable to coordinate with them.

Ben Pace

I mean, is there a part of the world that I should be coordinating with that I didn't mention? I think about companies and industries and I think about governments and politicians and I think about activism and protest movements and I think about sort of online communities and social medias. I'm not thinking specifically about certain dudes I know who clearly have IQ 75, who I knew when I was a teenager, but I'm not thinking about those specific people.

zhukeepa

In particular, all of the parts of human psychology that are irrational and unreasonable, I think you are not very willing to coordinate with. I think you are willing to, when you see them in individual people, put them aside and coordinate with the rest of them, but I don't think of you as being very willing to coordinate with the irrationalities in and of themselves.

Ben Pace

Do you mean I'm not aware of them or do you mean I'm not willing to...

zhukeepa

Work with them. I can imagine a dialogue with you where I'm like, Ben, what if we coordinate with the Ron Weasleys of the world? And you're like, but they're so lame. And I'm like, okay, they are kind of lame in the ways you're saying, but they're also people, they want to help and they can be very helpful, and actually most of the world is made up of them.

Ben Pace

I mean I'm open to the feedback slash critique that I'm not very in touch with median people or below median people, and being like, I think you would actually have a better model of the world if you did that more often is I think I would take that as a pointer in that direction and a reason to move it up to actually prioritize it.

zhukeepa

Yeah, I think you're saying something slightly different, which is something like...

Ben Pace

Well, I also, I don't know how much they respect each other.

zhukeepa

You don't what?

Ben Pace

I don't know. This whole conversation is kind of, it sounds kind of super classist or something in a way that I don't quite endorse. But anyway, I don't know how much such folk respect each other.

zhukeepa

You don't know how much what?

Ben Pace

I don't know how much the population of people whose IQ is between 70 and 100... I don't know how much they respect each other and whether they respect each other more than I'm respecting them. I think many such folk just getting through the day and everyone around them is occasionally source of interests and sources of annoyance.

I don't know. Probably that's too cynical of me.

Anyway, we can go hang out with some low IQ people sometime, next time you're in town if you like. Or I guess maybe I'll just find a place to do it.

zhukeepa

I can do that. I have a friend in mind.

Ben Pace

You have a friend in town that maybe sometime we could hang out?

zhukeepa

Yes. I am serious. I've hung out with him and I really like hanging out with him.

Ben Pace

Yeah, I'd be up for that sometime next time you're in town.

zhukeepa

I should wrap up soon-ish.

Ben Pace

Yeah, I think we're 10 minutes over. I also was hoping to leave on time.

Ben finally gets why Zhu doesn't want to start a new religion

zhukeepa

Cool. Yeah. I have one closing thought. Something feels important about the way you kept asking "why aren't we just trying to start a new religion rather than... isn't that clearly better than trying to reform the existing religions that are kind of obviously fucked, Alex?" Whereas my frame is more like, the thing I'm doing feels like it effectively is starting a new religion, in the same sort of way that it's also starting a new government. But, part of the deal is that this sort of "new religion" or "new government" is so different from what people typically mean by "religion" or "government" that using the same term feels misleading.

But even still, on a vibe level, when I imagine you saying "new religion", it's got more of a vibe of sort of just brushing aside all of the existing religions, whereas in my conception of things, it's very important to work with the existing stuff rather than brushing it aside.

Ben Pace

Yeah. I feel more like a Jeff Bezos kind of like our relationship to our competitors. Amazon is that when anyone says, what do you think about competitors? I say, what do you think about the users? Focus on the users. And again when you're like how do you build good stories and institutions that help people live good lives? I'm not like, well what does Christianity, what do the other people say? I'm like, what is the answer to that question? And how can we build an institution that embodies it?

And you might be like, by the way, do you know how this other religion, I was like, oh, that's a good hint. That's a good tip. I like some things there. I'll take some of that, but I'm not trying to reform it. It's not my first go-to. My first go-to is just focus on the answers to the question and to figure out how to build an institution around it that's fresh. It just, anyway, they're just so old. They were made thousand years ago. Most questions, they're kind of like, you should make Google Docs or you should have Glowfick. There's so many of their bottle mix we don't have anymore. So I don't expect their answers to help that much. And so many of the environments that there are wisdom was set up for very different to these current ones.

I do want to say something else in wrapping up in a second, but maybe you can respond to that first. More the logistical side effects.

zhukeepa

Yeah, in some sense, a core thesis of where I'm coming from is that I feel like working with people's irrationalities is like focusing on the users rather than the competition. For people in Christian Western culture, I think using the language of Christianity in good ways can be a very effective way to reach the users. That feels to me more like the kind of thing I'm doing than reforming existing religions. I'm agnostic as to what happens to the existing religions. They might just crumble. They might successfully get reformed from the inside, or they might turn out to be too bureaucratic and decrepit for that to happen. I don't know, and I don't really care.

Ben Pace

Okay, so you're saying you're trying to talk with people about how to live life well and so forth. And this is the most direct way you have to do it at the minute. And that's why you're using religion. Okay, That makes sense to me. That helps understanding your perspective.

Going forward

Ben Pace

So it sounded like maybe, so this is the fourth one of these we've done, I think? The first one was written and then this is the third call.

zhukeepa

Yeah, that's right.

Ben Pace

Okay. I want to say something that's puzzling to me, so it sounded like maybe we were going to have to wrap these up anyway for a while because we're busy. And I was going to say, I feel like we've done a fair bit of exploration, and it seems plausible that maybe we should pause here and come back if we have a different idea, and maybe try something else in the future, if we want to. But also we could just stop.

zhukeepa

I'm down to keep going. I definitely feel like there's still more to talk about, but I think I'm down either way.

Ben Pace

Okay. Nonetheless, I think you would say you were like, I'm busy the next two, three weeks.

zhukeepa

Yeah.

Ben Pace

Okay. Yeah, that also helps me, it works for me because I have a lot of work that I need to do and I'm probably going to be working weekends starting tomorrow.

zhukeepa

Okay.

Ben Pace

It's also plausible to me that... I can imagine that conversation being very different if we have a third person involved or something.

zhukeepa

I would be open to that.

Ben Pace

But I guess I meant someone, I didn't, to be clear, I didn't mean a bridging person. I meant a person with their own perspective or something that was a bit distinct from either of us.

zhukeepa

What about my Imam friend?

Ben Pace

That sounds so annoying, but it does sound like it would probably be somewhat interesting.

zhukeepa

I think you'd be surprised.

Ben Pace

I also think I'd be surprised. I also think I would be annoyed.

zhukeepa

Sorry. I think you'd be surprised by how not annoyed you might be.

Ben Pace

I mean, it depends what we talked about.

zhukeepa

Yeah.

Ben Pace

Anyway, that sounds incredibly annoying, but I'm probably open to it anyway.

So let's pick a date to check in. We should try something slightly different than. My vote is we don't just have exactly the same calls, just some more of them. I think we should do something slightly different. I don't know, watch a talk together, or invite a third person, or go and hang out with your friend.

zhukeepa

That sounds good. Let's just have something on the calendar. I should really get going now. Thanks, Ben.

Ben Pace

Thank you too. Okay, bye-bye!

zhukeepa

Goodbye.

Dialogues