I'm not sure why all the people who think harder than I do about the field aren't testing their "how to get alignment" theories on humans first…

Some of us are!

I mean, I don't know you, so I don't know if I've thought harder about the field than you have.

But FWIW, there's a lot of us chewing on exactly this, and running experiments of various sizes, and we have some tentative conclusions.

It just tends to drift away from LW in social flavor. A lot of this stuff you'll find in places LW-type folk tend to label "post-rationalist".

While your point is technically true, it's not relevant here. Bezzi's point stands even if we just talk about trans folk whom most people can readily tell are trans.

…I think most decent people would be willing to sacrifice their own life to prevent their civilization from going extinct, and I think it would be a very honorable thing to do.

While I agree, I want to take the opportunity to poke at something I often see in models like this one.

I think if you ask most people about this choice, they'd answer like you predict.

I think if you gave people a choice of buttons to push, one of which is this self-sacrifice button and the other being… uh… whatever the alternative is, there'd be some more hesitance but maybe not a ton.

But I suspect most people do not in fact understand what it means to die, or what they're agreeing to when they agree to sacrifice themselves for something.

I think this is precisely what changes when someone gets a terminal diagnosis. They come to understand in a lived way what it means that they, right there, inside that body and seeing from behind those eyes, are going to experience death. Their mortality isn't an abstraction for them anymore. They stop thinking "I'm going to die someday" like they're talking about a video came character and instead get that it means something way way deeper.

If you adjust the situation so that the person believes the argument that they need to die for the sake of their civilization, and then you hand them a gun with which to shoot themselves…

…I think you'd find the capacity to be "honorable" here dropping dramatically.

But not people saying they would shoot themselves in this hypothetical scenario! Because for the person thinking the thought experiment, the thought experiment doesn't embed the thought-experimenter.

I don't think this bears on the discussion of immortality vs. AI risk. The actions here are abstract enough to be more like the button-pushing case.

I just keep seeing this question of embedded agency getting skipped over and I think it's deeply important.

…a world where individual actions don't matter that much should be a predictable world. And ours very much isn't.

Can you say more? My first reaction is "Huh, I didn't think of that, that's interesting." My second thought is "Wait, what about turbulence the weather?"

RE the latter: the action of individual air molecules doesn't really matter that much, but the net effect is still very hard to predict with much precision. We can say some things about the overall general net effect, but we miss a lot of important details.

(I'm thinking of how my family and I had a plane flight scheduled to Florida last year, and surprise! there was a hurricane that might or might not be about to hit Orlando at the time. The predictions weren't clear even a day before the event. It wasn't clear whether we'd be able to go at all, or when.)

Am I missing your meaning?

I also agree. I was going to write a similar answer. I'll just add my nuance as a comment to Zach's answer.

I said a bunch about ontologies in my post on fake frameworks. There I give examples and I define reductionism in terms of comparing ontologies. The upshot is what I read Zach emphasizing here: an ontology is a collection of things you consider "real" together with some rules for how to combine them into a coherent thingie (a map, though it often won't feel on the inside like a map).

Maybe the purest example type is an axiomatic system. The undefined terms are ontological primitives, and the axioms are the rules for combining them. We usually combine an axiomatic system with a model to create a sense of being in a space. The classic example of this sort being Euclidean geometry.

But in practice most folk use much more fuzzy and informal ontologies, and often switch between seemingly incompatible ones as needed. Your paycheck, the government, cancer, and a sandwich are all "real" in lots of folks' worldview, but they don't always clearly relate the kinds of "real" because how they relate doesn't usually matter.

I think ontologies are closely related to frames. I wonder if frames are just a special kind of ontology, or maybe the term we give for a particular use of ontologies. Mentioning this in case frames feel more intuitive than ontologies do.

I really don’t know what you mean by any of this (especially the “anymore” part, but really all of it).

…

(I don’t think that it’s necessary to “orient to my tone”? In any case, generally speaking, if you assume that I mean just what I say, you won’t go far wrong.)

This is actually really clarifying. Thank you.

I now suspect there's a dimension of communication that's hyper-salient for me but invisible to you.

I won't try to convey that maybe invisible-to-you dimension here. I don't think that'd be helpful.

Instead I'll try to assume you have no idea what you're "saying" on that frequency. Basically that you probably don't mean things they way they implicitly land for me, and that you almost certainly don't consciously hold the tone I read in what you're saying.

That's as close as I can get to assuming that you "mean just what [you] say". Hopefully that'll smooth things out between us!

(ETA: I certainly don’t think that your question was absurd. If I did, I’d’ve said so, and not spent effort answering it!)

Okay, cool. Thanks for saying this!

> In case it’s not clear, the reason I asked you about the “falling in love” thing was to better understand what kind of thing “cake” as you mean it might even look like.

Indeed, that is also why I asked my counter-question; both to explain, and to understand.

I have to admit, I find this very confusing. I'm trying to understand what you mean by "cake". Maybe you were hoping to go "Here's 'cake' for falling in love. Now you try on this other topic, so I can thumbs-up or thumbs-down that you've understood what I mean by 'cake'." Is that it?

The thing is, I think I could provide a similar analysis, but I don't think it'd help me understand at all what you mean by "cake". That makes me pretty hesitant to spend the time and cognitive effort on producing that kind of matching analysis.

Okay! Great, thank you.

This confirms I'm very thoroughly confused about what "cake" means to you here!

I thought you were looking for tangible proof of benefits, or something you could concretely try, or something like that. But now I know I have no idea what you're looking for!

I'll give examples to highlight my confusion. In your "cake" for falling in love, you say:

I could say that falling in love is worthwhile for its own sake. Of course, there isn’t any way I could convince you of that, but that’s not unusual; the same applies to the experience of eating ice cream, etc. This boils down to “try it; you’ll like it!”.

I seem to recall saying very similar things about kenshō. That there's something of deep importance, that this "insight" amounts to acknowledging it, that this is something you'd be super grateful for if it were to happen for you, and that there's not really much of a way for me to convince you of any of this. It's just a "Take a look and see for yourself" kind of thing.

That doesn't seem to have satisfied you. You still asked for "cake".

In particular, what you say here sounds to me like what I'd guess “assurances of having cake” would be.

In your second paragraph of "cake" you say:

falling in love is just the first part of a process which (summed across all instances over a person’s lifetime) is likely to account for a significant chunk of the happiness, life satisfaction, joy, pleasure, etc., that one experiences in one’s lifetime.

This lands for me as what I'd guess “allusions to kinds of cake” would be.

I could say something very similar about kenshō. I suspect I did in that monster thread five years ago. That if & when this flash of clarity comes online, there'll be a sense of something like "Oh holy fuck, I've been just living on autopilot! I haven't been alive here! I've been ignoring what actually goddamned matters just to tell myself some stories and live in fantasy! Whoa!!!" And it's very much just a beginning.

(There's a quote that goes something like "You have two lives. Your second life begins when you realize you have only one life." Kenshō is about beginning your second life.)

But again, this doesn't seem to have satisfied your need for "cake".

Your third paragraph includes:

those of my friends who’ve fallen in love, and entered into (and stayed in) long-term term relationships on the basis of that love, seem to be much happier than they were previously, and in particular, seem to make one another happy, in ways observable in ordinary, everyday interactions. (Of course one can have experiences and observe things that point to the opposite conclusion—but, again, this is too well-trod a topic to productively re-tread here.)

So, on this regarding kenshō I've been maybe too vague. Very attuned to the ability of folk to point at evidence of the opposite conclusion.

But if I can make a similar caveat as you've made here, I think I can point pretty clearly at this.

The people I know who are on the other side of this are alive. Engaged. Interesting. They're themselves much more deeply. More interested in really playing the game of life. Less willing to tolerate bullshit, especially in their for-fun social interactions.

They also almost all have war stories involving the collapse after awakening. A lot of lies people live can't work once they admit to themselves that they're lies. And it's hard not to admit stuff like that in the midst of an explosion of Light, at least in my experience. Breakups, financial collapse, and physical illness are not uncommon. It's usually temporary and most of them say that they totally wouldn't have it any other way — at least once they're through the other side. Some do get stuck. And there's a potential survivorship bias here in my account.

It'd be weird to call that a benefit. It's more like an attribute I notice over and over again. I totally had that. Arguably I still do: it feels like a deep existential allergy to all lies and bullshit turned on deep in my core, and now I'm on a lifelong journey for total and ever-perfecting alignment with… well, truth.

But as things go in terms of "What can we see this actually doing in the world?", here are a few attributes. The Dark Night stuff can be awful to go through, but it's like the vomiting part of food poisoning. It's not like the point of that is to be enjoyable, but you still want to have done it.

Now, if my saying all that still doesn't count for you as "cake"… then I have no idea how to proceed. You're going to have to define what you're looking for differently if you want me to have any chance of answering you on this point.

Can you give some examples of things that those places have held?

Sure. I'll give just one for now. These take a while to name in writing, at least they way they occur to me.

Here's a recurring one: I'll be talking with someone in a coaching session, and I'll pick up on how they're "adding extra".

This is something that's easy to point out in live conversation or over video but I find tricky in writing. It's a tone thing. If I look at the cup next to me and note "This is a cup", I'm simply noting. There's nothing extra. But I can add extra with an emotional tone of "I keep this cup next to me to hydrate myself, because I take care of myself, which is something GOOD I do." I think this is easy to hear with some practice even if the words are identical ("This is a cup").

Usually when someone is adding extra, they have some unrecognized pain. Most of the time this pain roots down in a universal thing — something I've come to call "the pain of duality". It's a basic split from something core. It's roughly the same in everyone best as I can tell, but each person sort of holds and experiences it in their own way.

So when I see someone adding extra, and they've asked for my guidance, I'll sometimes guide them to awareness of this core pain:

• "Here it seems to me that you're saying XYZ [like "I need to finish my thesis"], but you're also adding something extra. It occurs to me as a tone of ABC [usually like there's something wrong with them, or that their value is based on something external, or that something is existentially wrong]. There's nothing wrong with that. I just imagine it's uncomfortable. Do you see what I'm pointing at?"
  • We do some calibration, and I adjust based on their feedback (like if I was missing them in some key way). Then if I still see this core pain in them and they agree I'm seeing them clearly, on to the next step:
• "Okay. Now, for me, in this spot I tend to feel PQR [something like "afraid something will go wrong if I don't take care of this task"]. But that's how the energy feels when it hits my thinking. Underneath that is something wordless. More like a creeping feeling, like reality itself is unsafe or unreliable."
  • The point here is to give an example of what it means to feel the energy behind something in consciousness. If my first example doesn't click for them, I'll give a few others.

Usually they either notice the core pain or adjust my perception of them. More often the former. It tends to result in a direct kind of seeing, the same way you can directly "see" the feeling of your tongue in your mouth: it's somehow more unmediated than thoughts about the thing are.

When I wrote Kenshō I might have called this "Looking at your soul pain". It's about seeing more directly instead of just thinking about mental models of the thing.

I just think that reifying Looking does something odd to this process. It's just noticing what you experience when you look where someone is pointing. Even if that someone is yourself.

Hopefully that's somewhat clear. With more time & effort I might have come up with a simpler example. ("Sorry this letter is so long, I didn't have time to write a shorter one.")

Now, all of that having been said, here’s a counter-question, before we get to kenshō: can you provide an analogous sort of answer, for “having a paranoid delusion” in place of “falling in love”? Does it exist? Does it have any value?

How on Earth is this relevant? I'm really not following you here. What do you hope to gain by having me try to grapple with this weird thing?

Maybe you're trying to… I don't know, get even with me for asking something you find absurd? Trying to defeat me in some kind of dual where you think I issued the first challenge? I really don't know.

In case it's not clear, the reason I asked you about the "falling in love" thing was to better understand what kind of thing "cake" as you mean it might even look like. It really does land as a type error to me. But if you could say "Oh, for falling in love, thus-and-such would be 'cake'", then I could go "Oh! Okay. Cool! So I think the analogy for kenshō might be XYZ. Does that work for you?" Then we could communicate.

The feel — and I could easily be misreading you here — but the feel I get from you here is like intellectual one-upmanship. Mental fencing.

If that's the case, please understand that I'm just not available for that. I will not engage with you at that level anymore.

If I've misread you, then please clarify what you're doing. I don't know how to orient to your tone here. If you meant it collaboratively, then please help me see how. I'd very much like to.

What part of this do you consider to be having "given me cake"?