I like this. I largely agree.

I'd like to pinpoint a few differences I notice. I hope the collective here takes this as me coming from a spirit of "Here's the delta I see" rather than "I disagree and here's why." By and large I really like the clarity Kaj has brought to this.

First, a meta thing:

While I liked Valentine’s recent post on kensho and its follow-ups a lot, one thing that I was annoyed by were the comments that the whole thing can’t be explained from a reductionist, third-person perspective.

I didn't mean to convey that it can't be explained this way. I now think I was combining a few different things in a way that accidentally made it hard to understand:

One key thing I now see is that Looking doesn't require self-reference — but most of the interesting applications of Looking that I'm aware of do require navigating self-reference. An example of this is the "get out of the car" problem. (I'll have more to say about Kaj's interpretation of that in a bit.)
The main thrust of what I'm poking at is a collection of results of Looking at ontology (whereas here Kaj focuses mostly on Looking at suffering). If we drag in an ontology to say "Okay, here's what Looking is, and now we've nailed it down", and then you use that definition of Looking instead of the phenomenological skill, then it's going to be enormously hard to Look at the ontology used to define Looking. And in this particular case, people seem to be prone to not noticing when they've made this error. (Again with the car/phone analogies.) I'm particularly concerned here because the culture around LW-style rationality seems to emphasize a very specific and almost mathematically precise ontology in a way that is often super useful but that I don't think is a necessary consequence of the epistemic orientation. That made me really hesitant to put a bunch of effort into spelling out what Looking might be within that favored ontology, since the whole point is to notice restrictions on epistemic strength imposed by ontological rigidity. I was (and am) concerned about early attempts to explain this stuff locking out a collective ability to understand.
With that said, my personal impression had been that it's actually quite easy to see what Looking is and how one might translate it into reductionist third-person perspectives. But my personal experience had been that whenever I tried to share that translation, I'd bounce off of weird walls of misunderstanding. After a while I noticed that the nature of the bounce had a structure to it, and that that structure has self-reference. (Once again, analogies along the lines of "get out of the car" and "look up from your phone" come to mind.) After watching this over several months and running some informal tests, and comparing it to things CFAR has been doing (successfully, strugglingly, or failing to do) over the six years I've been there, it became obvious to me that there are some mental structures people run by default that actively block the process of Looking. And for many people, those structures have a pretty strong hold on what they say and consciously think. I've learned to expect that explaining Looking to those structures simply will never work. (There are other structures in human mind design, though. And I claim there's a pretty reliable back door to such self-referential architectures. But explaining that explicitly seems to run into the same communication problem… which is why I'm writing this meta-ontology sequence.)

So… I'm happy to go with the main thrust of what I receive Kaj as expressing here. I just also want to add an asterix saying something like "Beware, we've now entered a realm where the illusion of safety has become stronger, and I fear this will make what's coming that much more painful by comparison and thus harder to understand."

I believe that this kind of thing is what Valentine means when he talks about Looking: being able to develop the necessary mental sharpness to notice slightly lower-level processing stages in your cognitive processes, and study the raw concepts which then get turned into higher-level cognitive content, rather than only seeing the high-level cognitive content.

Yep.

…with a caveat that I'm pretty sure Kaj gets and I think even said but that I don't know if the casual reader will reliably catch:

On the inside, before you Look, the thing you're about to Look at doesn't look on the inside like "high-level cognitive content". It looks like how things are. This ends up with me saying things that sound kind of crazy or nonsensical, but to me are obvious once I Look at them. (E.g., there are no objects. We create objects in order to think. Because language is suffused with object-ness, though, I don't know of any coherent way of talking about this.)

Understanding suffering is a special case of Looking, but a sufficiently important one that it deserves to be briefly discussed in some detail.

I want to highlight this. I quite agree, suffering is a really important special case, and I'm delighted with what Kaj did with it. And also, it's a special case. Nearly all discussion of enlightenment-flavored stuff I've encountered has been about the alleviation of suffering or the cultivation of happiness, and I think these are great and important things to emphasize… and I think there's something else in this domain that's more central to the rationality project here. (Although I do think that the path of alleviating suffering via e.g. Looking at the nature of the self does result in a bunch of the right kind of epistemic updates. I just suspect it's insufficient.)

So what’s all this “look up” and “get out of the car” stuff? […] You can’t defuse from the content of a belief, if your motivation for wanting to defuse from it is the belief itself.

This comes across to me as a great explanation of a special case. Kaj might mean the general thing, but I'm not sure.

I'm going to claim there are two kinds of problems that this "get out of the car" thing is pointing at:

Structural self-reference in defusion. In other words, if a belief you're fused with is being used somehow in the effort to defuse from it, then the defusion is likely to fail. Kaj gives one type of example of this. Another one is if the belief provides the framework by which you're orienting to the possibility of Looking at the belief in the first place. I have a post planned about two or three out that will go into more detail about this, but to (maybe dangerously?) gesture at the thing: Starting from reductionist materialism to frame the question of what our brains are doing when we Look at reductionist materialism yields a strange loop that often causes the process to glitch (and can reinforce an impression of real-ness that's strange the way believing in objectively existing objects is strange).
Orthogonality. If I try to argue with a paperclip maximizer about how maximizing paperclips isn't all there is to life, it will care to listen only to the extent that listening will help it maximize paperclips. I claim that by default, human mind design generates something analogous to a bunch of paperclip maximizers. If I'm stuck talking to one of someone's paperclip maximizers, then even if I see that there are other parts of their mind that would like to engage with what I'm saying, I'm stuck talking to a chunk of their mind that will never understand what I'm saying. (I'll have more to say about this in my next post (or the one after it if I need to split them again).)

The second case is pretty straightforward to bypass by Looking. The first is much trickier, but I think might be doable if you track a kind of phenomenological feedback loop that the self-reference generates and use that as a warning sign. (Unfortunately, I think there are structural reasons why the warning sign can't say anything much more specific than "Do something different." In short, the part of the mind that's trying to work out what to do is almost always using the belief in question, so no amount of instruction is going to help it get a meaningfully useful answer.)

All of this particularly applies for trying to overcome suffering. Because remember, suffering is caused by a belief that pain is intrinsically bad. That belief is what causes you to try to flinch away from pain in a way which, by itself, creates the suffering.

I like this. I hadn't quite thought of it this explicitly.

It also suggests some hope in approaching the domain from the angle of a desire for good epistemics instead, which is roughly where I've been coming from. I haven't yet noticed any self-referential glitch, instead finding things like the Litany of Tarski.

…but knowing the rhythm of this domain, I suspect this is just a description of my current ignorance.

But if you cared about things like saving the world, then you will still continue to work on saving the world, and you will be Looking at things which will help you save the world - including ones that increase your rationality.

This.

I've come to learn that the communities that talk most about enlightenment-related things are very particular about the word "enlightenment" and seemed to bristle at how I used it. So I'll add an adjustment to language (but not to what I have been meaning to convey) and clarify that I don't mean to imply that I am fully enlightened. I still suffer, I still usually operate under the delusion that I have a self (though I've seen through that one twice so far), and I haven't Looked carefully at impermanence or unsatisfactoriness as yet.

And with that said, I strongly resonate with this sentiment.

I'm writing what I'm writing, and I continue to teach at CFAR, and do all the things I'm doing, because I care to do some world-saving things.

And I see some things, via Looking, that I think are very important to share.

(It just takes a while to build a scaffold that might work for sharing it. And much appreciation for people like Kaj who build better scaffolds than I've managed so far!)