If, hypothetically, you discovered some alleged epistemic rationality technique while doing paperwork, I would certainly want you to either explain how you applied this technique originally (with a worked example involving your paperwork), or explain how the reader might (or how you did) apply the technique to some other domain (with a worked example involving something else, not paperwork), or (even better!) both.

This seems sensible, yes.

It would be very silly to just talk about the alleged technique, with no demonstration of its purported utility.

I agree that it seems silly to not demonstrate the utility of a technique when trying to discuss it! I try to give examples to support my reasoning when possible. What I attempted to do with that one passage that you seemed to have taken offense to was show that I could guess at one causal cognitive chain that would have led Valentine to feel the way they did and therefore act and communicate the way they did, not that I endorse the way Kensho was written -- because I did not get anything out of the original post.

There’s a lot of “<whatever> seems like it could be true” in your comment.

Here's a low investment attempt to point at the cause of what seems to you a verbal tic:

I can tell you that when I put “it seems to me” at the front of so many of my sentences, it’s not false humility, or insecurity, or a verbal tic. (It’s a deliberate reflection on the distance between what exists in reality, and the constellations I’ve sketched on my map.)

-- Logan Strohl, Naturalism

If you need me to write up a concrete elaboration to help you get a better idea about this, please tell me.

Are you really basing your views on this subject on nothing more than abstract intuition?

My intuitions on my claim related to rationality skill seem to be informed by concrete personal experience, which I haven't yet described in length, mainly because I expected that using a simple plausible made-up example would serve as well. I apologize for not adding a "(based on experience)" in that original quote, although I guess I assumed that was deducible.

That page seems to be talking about a four-year-old child, who has not yet learned about space, how gravity works, etc. It’s not clear to me that there’s anything to conclude from this about what sorts of epistemic rationality techniques might be useful to adults.

I'm specifically pointing at examples of deconfusion here, which I consider the main (and probably the only?) strand of epistemic rationality techniques. I concede that I haven't provided you useful information about how to do it -- but that isn't something I'd like to get into right now, when I am still wrapping my mind around deconfusion.

More importantly, it’s not clear to me how any of your examples are supposed to be examples of “epistemic confusion [that] can be traced to almost unrelated upstream misconceptions”. Could you perhaps make the connection more explicitly?

For the gravity example, the 'upstream misconception' is that the kid did not realize that 'up and down' is relative to the direction in which Earth's gravity acts on the body, and therefore the kid tries to fit the square peg of "Okay, I see that humans have heads that point up and legs that point down" into the round hole of "Below the equator, humans are pulled upward, and humans heads are up, so humans' heads point to the ground".

For the AI example, the 'upstream misconception' can be[1] conflating the notion of intelligence with 'human's behavior and tendencies that I recognize as intelligence' (and this in turn can be due to other misconceptions, such as not understanding how alien the selection process that underlies evolution is; not understanding how intelligence is not the same as saying impressive things in a social party but the ability to squeeze the probability distribution of future outcomes into a smaller space; et cetera), and then making a reasoning error that seems like anthromorphizing an AI, and concluding that the more intelligent a system would be, the more it would care about the 'right things' that us humans seem to care about.

The second example is a bit expensive to elaborate on, so I will not do so right now. I apologize.

Anyway, I intended to write this stuff up when I felt like I understood deconfusion enough that I could explain it to other people.

Similarly, it seems plausible to me that while attempting to fix one issue (similar to attempting to fix a confusion of the sort just listed), one could find themselves making almost unrelated upstream epistemic discoveries that might just be significantly more valuable).

And… do you have any examples of this?

I find this plausible based on my experience with deconfusion and my current state of understanding of the skill. I do not believe I understand deconfusion well enough to communicate it to people who have an inferential distance as huge as the one between you and I, so I do not intend to try.

[1]: There are a myriad of ways you can be confused, and only one way you can be deconfused.

I apologize for not providing a good enough example -- yes, it was made up. Here's a more accurate explanation of what causes me to believe that Valentine's sentiment has merit:

• It seems to me that a lot of epistemic confusion can be traced to almost unrelated upstream misconceptions. Examples: thinking that people must be suspended upside down below the equator, once someone understands the notion of an approximately spherical Earth; the illusion that mirrors create horizontal asymmetry but retain vertical symmetry; the notion that an AGI will automatically be moral. Similarly, it seems plausible to me that while attempting to fix one issue (similar to attempting to fix a confusion of the sort just listed), one could find themselves making almost unrelated upstream epistemic discoveries that might just be significantly more valuable). I do acknowledge that these epistemic discoveries do also seem object-level and communicable, and I do think that the sentiment that Valentine showed could make sense.
• It also seems that a lot of rationality skill involves starting out with a bug one notices ("hey, I seem to be really bad at going to the gym"), and then making multiple attempts to fix the problem (ideally focusing on making an intervention as close to the 'root' of the issue as possible), and then discovering epistemic rationality techniques that may be applicable in many places. I agree that it seems like really bad strategy to then not try to explain why the technique is useful by giving another example where the technique is useful and results in good object-level outcomes, instead of simply talking about (given my original example) paperwork for a sentence and then spending paragraphs talking about some rationality technique in the abstract.

I notice that I find Valentine's posts somewhat insightful, and believe they point at incredibly neglected research directions, but I notice a huge the distance seems to exist between what Valentine intends to communicate and what most readers seem to get.

Off the top of my head:

• Here's the Exit is written in a style that belies an astounding confidence that what Valentine says is applicable to the reader, no matter what. After a few commenters correctly critique the post, Valentine backs down and claims that it was meant to be an "invitation" for people who recognize themselves in the post to explore the thoughts that Valentine espouses in the post, and not a set of claims to evaluate. This feels slightly like a bait-and-switch, and worse, I feel like Valentine was acting in complete good faith while doing so, with a sort of very out-of-distribution model of communication they endorse for themselves.
• In We're already in AI takeoff, Valentine seems to claim that humans should not try to intervene at the egregore-level, because we are too weak to do so. When someone points out that this may not necessarily be correct, Valentine clarifies that what they meant was more that humans should not use shoulds to try to get themselves to do something that one could be confident that they physically cannot accomplish, and that solving AI alignment or existential risk can be one example of such a thing for many people. Again, I notice how Valentine makes a ton of sense, and points [badly] at very valuable rationalist insights and concepts when requested to clarify, but the way they pointed at it in their OP was, in my opinion, disastrously bad.
• The only thing I recall from Kensho is the Said baking metaphor for people not explaining why one should care about the meta-level thing by showing an object-level achievement done using the meta-level thing. And yet, I get the sentiment that Valentine seems to have been trying to communicate -- it sure seems like there are epistemic rationality techniques that seem incredibly valuable and neglected, and one could discover them in the course of doing something about as useless as paperwork, and talking about how you became more efficient at paperwork would seem like a waste of time to everyone involved.

The reason I wrote this down is because I think Valentine (and other people reading this) might find this helpful, and I didn't feel like it made sense to post this as a comment in any specific individual post.

I recently had to solve a Captcha to submit a reddit post using a new reddit account I made (because I did not use reddit until now). It was an extremely Kafkaesque experience: I tried the Captcha in good faith and Google repeatedly told me I did my job incorrectly, but did not explain why. This went on for multiple minutes, and I kept being told I was doing it wrong, even though I kept clicking on all the right boxes that contained parts of a bicycle or a motorcycle or whatever. The slow fade-in and fade-out images were the worst, and I consider this a form of low level torture when you are made to do this for extended periods of time.

I admit that I use an extremely unique browser setup: portrait mode, OpenBSD amd64 OS, Mozilla Firefox with uBlock Origin, an external keyboard where I use my arrow keys to control the mouse most of the time. I expect that such an out-of-distribution setup may have led the Captcha AI to be suspicious of me. All this was intended to improve my experience of using my machine and interfacing with the Internet. Worse, I was already signed into my Google account, so it didn't make sense that Google would still be suspicious of me being a bot.

I've decided on a systemic solution for this problem:

1. I shall never manually complete a Captcha again.
2. I shall use automated captcha-solvers, or failing that, pay other people to solve captchas for me, if I really must bypass a captcha.

One could interpret this as adversarial action against Google and Reddit, but it seems to me that when dealing with an optimizer that is taking constant adversarial action against you, and is credibly unwilling to attempt to co-operate and solve the problem you both face, the next step is to defect. Ideally you extricate yourself from the situation, but in some cases that isn't acceptable given your goals.

I expect that people who are paid to solve captchas probably are numb to this, or have been trained by the system to solve captchas more efficiently, such that they may be optimized for dealing with its Kafkaesque nature. I do not expect to feel like I would be putting them through the pain I would have experienced. I still do not consider it an ideal state of affairs, though.

This feels like roon-tier Twitter shitposting to me, Jacques. Are you sure you want to endorse more of such content on LessWrong?

I notice that Joe Carlsmith dropped a 127 page paper on the question of deceptive alignment. I am confused; who is the intended audience of this paper?

AFAICT nobody would actually read all 127 pages of the report, and most potential reasons for writing the report to me seem better served by faster feedback loops and significantly smaller research artifacts.

What am I missing?

I'm confused -- did you consider simply paying someone to build you such an extension? It seems like you could easily scope it down to whatever features you want, exactly how you want it, and ensure it is free of bugs.

Such investment makes sense if you've been wanting such a tool for "several years", as you put it.

You should probably ask saturn2 for permission before doing so.

https://www.lesswrong.com/posts/66DXhQJyPEJNsXgfw/an-alternative-way-to-browse-greaterwrong-2-0

@clone of saturn, aka "saturn2", runs GW