Core

Arrogance of the Humbled: When someone tries something and fails, they may learn the lesson that the problem is difficult instead of I suck at the problem, leading them to confidently claim furthermore the problem is difficult therefore no one will succeed at it, even incorrectly so, and be even more confident in their authority to make that claim for having failed at the problem.
Paradoxically, one might consider that failing at a problem should make you less confident in your ability to predict its resolution, and attribute yourself less authority on the relevant topics. Thus, arrogance.

Context

Eliezer yudkwosky, in Cat-Belling Problems, writes:

Many many engineers, in fact, historically rolled up their sleeves and got to work on building their designs for Perpetuum Mobiles. It was a larger-looming social phenomenon in Feynman's day -- one of the ways slightly smart engineers went Wrong back before AI or cryptocurrency. And I am not a postcognitive telepath -- I cannot read minds in the past -- but I wouldn't be surprised if many of those 1970s engineers saw themselves as hardheaded practical people with industry experience, whose effortful work learning about real metal gears had taught them the practical limitations of airy abstract theories like "conservation of energy". Which is to say, that they had tried their own hand at abstraction and not gotten anywhere, and learned from this an Arrogance of the Humbled[*] about the ultimate limits of mere thinking.

and

[*] A term and thesis coined by Duncan Sabien. I ought to write it up at some point, but meanwhile perhaps many readers, like myself, will find a whole useful thesis immediately apparent just from seeing the phrase "Arrogance of the Humbled".

Sabien's article can be found here, and it is both easy and fun to read. I recommend it. Since the thesis is clear in the article, I presume that Yud could find value in (1) rewriting it in his own style, (2) calling the attention of his readership to it or (3) expanding the idea, going deeper.

I cannot do (1) or (2), which makes me confident that it's impossible; therefore, all future effort on Arrogance of the Humbled should focus on tackling (3).

Personal-suckiness vs objective-difficulty

There are many ways to fail at solving a problem. Sometimes, you discover the cat-belling step and fail at it; then, you at least gain the insight that a solution to the problem can be productively investigated by considering how it tackles the cat-belling step.

Sometimes, you mostly learn that you suck at the problem - maybe you fall in depression, figure out that your work ethic is terrible, that you need more background knowledge or something. Then, you do not gain insight into how other people may fail at the problem (except insofar as they suck as much as you for the same reasons as you).

The missing insight, in both Sabien's piece and Yud's is: how do you know what direction to update on the personal-suckiness vs objective-difficulty axis?

I don't have a good, principled, fully general answer. Ideally, I'd want something like "keep track separately of what failing teaches about your thinkoomph in the topic, and what it teaches about the difficulty of the problem".
But critically, the two are probably not independent in your self-modeling. How often do you use objective metrics when thinking of how difficult it is to solve a problem cognitively?^[1]

Sabien gives as an example of Arrogance of the Humbled, which I'll investigate because it's fun:

For heavier-than-air flight (as opposed to e.g. balloons, which are kosher), the standard impossibility argument went:

• You're making your airplane out of some material with specific strength.
• To make an airplane big enough to fit a human, we need to scale-up the bird-sized design, with weight growing as but wing area grows as . So speed/power requirements are worse.
  • Therefore, we need to do much better than birds.
• Birds have better specific strength thanks to millions of years of evolution.
  • We can't do as well with steel beams.

Therefore, the airplane won't fly.

Then, the author in the example goes wrong by claiming that we'll need millions of years to get materials as good as bird (to do as well as them). That's just a straight-up bad take that indicates they didn't understand which parts of the impossibility argument were load-bearing (or they were grasping for a punchline, or whatever other reason people have to make bad arguments).

But tell me, knowing what the author knew, what flaw could you have found in the impossibility argument?

*

*

*

*

Well, as far as I can tell, the argument worked pretty well, I don't see any single core flaw that invalidates it. Airplanes did in fact address the problem as a whole, at every step of the impossibility argument:

• Use bracing and other arrangements, thus strength-per-weight is not limited by the material's specific strength.
• Don't scale up a bird nor fly the same way as the birds, so you can use a technique much more effective for your size (no flapping, leverage our combustion engines that have much more throughput than digestion and muscles).
  • And in fact, go much faster than birds.
• We don't design things the same way as evolution. For starters, we can understand bird's flight adaptation (e.g. hollow bones), but furthermore we can invent better things that evolution wouldn't come up with at all.
  • The specific strength of what material we make planes out of remains a central consideration of airplane design today, it's important to develop new lightweight but strong materials.

(and more)

Presumably, the author could have noticed that the impossibility argument was not an unaddressed cat-belling step, if they had interrogated Langley on how he was trying to better than birds. But that requires deep familiarity with the topic (here, Langley's prototype, not the impossibility argument).

So (1) If you are unfamiliar with the topic, don't assume you know a cat-belling step others are ignorant of.

If you have very confident priors in your superior thinkoomph, then I can do nothing for you, sorry. Yud has tried addressing epistemic modesty before, but not to my satisfaction; hopefully if he does write an article on Arrogance of the Humbled, he'll rephrase his position as someone with confidence in his superior thinkoomph.

Other sufficient or necessary conditions for updating on the personal-suckiness vs objective-difficulty axis are left as an exercise for the reader.

Having spent a couple hours thinking about this topic and finding no cat-belling step, no difficulty besides my own willingness to keep writing this article, I conclude that conditional on someone writing a proper article on Arrogance of the Humbled aiming to solve the issue of deconfusing personal-suckiness vs objective-difficulty, they'll probably find a satisfying solution. Good luck!

1. ^{^}

   Example of some metrics to help you deconfuse how much the problem is objectively difficult vs how much you personally suck at the problem:
   - "How much on a scale of 1-10 would an average person say that they struggle to solve this problem?" (although your perception of the "average person" is probably correlated with your own thinkoomph by selection effects)
   - "How long would domain experts take before giving up, if I asked them to solve this problem at dinner?"
   - "What fraction of all papers, weighted by author strength, on this topic have I read?"