I was young, and a mere Traditional Rationalist who knew not the teachings of Tversky and Kahneman.

He means Judgment under Uncertainty: Heuristics and Biases, almost certainly. I think at one point there was a reading group surrounding it, but I don't know what ever happened to them.

Today, one of the chief pieces of advice I give to aspiring young rationalists is "Do not attempt long chains of reasoning or complicated plans."

Advice more or less completely ignored by everyone, including EY himself.

To a Bayesian, on the other hand, if a hypothesis does not today have a favorable likelihood ratio over "I don't know", it raises the question of why you today believe anything more complicated than "I don't know". But I knew not the Way of Bayes, so I was not thinking about likelihood ratios or focusing probability density.

I want to point out that thinking about likelihood ratios or focusing probability density is independent of any knowledge of Bayes' theorem. I'd be surprised if any calculation actually occurred.

When I think about how my younger self very carefully followed the rules of Traditional Rationality in the course of getting the answer wrong, it sheds light on the question of why people who call themselves "rationalists" do not rule the world. You need one whole hell of a lot of rationality before it does anything but lead you into new and interesting mistakes.

I don't understand this argument. What does calling yourself rationalist have to do with not ruling the world? Traditional rationality has all sorts of specialized counter-memes against political activity. Hell, even LW has counter-memes against entering politics. Isn't it plausible that traditional rationalists eschew politics for these reasons, rather than EY's thesis, which always seems to be along the lines of "Traditional Rationality is good for nothing, or perhaps less than nothing."

Traditional Rationality is taught as an art, rather than a science; you read the biography of famous physicists describing the lessons life taught them, and you try to do what they tell you to do. But you haven't lived their lives, and half of what they're trying to describe is an instinct that has been trained into them.

The crushing irony of reading an autobiographical anecdote of EY's in an essay that implicitly says "learn this intuition I stumbled across" is almost too great to bear. Certainly LW-style rationality is no more a science than traditional rationality is.

The way Traditional Rationality is designed, it would have been acceptable for me to spend 30 years on my silly idea, so long as I succeeded in falsifying it eventually, and was honest with myself about what my theory predicted, and accepted the disproof when it arrived, et cetera.

I don't think there's enough data presented here to actually support this claim, depending upon what EY means by "acceptable." Would the community support him financially if he spent 30 years trying to demonstrate quantum consciousness, without producing something intrinsically valuable along the way? I don't think so; even Penrose had to produce sound mathematics to back up his crackpot-ness.

Traditional Rationalists can agree to disagree.

The LW advance in this area seems to consist entirely of agreeing to disagree after quoting Aumann's theorem and continuing the argument well past the point of diminishing returns. In that respect, Traditional Rationalists win, merely because they don't have to put up with as much bullshit from e.g., religious nutjobs playing at rationality.

Maybe that will be enough to cross the stratospherically high threshold required for a discipline that lets you actually get it right, instead of just constraining you into interesting new mistakes.

If modern LW is any indication, then it probably wasn't enough. Everyone talks about Bayes, but few people do any actual math. EY wrote a whole sequence on quantum physics, writing the Schrodinger operator exactly once. If math is what will save us from making interesting new mistakes, we clearly aren't doing enough of it.

The way Traditional Rationality is designed, it would have been acceptable for me to spend 30 years on my silly idea, so long as I succeeded in falsifying it eventually, and was honest with myself about what my theory predicted, and accepted the disproof when it arrived, et cetera. This is enough to let the Ratchet of Science click forward, but it's a little harsh on the people who waste 30 years of their lives.

I think this is a case where Eliezer's nontraditional career path caused him to miss out on some of the traditional guidance that young researchers get. If a graduate student tells their adviser that they want to work on some far-fetched research project like quantum neurology, the adviser will have some questions for their student like "What's the first step that you can take to conduct research on this?", "How likely is this to pan out?", "What publications can you get out of this?", and "Will this help you get a job?" Most young researchers have a mentor who is trying to help them get started on a successful career, and the mentor will steer them away from unproductive projects which don't leave them with good answers to these questions.

This careerism has its downsides, but it set a higher standard than mere falsifiability, which helps keep young researchers from wasting their careers pursuing some silly idea. You have to get tenure before you can do that. (The exception is when the whole field has already embraced the silly idea enough to publish articles about it in top journals and allow researchers to make a career out of it.)