Mandatory Secret Identities

Previously in seriesWhining-Based Communities

"But there is a reason why many of my students have achieved great things; and by that I do not mean high rank in the Bayesian Conspiracy.  I expected much of them, and they came to expect much of themselves." —Jeffreyssai

Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of...

...becoming a teacher and having their own martial arts dojo someday.

To see what's wrong with this, imagine going to a class on literary criticism, falling in love with it, and dreaming of someday becoming a famous literary critic just like your professor, but never actually writing anything.  Writers tend to look down on literary critics' understanding of the art form itself, for just this reason.  (Orson Scott Card uses the analogy of a wine critic who listens to a wine-taster saying "This wine has a great bouquet", and goes off to tell their students "You've got to make sure your wine has a great bouquet".  When the student asks, "How?  Does it have anything to do with grapes?" the critic replies disdainfully, "That's for grape-growers!  I teach wine.")

Similarly, I propose, no student of rationality should study with the purpose of becoming a rationality instructor in turn.  You do that on Sundays, or full-time after you retire.

And to place a go stone blocking this failure mode, I propose a requirement that all rationality instructors must have secret identities.  They must have a life outside the Bayesian Conspiracy, which would be worthy of respect even if they were not rationality instructors.  And to enforce this, I suggest the rule:

  Rationality_Respect1(Instructor) = min(Rationality_Respect0(Instructor), Non_Rationality_Respect0(Instructor))

That is, you can't respect someone as a rationality instructor, more than you would respect them if they were not rationality instructors.

Some notes:

• This doesn't set Rationality_Respect1 equal to Non_Rationality_Respect0.  It establishes an upper bound.  This doesn't mean you can find random awesome people and expect them to be able to teach you.  Explicit, abstract, cross-domain understanding of rationality and the ability to teach it to others is, unfortunately, an additional discipline on top of domain-specific life success.  Newton was a Christian etcetera.  I'd rather hear what Laplace had to say about rationality—Laplace wasn't as famous as Newton, but Laplace was a great mathematician, physicist, and astronomer in his own right, and he was the one said "I have no need of that hypothesis" (when Napoleon asked why Laplace's works on celestial mechanics did not mention God).  So I would respect Laplace as a rationality instructor well above Newton, by the min() function given above.

• We should be generous about what counts as a secret identity outside the Bayesian Conspiracy.  If it's something that outsiders do in fact see as impressive, then it's "outside" regardless of how much Bayesian content is in the job.  An experimental psychologist who writes good papers on heuristics and biases, a successful trader who uses Bayesian algorithms, a well-selling author of a general-audiences popular book on atheism—all of these have worthy secret identities.  None of this contradicts the spirit of being good at something besides rationality—no, not even the last, because writing books that sell is a further difficult skill!  At the same time, you don't want to be too lax and start respecting the instructor's ability to put up probability-theory equations on the blackboard—it has to be visibly outside the walls of the dojo and nothing that could be systematized within the Conspiracy as a token requirement.

• Apart from this, I shall not try to specify what exactly is worthy of respect.  A creative mind may have good reason to depart from any criterion I care to describe.  I'll just stick with the idea that "Nice rationality instructor" should be bounded above by "Nice secret identity".

But if the Bayesian Conspiracy is ever to populate itself with instructors, this criterion should not be too strict.  A simple test to see whether you live inside an elite bubble is to ask yourself whether the percentage of PhD-bearers in your apparent world exceeds the 0.25% rate at which they are found in the general population.  Being a math professor at a small university who has published a few original proofs, or a successful day trader who retired after five years to become an organic farmer, or a serial entrepreneur who lived through three failed startups before going back to a more ordinary job as a senior programmer—that's nothing to sneeze at.  The vast majority of people go through their whole lives without being that interesting.  Any of these three would have some tales to tell of real-world use, on Sundays at the small rationality dojo where they were instructors.  What I'm trying to say here is: don't demand that everyone be Robin Hanson in their secret identity, that is setting the bar too high.  Selective reporting makes it seem that fantastically high-achieving people have a far higher relative frequency than their real occurrence.  So if you ask for your rationality instructor to be as interesting as the sort of people you read about in the newspapers—and a master rationalist on top of that—and a good teacher on top of that—then you're going to have to join one of three famous dojos in New York, or something.  But you don't want to be too lax and start respecting things that others wouldn't respect if they weren't specially looking for reasons to praise the instructor.  "Having a good secret identity" should require way more effort than anything that could become a token requirement.

Now I put to you:  If the instructors all have real-world anecdotes to tell of using their knowledge, and all of the students know that the desirable career path can't just be to become a rationality instructor, doesn't that sound healthier?


Part of the sequence The Craft and the Community

Next post: "Beware of Other-Optimizing"

Previous post: "Whining-Based Communities"

177 comments, sorted by
magical algorithm
Highlighting new comments since Today at 1:01 AM
Select new highlight date
Moderation Guidelines: Reign of Terror - I delete anything I judge to be annoying or counterproductiveexpand_more

What does this post even mean? I don't have access to my own respect function, and I don't know if I'd mess with it this way even if I did.

If you were to say tomorrow "I've been lying about the whole AI programmer thing; I actually live in my parents' basement and have never done anything worthwhile in any non-rationality field in my entire life," then would I have to revise my opinion that you're a very good rationality teacher? Would I have to deny having learned really valuable things from you?

Or would I have to say, "Well, this guy named Eliezer taught me everything I know, he's completely opened my mind to new domains of knowledge, and you should totally read everything he's written - but he's not all that great and I don't have any respect for him and you shouldn't either" when referring people to your writing?

Or to put it another way...let's say there are two rationality instructors in my city. One, John, is a world famous physicist, businessman, and writer. The other, Mary, has no particular accomplishments outside her rationality instruction work. However, Mary's students have been observed to do much better at their careers than John's, and every time the two dojos go up against each other in Rationalist Debating or calibration tests or any other kind of measurement, Mary's students do better. Wouldn't it be, well, irrational for me to go to John's dojo instead of Mary's? Would the Bayesian Police have to surround Mary's dojo and make sure her students don't say nice things about her or pay her more money than John is making?

If you were to say tomorrow "I've been lying about the whole AI programmer thing; I actually live in my parents' basement and have never done anything worthwhile in any non-rationality field in my entire life," then would I have to revise my opinion that you're a very good rationality teacher? Would I have to deny having learned really valuable things from you?

But the fact that reality doesn't disentangle this way, is in a sense the whole point - it's not a coincidence that things are the way they are.

If we get far enough to have external real-world standards like those you're describing, then yes we can toss the "secret identity" thing out the window, so long as we don't have the problem of most good students wanting only to become rationality instructors themselves as opposed to going into other careers (but a teacher who raised their students this way would suffer on the 'accomplished students' metric, etc.). But on the other hand I still suspect that the instructors with secret identities would be revealed to do better.

I've never seen anything from Eliezer that proves that he's done anything at all of value except be a rationality teacher. I know of two general criteria by which to judge someone's output in a field that I am not a part of:

1) Academic prestige (degrees, publications, etc.) and 2) Economic output (making things that people will pay money for).

Eliezer's institution doesn't sell anything, so he's a loss on part 2. He doesn't have a Ph.D or any academic papers I can find, so he's a loss on part 1, as well. Can SIAI demonstrate that it's done anything except beg for money, put up a nice-looking website, organize some symposiums, and write some very good essays?

To be honest, I'd say that his output matches the job description of "philosopher" than "engineer" or "scientist". Not that there's anything wrong with that. Many works that fall broadly under the metric of philosophy have been tremendously influential. For example, Adam Smith was a philosopher.

Eliezer seems to have talents both for seeing through confusion (and its cousin, bullshit) and for being able to explain complicated things in ways that people can understand. In other words, he'd be an amazing university professor. I just haven't seen him prove that he can do anything else.

Yes - in fact, the only thing that leads me to suspect that EY and SIAI are doing anything worth doing is the quality of EY's writings on rationality.

EY has a lengthy article in this volume if that counts as academic.

As has been said, being a theoretician seems distinct enough from teaching that it should count as a day job. I still view Eliezer as more of a teacher than a theoretician, but I don't think Eliezer is saying teachers don't have to be completely divorced from their subject in their day job to avoid affective death spirals.

Right. Our difference of opinion here is clearly nontrivial. I'll put it on the list of things to write posts about.

Are you saying that teachers who don't externally practice the thing they're teaching won't make good teachers? Or that they're not worthy of respect at all? If the former, I agree with Yvain and others that we have better metrics for determining teacher quality. If the latter, I'm not sure why this would be the case. The comparison to literary critics doesn't answer that question; it just accesses our assumed cached thoughts about literary critics. What's the problem with people wanting to be literary critics?

The post proposes a required formula for respect, but it never explains what quantity that formula intends to maximize. What's the goal here?

Is the point about respect for instructors supposed to generalize to instructors of disciplines other than rationality?

If so, what do you make of Nadia Boulanger? Her accomplishments as a musician (or otherwise) are unimpressive relative to those of her students and peers, and yet she is regarded as one of the greatest music teachers ever, and is accorded correspondingly deep respect by music historians, composers, etc. Are they all wrong to respect her so much, or does it not apply to music or this case?

It seems to me that a better formula for determining respect would somehow reflect the respect given to her students which they say is significantly due to her influence as a teacher. For example, if Aaron Copland singles her out as an amazing teacher who profoundly affected his musical life & education, then she deserves some of the respect given to him. And likewise for her many other students who went on to do great things.

There seems to be an implicit underlying belief in this post that teaching is not (or should not be) an end in and of itself, or at least not a worthy one. I think Boulanger and teachers of her caliber show that that's just not the case.

I was thinking about that - a clause for respecting teachers with great students, should they have them. It still gives people the right incentives.

You've got things the wrong way round. It is the quality of the teacher's students that tell us whether we wish to study under her. The teachers own achievements are a proxy which we resort to because we need to decide now, we cannot wait to see the longer term effects on last years students.

Another proxy is the success of the teacher in getting her students through examinations. This is a proxy because we don't really want the certificate, we what the achievement that we think it heralds. We can assess the strength of this proxy checking whether success in the examinations really does herald success in real life.

I agree with the conclusion of the original post but find the argument for it defective. The key omission is that we don't have a tradition of rationality dojo's, so we do not yet have access to records of whose pupils went on to greatness. Nor do we have records that would validate an examination system.

Notice that the problems of timing are inherent. The first pupils, who went on to real world success, prove their teachers skill in an obvious way, but how did they choose their teacher? Presumably they took a risk, relying on a proxy that was available in time for the forced choice they faced.

Yes, precisely. The issue isn't how we can become a better teacher, or find one to study under. The first question, that MUST be asked before all others, is: what does it mean to be a good teacher, and how can we define the relevant differences between teachers?

Once that question has an answer, we can begin searching for ways to make ourselves better match that defined meaning, or in signal traits in others that indicate they're likely to match that definition well.

Concepts like "has students that will accomplish great things" aren't useful for a variety of reasons. And once someone has developed a reputation for being a great teacher, they're likely to attract students with a lot of potential (assuming there are working metrics for potential that are actually consulted, as opposed to rich people simply buying a place for their talentless children). The reputation alone would result in the teacher's students doing better than most.

Evaluating the teacher requires that we have some way of determining, or at least guessing, what a student's performance would have been without the teaching.

Isn't teaching itself a skill? So what that she was a bad musician, she was obviously a first rate teacher (independent of the subject that she taught).

As long as we determine how much of their students' success is attributable to the teacher, it seems reasonable. It seems we could make those sorts of judgments by:

  • comparing the success of the students of a teacher with the success of students of other teachers having equally talented students (e.g., compare Boulanger's students' success with that of students of contemporaneous Fontainebleau teachers); or
  • when successful people have typically studied with many different teachers, asking them how much of their success they attribute to the influence of their various teachers.

I do find cases like this surprising, though. What was it that she was able to teach to her students that she could not put to use herself?

She gives a pattern of feedback that makes the students practice well? In the sense that she gives positive feedback she functions more as a motivator than as a teacher. Her skill is teaching, it's only happenstance that she teaches music; has she taught shoe polishing or finger painting she would have produced the best shoe polishers and the most skilled finger painters.

Perhaps she doesn't have many complex skills but has strong fundamentals (think Tim Duncan of the NBA Spurs). She might make her students practice the fundamentals which will allow them to do more complex work as they get older.

Finally, she might have knowledge more advanced than her skill. She might not have the hand eye coordination or the processing speed to play sophisticated music but she might know how it's done. Imagine a 5 foot tall jewish guy that loves basketball. He's not gonna make the NBA. It's simply not gonna happen. However, he might understand the game better than many NBA players. Likewise he might be the best basketball coach in the world even though his athleticism (and hence his basketball playing skills) is less than that of NBA players. Likewise the teacher might have had a strong theoretical understanding but not have had the ability to put her theoretical knowledge into practice.

The first thing that comes to mind is maybe she's able to teach students how to practice more in their youth than she did.

That'd work at least.

I was thinking of Nadia Boulanger, with Astor Paizzolla as the distinguished pupil. Piazzolla was trying to be a classical composer, but Boulanger said his classical music was lifeless, it was his tangos that had fire.

Perhaps the multi-talented young pupil faces perilous choices about where to focus his energies. Whether the older great teacher can warn effectively against the common errors probably depends on having a breadth of experience, perhaps having put in many years as a mediocre teacher, following up pupils and noticing how things worked out. The teacher's own youthful errors might be uniformative even if severe.

Yeah, definitely surprising, but genius in any form is surprising.

There is an essay here by a former student that gives a sense of how she taught. And Philip Glass describes her as the decisive influence on him in this article, which also talks about her teaching a little.

Here's an interesting passage from Nadia Boulanger: A Life in Music:

It was Nadia's manner rather than her materials that was unique. Her intensity, her emotional involvement with her students, her broad knowledge of music in general, and her ability to project her own passionate enthusiasm for each detail as well as its over-all form, were the qualities that made her extraordinary. Her electric personality brought a distinctiveness to everything that Nadia did. In this, lies what one reviewer called "the difference between good teaching and great teaching," for in the latter "the student feels that the teaching enacts an extraordinarily intimate and demanding relation between the teacher and his subject, a relation such that the teacher's sense of his subject is indistinguishable from his sense of life."


You're wasting huge amounts of optimization power, here, in two different ways. Firstly, you're saying that no one should focus his efforts on becoming a good rationality instructor, that any work he does on that is entirely meaningless unless he is at least as good at something else. Secondly, you're saying that no one should focus his efforts on instructing people in rationality, that they should spend most of their time on whatever other thing it is that makes them impressive. If you have someone who is naturally better at instructing people in rationality than in anything else, you are wasting most of the surplus you could have gained from him in these two ways.

I'm sympathetic to your concern, but surely there must be a way we can avoid throwing out the baby with the bathwater?

Well... go ahead and suggest a way to avoid throwing out the baby with the bathwater? I mean, we're talking about some pretty scary bathwater here.

Personally I suspect that the bathwater only really gets dirty when you are teaching something that is essentially useless in modern society, like martial arts or literary criticism. Most people who study, say, engineering don't do so in the hopes of becoming teachers of engineering.

Now you might say that this is because teachers of engineering are expected to also do research, but firstly that doesn't explain the disparity between fields, and secondly, I don't think that the example of tertiary education is one to aspire to in this way. I seem to recall you are an autodidact, so you may not have the same trained gut reaction I do, but I have seen too many people who did not have the skill of teaching but were good researchers teaching horribly, and I remember one heartbreaking example of an excellent teacher denied tenure because the administrators felt his research was not up to snuff too well, to want to optimize rationality teachers on any basis other than their ability to teach rationality.

Personally I suspect that the bathwater only really gets dirty when you are teaching something that is essentially useless in modern society, like martial arts or literary criticism. Most people who study, say, engineering don't do so in the hopes of becoming teachers of engineering.

Fair point. But this depends on things starting out healthy so that they stay healthy.

Martial arts seem to get an unreasonably bad rep on LW. It's at least as useful as painting or writing fiction, and I consider those to be fine personal development endeavours.

While I think martial arts are pretty useful by hobby standards (although their usefulness is broad enough that they might not be optimal for specialists in several fields), several historical and cultural factors in their practice have combined to create an unusually fertile environment for certain kinds of irrationality.

First, they're hard to verify: what works in point sparring might not work in full-contact sparring, and neither one builds quite the same skillset that's useful for, say, security work, or for street-level self-defense, or for warfare. It's difficult to model most of the final applications, both because they entail an unacceptably high risk of serious injury in training and because they involve psychological factors that don't generally kick in on the mat.

Second, they're all facets of a field that's too broad to master in its entirety in a human lifetime. A serious amateur student can, over several years, develop a good working knowledge of grappling, or of aikido-style body dynamics, or empty-hand striking, or one or two weapons. The same student cannot build all of the above up to an acceptable level of competence: even becoming sort of okay at the entire spectrum of combat is a full-time job. (Many martial arts claim to cover all aspects of fighting, but they're wrong.)

Despite this, though, almost every martial art claims to do the best job of teaching practical fighting for some value of "practical", and every martial art takes a lot of pride in its techniques. As a consequence, there's a lot of posturing going on between nearly incommensurate systems. There have been various attempts at comparing them anyway (MMA is the most popular modern framework): they're better than nothing, but in practice usually come out too context-dependent to be very useful from a research perspective.

On top of that, there's a tradition of secrecy, especially in older systems (koryu, in Japanese martial arts parlance). Until well after WWII, it was uncommon for any system to open its doors to ethnic outsiders, often even to familial outsiders. Until the Eighties it was uncommon for systems to welcome cross-training in their students. Many still require instructors to have trained in only the system they teach. This is intended to prevent memetic cross-contamination but in practice serves to foster the wide range of biases that come with isolation and hierarchy: you can make almost anything work on your own students, as Eliezer's memorable example about ki powers demonstrates. (If you're feeling uncharitable, you could probably make an analogy here to the common cultic practice of isolation.)

Finally, a lot of selection pressure's eased off the martial arts in the modern era. During the Sengoku era, for example, Japanese martial arts were clannish and highly secretive, but it didn't matter too much: two hundred years of warfare made it very clear which taught viable techniques, if only by extinguishing poorer schools. Most other martial cultures were in a position to gain similar feedback, if less intensely. In the 20th century, though, martial arts grew more or less disconnected from martial applications: most militaries still teach simplified systems, but martial arts skill rarely decides engagements, and when it does it's in a narrower range of situations. Same goes for all the civilian jobs where martial arts are useful: there's feedback, but it's narrow, uncommon, and hyperspecialized.

I think there are ways around all of these problems, but no arts that I know of have done a very good job of engaging them systematically (though at least the more modern intersectional martial arts are trying -- JKD comes to mind). This actually wouldn't be a bad exercise in large-scale instrumental optimization, except that it requires a pool of talent that at present doesn't exist in any organized way.

(Disclaimer: as is probably obvious by now, I am a martial artist.)

Thanks for a thoughtful reply!

You could say much the same about painting/dancing/cooking/writing: There are many different sub-arts; it's hard to master all of them; practitioners can become unduly wedded to a single style; there are examples of styles that have "gone bonkers"; there are many factors in place that hurt the rationality of practitioners.

These are all valid concerns, but I don't think they're particularly problematic within martial arts in comparison to other hobbies.

You could say point 2 about those, but points 1 and 3 stand.

If you are half-way decent at painting/dancing/cooking/writing and think you're pretty good, it is unlikely to get your face stove in the first time you try it seriously. This leads to your getting feedback and improving. You can watch serious, nothing-held-back demonstrations as public performances (or to take home and study, in the case of writing) for a nominal fee.

Really? I've always thought the opposite; that there's a common sense on this site that martial arts are a discipline worthy of taking seriously and investing far more attention in than I would have thought they merited with respect to their applications to rationality. I may be very interested in martial arts, but in most of my social outlets I don't have nearly as much of a sense of it being a shared interest.

Painting and writing fiction produce items that can then be enjoyed by many other people who are neither writers nor painters. Martial arts produces almost nothing, aside from an occasional sports event.

[mistake] How about RatRespect1 = min(RatRespect0, sqrt(RatRespect0)^2 + (NonRatRespect0)^2))?

[edit] Confound you, Pythagoras! What I meant to say was...

RatRespect1 = min(RatRespect0, sqrt(RatRespect0 x NonRatRespect0)

There's no sudden ceiling, but you still get wiped for neglecting the real world.

Do you people actually think in terms of equations like this? Once you begin throwing in exponents, I think the metaphorical/illustrative value of expressing things in math drops off quickly.

Do you people actually think in terms of equations like this?

Not very well in my case, it seems, my apologies. Exponents now thrown out again.

That wasn't meant as a criticism of you specifically. I've just noticed that people on this site like to use equations to describe thought processes, some of which might be better communicated using everyday language. I'd argue Eliezer's post is an even worse example -- why not just say "the lesser of the two quantities" or something?

To be fair, for people who are used to thinking in math, pseudo-mathematical notation is as readable as English, with advantages of brevity and precision.

"People used to thinking in math" currently describes a large portion of users on this site. Use of gratuitous mathematical notion is likely to help keep it that way.

"Use of gratuitous mathematical notion is likely to help keep it that way."

Is that desirable? (Not saying you're implying it is.) The community could probably benefit from some smart humanities types.

Is that desirable? (Not saying you're implying it is.) The community could probably benefit from some smart humanities types.

I was actually trying to imply that it isn't desirable, so yes, I agree fully.

First post, so I'll be brief on my opinion. I would say "it depends". To communicate between people and even to clarify one's own thoughts, a formal language, with an appropriate lexicon and symbols, is a key facilitator.

As for desirability of audience, the About page says "Less Wrong is an online community for discussion of rationality", with nothing about exclusivity. I would suggest that if a topic is of the sort that newbies and lay people would read, then English is better; if more for the theorists, then math is fine.

I personally find "min(A,B)" clearer than "the lesser of A and B", but I'm on the autistic spectrum.'s a little awkward reading the math without TeX, but I think assuming all variables real, that simplifies to RatRespect1=RatRespect0

I agree with this comment vociferously.

The upper bound isn't a terrible idea, but it would, for example, knock E.T. Jaynes out of the running as a desirable rationality instructor, as the only unrelated competent activity I can find for him is the Jaynes-Cumming Model of atomic evolution, which I have absolutely zero knowledge of.

knock E.T. Jaynes out of the running

Dude, what on Earth are you talking about. E. T. Jaynes was a Big Damn Polymath. I seem to also recall that in his later years he was well-paid for teaching oil companies how to predict where to drill, though that's not mentioned in the biography (and wouldn't rank as one of his most significant accomplishments anyway).

Not something I was aware of, but good to know.

I wasn't aware of anything from before his career as an academic, 1982-onward. His wikipedia article doesn't mention anything but the atom thing. But he certainly set out to be a Professor of rationality-topics.

Regardless of the merits of E. T. Jaynes, we should place the activity of a rationality instructor in a separate mental bucket than a rationality theoretician. I would say that making a significant original intellectual advance counts as a real accomplishment.

Has this requirement been successfully implemented for CFAR instructors?

It's not a formal requirement, but I'm personally impressed by the prior accomplishments of some of the CFAR staff and some of the careers of CFAR consultants. And there is at least one person who somehow has a non-CFAR career while also working full-time for CFAR.

EDIT: Read about them here.

  1. In the boost phase of a newly launched idea, it's actually a really good idea to train teachers. That gives you exponential growth.

  2. It's a fail if the discipline gets into a death spiral about teaching teachers to teach teachers, iff the recursion lacks a termination condition. (Suitable conditions left as an exercise.)

  3. Even in the cruise phase, an idea needs a teacher replacement rate of >= 1.

  4. In cruise phase, it's a fail if every student wants to teach. But I don't see how it's a fail if some students want to teach and proceed to do so. Nor do I see how it's a fail if they end up being most of the teachers.

  5. The intersection of two very rare categories of people is nobody.

  6. Aren't you the same guy who, just a few days ago, pointed out how much better a trained professional is at his job than some volunteer? Teaching is a nontrivial skill.

In the boost phase of a newly launched idea, it's actually a really good idea to train teachers. That gives you exponential growth.

Most memes that grow exponentially do not manage to stay sane. What's best for the exponential growth of a meme (as though it were a bacterium with no identity other than itself) may not be best for the culturation of a cause. Exponential growth is good, I agree, but the fastest possible exponential growth... seems more doubtful.

Exponential growth around friends giving friends copies of a fixed book, seems safer and shallower; the book won't change as it's passed around. The building-up of rationality dojos is a longer, slower endeavor which should be more carefully gotten right.

Plus you can look for students who've already done something interesting with their lives and train them to be the teachers.

Hmm. I'll concede a measured rate of startup, although I can't offhand think of any meme that got deranged by fast growth (and was sane to begin with).

Perhaps, adopt the martial art idea of not giving out too many certificates-to-teach, and using lineage to check accreditation?

Well if we develop rationality tests, then you should rely on the teachers who help their students do better on tests. And if you can't develop tests, then I don't see why you'd think you had evidence that any particular person was good at teaching rationality. Relying on their ability to do something useful as a predictor of their ability to teach rationality seems nearly as bad as relying on their publication record, or their IQ, or wealth, etc. I say focus on developing tests.

What work is the word "secret" doing in this post? It seems to me that you're talking about public identities, ones visible to outsiders, ones that potential students (not yet enrolled in the Conspiracy) can look at to evaluate would-be instructors. Are you using the phrase "secret identities" merely because it sounds cool?

Ditto with "conspiracy." I'd argue that giving LW the language and trappings of a 12-year old boys' club is ultimately detrimental to its mission, but it looks like I'm in the minority.

The business about the Bayesian Conspiracy is, I think, more an in-joke than anything else. Eliezer's written various bits of fiction set in a future world featuring an actual "Bayesian Conspiracy", and he's on record as saying that there's something to be said for turning things like science and rationality into quasi-mystery-religions (though I expect he'd hate that way of putting it) -- but he's not suggesting that we actually should, nor trying to do so.

Dunno whether such things help or hinder the mission of LW. I think it would be difficult to tell.

It just seems at odds with the scientific ethos of cutting out the bullshit whenever possible. Instead, Eliezer seems bent on injecting bullshit back into the mix, which I'd argue comes at the expense of clarity, precision, and credibility. However, I do realize it's a calculated decision intended to give normally dry ideas more memetic potential, and I'm not in a position to say the trade-off definitely isn't worth it.

It just seems at odds with the scientific ethos

Deliberately so. The original OB posts started with it as a thought experiment, "what if we kept science secret, so people would appreciate its Awesome Mysteries?"

Despite that, I think that whole style is a tremendous mistake. It's an interesting thought experiment, but we should be clear that it runs completely counter to the things that actually bring about accurate results.

Ironic, rather. I considered "Mandatory Alternate Lives" but "alternate life" simply doesn't have the phrase-recognition impact of "secret identity". There is no phrase that means exactly what I want; so I use "secret identity" in an obviously inappropriate way.

I think the phrase you want is "day job".

If anything it's the teaching that ought to be under a nom de plume. I've heard more than once the complaint about universities, that they care more about getting an impressive name than whether he can teach.

How much of Objectivism's failure was due to its teachers not having developed sufficient awesomeness elsewhere, and how much was due to the fact that it, say, tried to claim that it had the One True Method of Thought, instead of fostering an environment where all teachings were conjectural, teachers were facilitators of investigation instead of handers-down of The Answer, and everyone together tried to figure out what worked?

I mean, to what extent can we avoid similar failure modes by fostering a culture that doesn't reify anyone's teachings, but that instead tries to foster a culture of experimenting, thinking up new strategies, pooling data, and asking how we can tell what does and doesn't work?

Some thoughts from my experience in a martial arts dojo:

  1. We avoid lots of failure modes by making sure (as far as reasonably possible) that people are there to train first and everything else second. One consequence of this is that we don't attach a whole lot of our progress to any particular instructor; we're blessed with a number of people who are really good at aikido, and we learn from all of them, and from each other.

  2. On setting the bar too high for instructors: Most martial arts rely on a hierarchy of instructors, where the average dojo head is a reasonably normal person who is expert but not necessarily elite at the discipline. The "famous" people in the art travel around and deliver seminars to everybody else. Dojo head type people will also travel to attend more seminars than the average junior student, for obvious reasons.

All sorts of human enterprises work this way (although the formality of the hierarchy varies widely); everything from yoga to religions to Linux Users Groups. It's a good system.

How much of what you're trying to do could be accomplished by largely tabooing the term "rationality" in rationality dojos, and having the community be really really attached to that tabooing? So that the dojos are for "finding ways of thinking that actually bring accurate beliefs" and "finding ways of thinking that actually help people reach their goals", with mostly no mention of a term like "rationality" that's easy to reify? If we talked like that, actual and prospective students and teachers might naturally look outward, to the evidence that various thinking processes were or weren't helping. Such evidence would be found partly in terms of the actual "day job" accomplishments (or lack of accomplishments) of the teacher, and also in terms of "day job" accomplishments of the students after vs. before joining the group, and also in terms of any measures that a group of active, experimentally minded rationality students could think up of whether they were actually becoming better at forming accurate beliefs.

You can taboo a word, or even a concept, but you can't taboo a meaningful regularity, pretend that it's not there. The problem with belief-in-belief-in-rationality is the same as with other lost concepts, one of the essential lessons to learn, not something to shoo away. If you can't attain even this, what aspirations?

You can taboo a word, or even a concept, but you can't taboo a meaningful regularity, pretend that it's not there.

I'm not proposing we pretend there's no regularity to "types of thinking that help us form accurate beliefs, across domains". Not at all. I'm proposing we stay attentive to the evidence as to what those types of thinking actually are and aren't, by spelling out our full goal as much as possible. If we use the term "rationality" as a shorthand instead of spelling out that we're after "types of thinking that actually help us form accurate beliefs", it's easy for the term "rationality" to become un-glued from the goal. So that "rationality" gets glued to "that thing we do in the rationality dojo" or to "whatever the Great Teacher said" or to "anything that sets me apart from others and lets me feel superior, like using long sentences and being socially awkward", instead of being a term for those meaningful regularities we're actually trying to study (the meaningful regularities in thinking methods that actually work).

The problem with belief-in-belief-in-rationality is the same as with other lost concepts, one of the essential lessons to learn, not something to shoo away.

Well, yes, I agree that a rationality dojo should talk about lost purposes, about the trouble with belief in belief in general, and about what exactly goes wrong when people speak overmuch of "rationality" instead of keeping their eyes on the prize. Is this supposed to be in tension with the suggestion that we, as a community, build a strong norm against talking overmuch of "rationality" and for, instead, speaking of "kinds of thinking that help us form accurate beliefs / achieve our goals"? I'm imagining that it's precisely by having a really clear view of the standard "lost purposes" failure modes, and of their application to "rationality" learning, that we can maintain such a norm.

But for some reason we are talking about a specific failure mode, one that is not necessarily the single best case to demonstrate the general principles, and one that by itself is clearly insufficient. Investing disproportionally in this single case must have additional purposes.

I can see two goals:

  • Safeguarding the movement on early stages, where it's easy to start in a wrong direction
  • Acting as a safety vent, compensating for the difficulty in certifying the sanity of the movement.

Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of...

...becoming a teacher and having their own martial arts dojo someday.

I do not think this analogy fits. Martial arts is a self-contained bubble. What else is there to do but teach? To use a variation on the analogy, if someone being trained in the United States Marine Corps were given the question of what a truly dedicated student of the USMC were to become, they would probably answer along the lines of someone who kills things and doesn't die while doing it.

(Minor point) Martial arts do have tournaments and the like, so I suppose that is an alternate path. There is an inherent time-limit on those activities because of the human aging process, however, and after you retire from fighting, what else is there to do but teach what you learned?

To see what's wrong with this, imagine going to a class on literary criticism, falling in love with it, and dreaming of someday becoming a famous literary critic just like your professor, but never actually writing anything.

This second analogy is in a world where there is something more to do than teach. Choosing not-writing over writing is a failure mode in literary arts but not for literary criticism. Literary criticism is too specific. If you study literary criticism it naturally follows that you will become a literary critic. (Minor point) I could be completely misunderstanding what you meant but "writing anything".

Also, in a sense you digress from the pattern in the first analogy. Being a student of martial arts is to teaching martial arts as being a student literary criticism is to teach literary criticism.

In my opinion, the lesson to be learned here is to study something that can be applied to the real world in a form other than simply teaching other people what you learned. (Someone else mentioned something similar to this in the comments, but I cannot find it again via skimming.)

All that being said, this statement does loosely follow:

Similarly, I propose, no student of rationality should study with the purpose of becoming a rationality instructor in turn. You do that on Sundays, or full-time after you retire.

But I imagine that a lot of people who desire to eventually teach will do so after a full life of other activities. I also imagine this is a decent way to keep generational bias out of a system of education, since the next generation of students will likely spawn new variations and having those variations inserted back into the system by a "newer" instructor can encourage growth. Having the same ol' instructors around limits that somewhat.

The computer science analogy of this would be classes taught by old fogies who still think of programming as playing with decks of punch cards. Even if they move beyond that in their knowledge, their mind is potentially tainted by what-once-was.


I wonder if this idea comes as a shock because everyone was planning on becoming rationality instructors, i.e., I should have warned everyone about this much earlier?

Is it offputting on some other level?

But I must also consider that it might really be that stupid. Damn, now I wish I knew the actual number of upvotes and downvotes!

I don't think too many people are actually considering "rationality instructor" as a career path at this point - which reminds me - what exactly are your plans for this rationality dojo thing anyway? Is it just something you like to talk about, or something you plan to one day set up? Are you hoping people from Less Wrong will start the first ones, or that people from Less Wrong will be students in ones set up in some other way?

When you (or anyone) says "rationality dojo", how literally is it meant? Is it specifically a physical meeting, rather than a web community? More literally, is this meant as a meeting of equals or an instructor with pupils? How much of the formalism of the dojo would you import? How would you change the relationship of sensei and pupil? I'm not so sure about wearing robes, and I draw the line at getting thwacked on the head with a stick.

I am keen to increase the number of rational people, but there are a great many means by which a thing passes from mind to mind, and I'm not sure a dojo would be the first model I'd reach for - have I missed a post where this model is set out in more detail?

I don't think I can afford to divert my attention into setting one up, but I've heard others already discussing it, so it's worth placing some Go stones around it.

Really? If it's not too private, who's been discussing it?

I don't know if I'm part of who Eliezer heard, but I'm planning on trying to start a rationality training group on Saturdays in the SF bay area, for middle and high school students with exceptional mathematical ability. I want to create a community that thinks about thinking, considers which kinds of thinking work for particular tasks (e.g., scientific progress; making friends), and learns to think in ways that work. The reason I'm focusing on kids with exceptional mathematical ability is that I'm hoping some of them will go on to do the kind of careful science humanity needs, with the rationality to actually see what actually helps. The aim is not so much to teach rationality knowledge, since AFAICT the "art of human rationality" is mostly a network of plausible guesswork at this point, but to get people aiming, experimenting, measuring, and practicing in a community, sharing results, trying to figure out what works and actually trying the best ideas (real practice; community resistance to akrasia). With some mundane math teaching mixed in.

As to "day job" credentials, I've had unusual success teaching mathematical thinking (does this count as "day job"? at least math teaching success is measurable by, say, the students' performance on calculus exams), bachelor degrees in math and "great books", and two or three years' experience doing scientific research in various contexts. I don't know if this would put me above or below Eliezer's suggested bar to a stranger.

You're focusing on easy-to-verify credentials of the sort you'd list on a resume to be hired by some skeptical HR person. You have a secret identity.

My secret identity just says that some combination of you and Michael Vassar thought I was worth taking a chance on. I was trying to do some analog of cross-validation, where we ask whether someone who was basically following your procedure but who didn't know me or have particular faith in your or Michael Vassar's judgment, would think it okay for me to try teaching. I was figuring that your focus on day job impressiveness was an attempt to get away from handed-down lineages of legitimacy / "true to the Real Teachings"ness, which Objectivism or martial arts traditions or religions sometimes degenerate into.

More of an attempt to make sure that people write instead of just doing literary criticism.

Got it. Sorry; I think I rounded you to the nearest cliche, maybe because of the emotional reaction you suggested some of us might be having.

I wonder if this idea comes as a shock because everyone was planning on becoming rationality instructors, i.e., I should have warned everyone about this much earlier?

FWIW, part of my own emotional reaction to it did come from that, though I noticed and have my reaction tagged as an emotionally contaminated thing to be wary of.

Speaking only for myself--I am here, consciously and explicitly, to learn rationality for its own benefits. I have no overwhelming interest in teaching others and, all else equal, have other things I would prefer to be doing with my life.

I didn't vote either way on the post because I am ambivalent to it. It felt underdeveloped compared to your usual material, and to some extent seems like you're getting ahead of yourself on this "teaching rationality" thing--the current understanding of applied rationality in this community here doesn't seem to justify raising the concern yet.

Perhaps the idea would have been better presented in the context of one of your parables/short stories/&c.?

Thales, so the story goes, because of his poverty was taunted with the uselessness of philosophy; but from his knowledge of astronomy he had observed while it was still winter that there was going to be a large crop of olives, so he raised a small sum of money and paid round deposits for the whole of the olive-presses in Miletus and Chios, which he hired at a low rent as nobody was running him up; and when the season arrived, there was a sudden demand for a number of presses at the same time, and by letting them out on what terms he liked he realized a large sum of money, so proving that it is easy for philosophers to be rich if they choose, but this is not what they care about.

Huh? There is no way that knowledge of astronomy could possibly have told him about the olive crop. It seems more likely that his useful knowledge was of economics and business, but that he made up a story about astronomy to impress his peers.