What does this post even mean? I don't have access to my own respect function, and I don't know if I'd mess with it this way even if I did.
If you were to say tomorrow "I've been lying about the whole AI programmer thing; I actually live in my parents' basement and have never done anything worthwhile in any non-rationality field in my entire life," then would I have to revise my opinion that you're a very good rationality teacher? Would I have to deny having learned really valuable things from you?
Or would I have to say, "Well, this guy named Eliezer taught me everything I know, he's completely opened my mind to new domains of knowledge, and you should totally read everything he's written - but he's not all that great and I don't have any respect for him and you shouldn't either" when referring people to your writing?
Or to put it another way...let's say there are two rationality instructors in my city. One, John, is a world famous physicist, businessman, and writer. The other, Mary, has no particular accomplishments outside her rationality instruction work. However, Mary's students have been observed to do much better at their careers than John's, and every time ...
If you were to say tomorrow "I've been lying about the whole AI programmer thing; I actually live in my parents' basement and have never done anything worthwhile in any non-rationality field in my entire life," then would I have to revise my opinion that you're a very good rationality teacher? Would I have to deny having learned really valuable things from you?
But the fact that reality doesn't disentangle this way, is in a sense the whole point - it's not a coincidence that things are the way they are.
If we get far enough to have external real-world standards like those you're describing, then yes we can toss the "secret identity" thing out the window, so long as we don't have the problem of most good students wanting only to become rationality instructors themselves as opposed to going into other careers (but a teacher who raised their students this way would suffer on the 'accomplished students' metric, etc.). But on the other hand I still suspect that the instructors with secret identities would be revealed to do better.
I've never seen anything from Eliezer that proves that he's done anything at all of value except be a rationality teacher. I know of two general criteria by which to judge someone's output in a field that I am not a part of:
1) Academic prestige (degrees, publications, etc.) and 2) Economic output (making things that people will pay money for).
Eliezer's institution doesn't sell anything, so he's a loss on part 2. He doesn't have a Ph.D or any academic papers I can find, so he's a loss on part 1, as well. Can SIAI demonstrate that it's done anything except beg for money, put up a nice-looking website, organize some symposiums, and write some very good essays?
To be honest, I'd say that his output matches the job description of "philosopher" than "engineer" or "scientist". Not that there's anything wrong with that. Many works that fall broadly under the metric of philosophy have been tremendously influential. For example, Adam Smith was a philosopher.
Eliezer seems to have talents both for seeing through confusion (and its cousin, bullshit) and for being able to explain complicated things in ways that people can understand. In other words, he'd be an amazing university professor. I just haven't seen him prove that he can do anything else.
Yes - in fact, the only thing that leads me to suspect that EY and SIAI are doing anything worth doing is the quality of EY's writings on rationality.
Is the point about respect for instructors supposed to generalize to instructors of disciplines other than rationality?
If so, what do you make of Nadia Boulanger? Her accomplishments as a musician (or otherwise) are unimpressive relative to those of her students and peers, and yet she is regarded as one of the greatest music teachers ever, and is accorded correspondingly deep respect by music historians, composers, etc. Are they all wrong to respect her so much, or does it not apply to music or this case?
It seems to me that a better formula for determining respect would somehow reflect the respect given to her students which they say is significantly due to her influence as a teacher. For example, if Aaron Copland singles her out as an amazing teacher who profoundly affected his musical life & education, then she deserves some of the respect given to him. And likewise for her many other students who went on to do great things.
There seems to be an implicit underlying belief in this post that teaching is not (or should not be) an end in and of itself, or at least not a worthy one. I think Boulanger and teachers of her caliber show that that's just not the case.
You've got things the wrong way round. It is the quality of the teacher's students that tell us whether we wish to study under her. The teachers own achievements are a proxy which we resort to because we need to decide now, we cannot wait to see the longer term effects on last years students.
Another proxy is the success of the teacher in getting her students through examinations. This is a proxy because we don't really want the certificate, we what the achievement that we think it heralds. We can assess the strength of this proxy checking whether success in the examinations really does herald success in real life.
I agree with the conclusion of the original post but find the argument for it defective. The key omission is that we don't have a tradition of rationality dojo's, so we do not yet have access to records of whose pupils went on to greatness. Nor do we have records that would validate an examination system.
Notice that the problems of timing are inherent. The first pupils, who went on to real world success, prove their teachers skill in an obvious way, but how did they choose their teacher? Presumably they took a risk, relying on a proxy that was available in time for the forced choice they faced.
In the boost phase of a newly launched idea, it's actually a really good idea to train teachers. That gives you exponential growth.
It's a fail if the discipline gets into a death spiral about teaching teachers to teach teachers, iff the recursion lacks a termination condition. (Suitable conditions left as an exercise.)
Even in the cruise phase, an idea needs a teacher replacement rate of >= 1.
In cruise phase, it's a fail if every student wants to teach. But I don't see how it's a fail if some students want to teach and proceed to do so. Nor do I see how it's a fail if they end up being most of the teachers.
The intersection of two very rare categories of people is nobody.
Aren't you the same guy who, just a few days ago, pointed out how much better a trained professional is at his job than some volunteer? Teaching is a nontrivial skill.
No.
You're wasting huge amounts of optimization power, here, in two different ways. Firstly, you're saying that no one should focus his efforts on becoming a good rationality instructor, that any work he does on that is entirely meaningless unless he is at least as good at something else. Secondly, you're saying that no one should focus his efforts on instructing people in rationality, that they should spend most of their time on whatever other thing it is that makes them impressive. If you have someone who is naturally better at instructing people in rationality than in anything else, you are wasting most of the surplus you could have gained from him in these two ways.
I'm sympathetic to your concern, but surely there must be a way we can avoid throwing out the baby with the bathwater?
Personally I suspect that the bathwater only really gets dirty when you are teaching something that is essentially useless in modern society, like martial arts or literary criticism. Most people who study, say, engineering don't do so in the hopes of becoming teachers of engineering.
Now you might say that this is because teachers of engineering are expected to also do research, but firstly that doesn't explain the disparity between fields, and secondly, I don't think that the example of tertiary education is one to aspire to in this way. I seem to recall you are an autodidact, so you may not have the same trained gut reaction I do, but I have seen too many people who did not have the skill of teaching but were good researchers teaching horribly, and I remember one heartbreaking example of an excellent teacher denied tenure because the administrators felt his research was not up to snuff too well, to want to optimize rationality teachers on any basis other than their ability to teach rationality.
While I think martial arts are pretty useful by hobby standards (although their usefulness is broad enough that they might not be optimal for specialists in several fields), several historical and cultural factors in their practice have combined to create an unusually fertile environment for certain kinds of irrationality.
First, they're hard to verify: what works in point sparring might not work in full-contact sparring, and neither one builds quite the same skillset that's useful for, say, security work, or for street-level self-defense, or for warfare. It's difficult to model most of the final applications, both because they entail an unacceptably high risk of serious injury in training and because they involve psychological factors that don't generally kick in on the mat.
Second, they're all facets of a field that's too broad to master in its entirety in a human lifetime. A serious amateur student can, over several years, develop a good working knowledge of grappling, or of aikido-style body dynamics, or empty-hand striking, or one or two weapons. The same student cannot build all of the above up to an acceptable level of competence: even becoming sort of okay at the entire sp...
To be fair, for people who are used to thinking in math, pseudo-mathematical notation is as readable as English, with advantages of brevity and precision.
"People used to thinking in math" currently describes a large portion of users on this site. Use of gratuitous mathematical notion is likely to help keep it that way.
knock E.T. Jaynes out of the running
Dude, what on Earth are you talking about. E. T. Jaynes was a Big Damn Polymath. I seem to also recall that in his later years he was well-paid for teaching oil companies how to predict where to drill, though that's not mentioned in the biography (and wouldn't rank as one of his most significant accomplishments anyway).
Well if we develop rationality tests, then you should rely on the teachers who help their students do better on tests. And if you can't develop tests, then I don't see why you'd think you had evidence that any particular person was good at teaching rationality. Relying on their ability to do something useful as a predictor of their ability to teach rationality seems nearly as bad as relying on their publication record, or their IQ, or wealth, etc. I say focus on developing tests.
(Blinks.)
I wonder if this idea comes as a shock because everyone was planning on becoming rationality instructors, i.e., I should have warned everyone about this much earlier?
Is it offputting on some other level?
But I must also consider that it might really be that stupid. Damn, now I wish I knew the actual number of upvotes and downvotes!
I don't know if I'm part of who Eliezer heard, but I'm planning on trying to start a rationality training group on Saturdays in the SF bay area, for middle and high school students with exceptional mathematical ability. I want to create a community that thinks about thinking, considers which kinds of thinking work for particular tasks (e.g., scientific progress; making friends), and learns to think in ways that work. The reason I'm focusing on kids with exceptional mathematical ability is that I'm hoping some of them will go on to do the kind of careful science humanity needs, with the rationality to actually see what actually helps. The aim is not so much to teach rationality knowledge, since AFAICT the "art of human rationality" is mostly a network of plausible guesswork at this point, but to get people aiming, experimenting, measuring, and practicing in a community, sharing results, trying to figure out what works and actually trying the best ideas (real practice; community resistance to akrasia). With some mundane math teaching mixed in.
As to "day job" credentials, I've had unusual success teaching mathematical thinking (does this count as "day job"? at least math teaching success is measurable by, say, the students' performance on calculus exams), bachelor degrees in math and "great books", and two or three years' experience doing scientific research in various contexts. I don't know if this would put me above or below Eliezer's suggested bar to a stranger.
Thales, so the story goes, because of his poverty was taunted with the uselessness of philosophy; but from his knowledge of astronomy he had observed while it was still winter that there was going to be a large crop of olives, so he raised a small sum of money and paid round deposits for the whole of the olive-presses in Miletus and Chios, which he hired at a low rent as nobody was running him up; and when the season arrived, there was a sudden demand for a number of presses at the same time, and by letting them out on what terms he liked he realized a large sum of money, so proving that it is easy for philosophers to be rich if they choose, but this is not what they care about.
"no way...could possibly..." (emphasis added)
This is a good example of what I meant over in the evolutionary psychology thread; coming up with evolutionary psychology explanations is a good practice to avoiding succumbing to 'arguments from incredulity', as I like to call this sort of comment.
"Oh, I couldn't think of how astronomy could possibly be useful in weather or crop forecasting, so I'll just assume the stories about Thales are a lie."
I'll leave this here for you.
" Forecasting Andean rainfall and crop yield from the influence of El Niño on Pleiades visibility", Nature 403, 68-71 (6 January 2000):
..."Farmers in drought-prone regions of Andean South America have historically made observations of changes in the apparent brightness of stars in the Pleiades around the time of the southern winter solstice in order to forecast interannual variations in summer rainfall and in autumn harvests. They moderate the effect of reduced rainfall by adjusting the planting dates of potatoes, their most important crop1. Here we use data on cloud cover and water vapour from satellite imagery, agronomic data from the Andean altiplano and an index of El Niño va
Some thoughts from my experience in a martial arts dojo:
We avoid lots of failure modes by making sure (as far as reasonably possible) that people are there to train first and everything else second. One consequence of this is that we don't attach a whole lot of our progress to any particular instructor; we're blessed with a number of people who are really good at aikido, and we learn from all of them, and from each other.
On setting the bar too high for instructors: Most martial arts rely on a hierarchy of instructors, where the average dojo head is a reasonably normal person who is expert but not necessarily elite at the discipline. The "famous" people in the art travel around and deliver seminars to everybody else. Dojo head type people will also travel to attend more seminars than the average junior student, for obvious reasons.
All sorts of human enterprises work this way (although the formality of the hierarchy varies widely); everything from yoga to religions to Linux Users Groups. It's a good system.
How much of what you're trying to do could be accomplished by largely tabooing the term "rationality" in rationality dojos, and having the community be really really attached to that tabooing? So that the dojos are for "finding ways of thinking that actually bring accurate beliefs" and "finding ways of thinking that actually help people reach their goals", with mostly no mention of a term like "rationality" that's easy to reify? If we talked like that, actual and prospective students and teachers might naturally look outward, to the evidence that various thinking processes were or weren't helping. Such evidence would be found partly in terms of the actual "day job" accomplishments (or lack of accomplishments) of the teacher, and also in terms of "day job" accomplishments of the students after vs. before joining the group, and also in terms of any measures that a group of active, experimentally minded rationality students could think up of whether they were actually becoming better at forming accurate beliefs.
What work is the word "secret" doing in this post? It seems to me that you're talking about public identities, ones visible to outsiders, ones that potential students (not yet enrolled in the Conspiracy) can look at to evaluate would-be instructors. Are you using the phrase "secret identities" merely because it sounds cool?
Ditto with "conspiracy." I'd argue that giving LW the language and trappings of a 12-year old boys' club is ultimately detrimental to its mission, but it looks like I'm in the minority.
How much of Objectivism's failure was due to its teachers not having developed sufficient awesomeness elsewhere, and how much was due to the fact that it, say, tried to claim that it had the One True Method of Thought, instead of fostering an environment where all teachings were conjectural, teachers were facilitators of investigation instead of handers-down of The Answer, and everyone together tried to figure out what worked?
I mean, to what extent can we avoid similar failure modes by fostering a culture that doesn't reify anyone's teachings, but that instead tries to foster a culture of experimenting, thinking up new strategies, pooling data, and asking how we can tell what does and doesn't work?
Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of...
...becoming a teacher and having their own martial arts dojo someday.
I do not think this analogy fits. Martial arts is a self-contained bubble. What else is there to do but teach? To use a variation on the analogy, if someone being trained in the United States Marine Corps were given the question of what a truly dedicated student of the USMC were to become, they would probably answer along the lines of someone who kills th...
Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of.. becoming a teacher and having their own martial arts dojo someday.
I think that accademia is also subject to this mode of failure. As an exercise, try to think of great literary figures who were also professors of literature at major universities. Off the top of my head, I can think of exactly one: Vladimir Nabokov, and he was notably contemptuous of his colleagues. Can anyone else think up anymore?
Unsurprisingly, Paul Graham h...
Obviously success in other realms is bayesian evidence that someone would make a better rationality instructor. But as many others have argued, in this post Eliezer exaggerates the importance of this type of evidence.
I have a question: why are you panicking about this now? It's not like we have a huge problem yet with too many teachers, or too many freshly founded schools.
Yes, and my impression has been that annointed disciples are generally the instigators of things going subtly wrong in self-reinforcing ways. People with big, novel ideas are not necessarily good judges of character.
I've been expecting a deliberately daft post from Eliezer Yudkowsky and/or Robin Hanson to see whether we vote them up just based upon status.
I think this is it.
Makes sense, though I will quibble with your opening line. What you say about martial arts dojos was probably true up until about twenty years ago, but today I suspect a sufficiently dedicated martial arts student is in fact dreaming of becoming a champion MMA fighter.
And you know, now that I think about it, even twenty years ago, I'm not sure anyone was dreaming of becoming a dojo owner. That was just what they could practically achieve. But they were dreaming of becoming a Dark Lord:
"Surely you've wanted to hurt people," said Professor Quirrell. "You wanted to hurt those bullies today. Being a Dark Lord means that people you want to hurt get hurt.
I guess the failure mode that you're concerned with is a slow dilution because errors creep in with each successive generation and there's no external correction.
I think that the way we currently prevent this in our scientific efforts is to have both a research and a teaching community. The research community is structured to maximise the chances of weeding out incorrect ideas. This community then trains the teachers.
The benefits of this are that you get the people who are best at communicating doing the teaching and the people who are the best at research...
The mini-intro to this post on the craft and community sequence page says that it was not well received. But the requirements that this write up recommends really act as beautiful safeguard against becoming pedantic. If I hadnt read this page quite early (before I got past the 25% mark on the sequences), I doubt I would have stopped myself from falling into a happy death spiral (I honestly still really struggle with that one all the time).
It's really hard for me even now to "not speak over much of the way" (though, I mostly think it to myself, ...
If you have "something to protect", if your desire to be rational is driven by something outside of itself, what is the point of having a secret identity? If each student has that something, each student has a reason to learn to be rational -- outside of having their own rationality dojo someday -- and we manage to dodge that particular failure mode. Is having a secret identity a particular way we could guarantee that each rationality instructor has "something to protect"?
Previously in series: Whining-Based Communities
"But there is a reason why many of my students have achieved great things; and by that I do not mean high rank in the Bayesian Conspiracy. I expected much of them, and they came to expect much of themselves." —Jeffreyssai
Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of...
...becoming a teacher and having their own martial arts dojo someday.
To see what's wrong with this, imagine going to a class on literary criticism, falling in love with it, and dreaming of someday becoming a famous literary critic just like your professor, but never actually writing anything. Writers tend to look down on literary critics' understanding of the art form itself, for just this reason. (Orson Scott Card uses the analogy of a wine critic who listens to a wine-taster saying "This wine has a great bouquet", and goes off to tell their students "You've got to make sure your wine has a great bouquet". When the student asks, "How? Does it have anything to do with grapes?" the critic replies disdainfully, "That's for grape-growers! I teach wine.")
Similarly, I propose, no student of rationality should study with the purpose of becoming a rationality instructor in turn. You do that on Sundays, or full-time after you retire.
And to place a go stone blocking this failure mode, I propose a requirement that all rationality instructors must have secret identities. They must have a life outside the Bayesian Conspiracy, which would be worthy of respect even if they were not rationality instructors. And to enforce this, I suggest the rule:
Rationality_Respect1(Instructor) = min(Rationality_Respect0(Instructor), Non_Rationality_Respect0(Instructor))
That is, you can't respect someone as a rationality instructor, more than you would respect them if they were not rationality instructors.
Some notes:
• This doesn't set Rationality_Respect1 equal to Non_Rationality_Respect0. It establishes an upper bound. This doesn't mean you can find random awesome people and expect them to be able to teach you. Explicit, abstract, cross-domain understanding of rationality and the ability to teach it to others is, unfortunately, an additional discipline on top of domain-specific life success. Newton was a Christian etcetera. I'd rather hear what Laplace had to say about rationality—Laplace wasn't as famous as Newton, but Laplace was a great mathematician, physicist, and astronomer in his own right, and he was the one who said "I have no need of that hypothesis" (when Napoleon asked why Laplace's works on celestial mechanics did not mention God). So I would respect Laplace as a rationality instructor well above Newton, by the min() function given above.
• We should be generous about what counts as a secret identity outside the Bayesian Conspiracy. If it's something that outsiders do in fact see as impressive, then it's "outside" regardless of how much Bayesian content is in the job. An experimental psychologist who writes good papers on heuristics and biases, a successful trader who uses Bayesian algorithms, a well-selling author of a general-audiences popular book on atheism—all of these have worthy secret identities. None of this contradicts the spirit of being good at something besides rationality—no, not even the last, because writing books that sell is a further difficult skill! At the same time, you don't want to be too lax and start respecting the instructor's ability to put up probability-theory equations on the blackboard—it has to be visibly outside the walls of the dojo and nothing that could be systematized within the Conspiracy as a token requirement.
• Apart from this, I shall not try to specify what exactly is worthy of respect. A creative mind may have good reason to depart from any criterion I care to describe. I'll just stick with the idea that "Nice rationality instructor" should be bounded above by "Nice secret identity".
• But if the Bayesian Conspiracy is ever to populate itself with instructors, this criterion should not be too strict. A simple test to see whether you live inside an elite bubble is to ask yourself whether the percentage of PhD-bearers in your apparent world exceeds the 0.25% rate at which they are found in the general population. Being a math professor at a small university who has published a few original proofs, or a successful day trader who retired after five years to become an organic farmer, or a serial entrepreneur who lived through three failed startups before going back to a more ordinary job as a senior programmer—that's nothing to sneeze at. The vast majority of people go through their whole lives without being that interesting. Any of these three would have some tales to tell of real-world use, on Sundays at the small rationality dojo where they were instructors. What I'm trying to say here is: don't demand that everyone be Robin Hanson in their secret identity, that is setting the bar too high. Selective reporting makes it seem that fantastically high-achieving people have a far higher relative frequency than their real occurrence. So if you ask for your rationality instructor to be as interesting as the sort of people you read about in the newspapers—and a master rationalist on top of that—and a good teacher on top of that—then you're going to have to join one of three famous dojos in New York, or something. But you don't want to be too lax and start respecting things that others wouldn't respect if they weren't specially looking for reasons to praise the instructor. "Having a good secret identity" should require way more effort than anything that could become a token requirement.
Now I put to you: If the instructors all have real-world anecdotes to tell of using their knowledge, and all of the students know that the desirable career path can't just be to become a rationality instructor, doesn't that sound healthier?
Part of the sequence The Craft and the Community
Next post: "Beware of Other-Optimizing"
Previous post: "Whining-Based Communities"