A rather out-of-context takeout of:
I recently read of a certain theist that he had defeated Christopher Hitchens in a debate (severely so; this was said by atheists). And so I wrote at once to the Bloggingheads folks and asked if they could arrange a debate. This seemed like someone I wanted to test myself against. Also, it was said by them that Christopher Hitchens should have watched the theist's earlier debates and been prepared, so I decided not to do that, because I think I should be able to handle damn near anything on the fly, and I desire to learn whether this thought is correct; and I am willing to risk public humiliation to find out. Note that this is not self-handicapping in the classic sense—if the debate is indeed arranged (I haven't yet heard back), and I do not prepare, and I fail, then I do lose those stakes of myself that I have put up; I gain information about my limits; I have not given myself anything I consider an excuse for losing.
I don't expect I can handle 'anything' domain-unspecifically on the fly. I thought I should be able to handle arguments William Lane Craig made, or tactics he used, on the fly. The entire article is about "Don't guess ...
I assumed you didn't mean literally anything. But I'm also assuming you know very little about debate tactics, is that correct? If so, the sentence I quoted seems to imply you seem to think you should be able to handle quite a wide range of things on the fly.
Craig is selective about who he debates, but mostly he seems to be optimizing for how big of names his opponents are. As much as I respect your work, Eliezer, you simply aren't as big of a name as Sam Harris or Christopher Hitchens (who've debate Craig) or Jerry Coyne (who recently declined a debate invitation from Craig).
I can suspect him of cowardice when he refuses to debate Jeffrey Jay Lowder (who like Craig and unlike most of Craig's opponents, has a background in college debate), but unless you've got some amazing debate performances under your belt that I don't know about, I don't think fear of losing is the reason he refused to debate you.
Filling in some details of the quote, since I was involved...
I was one of the atheists who said that (theist) William Lane Craig beat (atheist) Christopher Hitchens soundly, here and here. Also, I doubt the "Bloggingheads folks" said that Hitchens should have prepared more — that's what I and perhaps other public atheists said.
Also, if someone doesn't already believe that debates are mostly about debating skill (e.g. using the clock) rather than argument quality, a quick review of how Craig routinely dominates atheists in debates, while arguing not just for theism but also for Christian particularism, might change one's mind.
Good post.
When I notice someone outperforming me in figuring out some part of the world, I like to ask why they're able to do that.
Sometimes the most obvious explanation is sheer IQ. There's just no way I'm going to master model theory as quickly as Paul Christiano or Eliezer Yudkowsky.
In other cases, the most obvious explanation is superior rationalist practice, though this happens less often because applied rationality isn't yet a well-developed field with a high ceiling of human performance, and I'm fortunate enough to be pretty involved with one tentpole of the field.
And with other people, the standout explanation is that they just know way more shit than I do. As it turns out, many of these people are Carl Shulman.
My current strategy for catching up with Carl, or at least for falling behind him less quickly, is to listen to nonfiction audiobooks at 2x speed whenever I'm traveling, and whenever my eyes have given up for the day but my brain still wants more. Then I flip open the Audible app on my phone, put on my eye mask, and start learning about Cold War nuclear security.
Of course, I can't remember the exact details of most of the content, but I remember enough to know where...
I think I can somewhat identify with this.
Growing up I was always told I was really smart; I had teachers telling me I was the smartest in class, right in front of the other students. I got into the gifted children program and I got into the national-level math competition.
What did that all do? Did it make me grow into the next Einstein or Witten? Nope. All that praise just got to my head. At 15 my head was larger than the Hindenburg, and just as doomed to catastrophic failure.
By high school I didn't even study. I didn't put effort into anything. And soon...
I had somewhat of your problem, but fortunately my comparison group consisted of famous scientists and mathematicians, and also competitive chess players -- so instead of thinking how smart I am compared to other people, I was more amazed at the stupidity I saw around me. Specific things that help put me in my place were being clobbered at chess by a kid half my height (much younger than me), getting clobbered at chess by a guy playing against 20 other people simultaneously (this happened on a regular basis with different such people), and realizing that even with the advantage of hindsight, I couldn't understand quantum and general relativity. So I've long known that I'm not memorably smart.
On the subject of embarrassing teachers: I had a college professor who thought endorphins were cells and another who thought only humans ever made tools. A claim he repeated even after I showed him videos of crows making tools.
I did not feel especially good about knowing better regarding these things (and I have many other stories on this topic). I never felt like I had all that much knowledge or special talents and, as I know actual geniuses, I was pretty sure I was not one.
I guess surrounding yourself with people smarter than you are could serve as a sort of humility preserver but apparently that doesn't always work.
As someone who thinks a lot about AI, the fact that domain expertise almost always trumps "raw" IQ seems to me to have very significant implications about the relationship between human learning, intelligence, and the structure of real world problems.
You left out discussion of domain specific intelligence, and I'd like to know why that is. Maybe it was intentional, maybe not. Because of this reason, I suggest your early misconceptions could have just as easily stemmed from misunderstanding what intelligence is (it's not like even the experts understand it well).
For example, it's easy to think you're the best at everything if you really are prodigiously talented just at math and merely very talented in other things like language. On top of this, consider the fact you're more likely to seriously compete ...
This is a good picture of the value of experience.
If the system you're looking at is simple enough for you to entirely analyse and understand it when encountered, experience won't add anything. However even for smart humans this restricts you to pretty simple systems!
Experience allows you to observe and make models of the emergent patterns of high-level behaviour of a system, without understanding the details of how they follow from the lower-level details. In many complex systems this high-level behaviour can be very simple. For example the sun rises once every day. You could understand this in terms of the physics of the situation, but you can simply learn the pattern from experience.
Just as an aside, and not to criticize your frustration at your grade school math teacher, it may be worth spending some time thinking about whether negative numbers in fact exist and what exactly do you mean when you confidently assert that they do.
This post reminded me of this interview with Jeremy Howard, multiple-times winner of Kaggle data prediction contest. The article is titled "Specialist Knowledge Is Useless and Unhelpful" and includes this:
Q. How have experts reacted?
A. The messages are uncomfortable for a lot of people. It's controversial because we're telling them: "Your decades of specialist knowledge are not only useless, they're actually unhelpful; your sophisticated techniques are worse than generic methods."
It's only anecdotal evidence, but still; I think tha...
Experience was mentioned in the comments, but one thing that is missing is emphasizing the value of the subconscious brain(System 1 in Kahneman terminology). A lot of things experts "know" they cant verbalize. Consider a chess master, through thousands of games he has internalized a lot of patterns and "intuitively" knows what moves to consider. He doesnt have to be particularly rational in his thinking or practicing of chess, just sheer repetition is what gave him the skills.
The real power is in the brain(System 1) automatically picking up the patterns without we being aware of it.
Related to: Trusting Expert Consensus
In the sequences, Eliezer tells the story of how in childhood he fell into an affective death spiral around intelligence. In his story, his mistakes were failing to understand until he was much older that intelligence does not guarantee morality, and that very intelligent people can still end up believing crazy things because of human irrationality.
I have my own story about learning the limits of intelligence, but I ended up learning a very different lesson than the one Eliezer learned. It also started somewhat differently. It involved no dramatic death spiral, just being extremely smart and knowing it from the time I was in kindergaarden. To the point that I grew up with the expectation that, when it came to doing anything mental, sheer smarts would be enough to make me crushingly superior to all the other students around me and many of the adults.
In Harry Potter and the Methods of Rationality, Harry complains of having once had a math teacher who didn't know what a logarithm is. I wonder if this is autobiographical on Eliezer's part. I have an even better story, though: in second grade, I had a teacher who insisted there was no such thing as negative numbers. The experience of knowing I was right about this, when the adult authority figure was so very wrong, was probably not good for my humility.
But such brushes with stupid teachers probably weren't the main thing that drove my early self-image. It was enough to be smarter than the other kids around me, and know it. Looking back, there's little that seems worth bragging about. I learned calculus at age 15, not age 8. But that was still younger than any of the other kids I knew took calculus (if they took it at all). And knowing I didn't know any other kids as smart as me did funny things to my view of the world.
I'm honestly not sure I realized there were any kids in the whole world smarter than me until sophomore year, when I qualified to go to a national-level math competition. That was something that no one else at my high school managed to do, not even the seniors... but at the competition itself, I didn't do particularly well. It was one of the things that made me realize that I wasn't, in fact, going to be the next Einstein. But all I took from the math competition was that there were people smarter than me in the world. It didn't, say, occur to me that maybe some of the other competitors had spent more time practicing really hard math problems.
Eliezer once said, "I think I should be able to handle damn near anything on the fly." That's a pretty good description of how I felt at this point in my life. At least as long as we were talking about mental challenges and not sports, and assuming I wasn't going up against someone smarter than myself.
I think my first memory of getting some inkling that maybe sufficient intelligence wouldn't lead to automatically being the best at everything comes from... *drum roll* ...playing Starcraft. I think it was probably junior or senior that I got into the game, and at first I just did the standard campaign playing against the computer, but then I got into online play, and promptly got crushed. And not just by one genius player I encountered on a fluke, but in virtually every match.
This was a shock. I mean, I had friends who could beat me at Super Smash Bros, but Starcraft was a strategy game, which meant it should be like chess, and I'd never had any trouble beating my friends at chess. Sure, when I'd gone to local chess tournaments back in grade school, I'd gotten soundly beat by many of the older players then, but it's not like I'd ever expected all older people to be as stupid as my second grade teacher. But by the time I'd gotten into Starcraft, I was almost an adult, so what was going on?
The answer of course was that most of the other people playing online had played a hell of a lot more Starcraft than me. Also, I'd thought I'd figured the game designer's game-design philosophy (I hadn't), which had let me to make all kinds of incorrect assumptions about the game, assumptions which I could have found out were false if I'd tested them, or (probably) if I'd just looked for an online guide that reported the results of other people's tests.
It all sounds very silly in retrospect, and it didn't change my worldview overnight. But it was (among?) the first of a series of events that made me realize that trying to master something just by thinking about it tends to go badly wrong. That when untrained brilliance goes up against domain expertise, domain expertise will generally win.
A whole bunch of caveats here. I'm not denying that being smart is pretty awesome. As a smart person, I highly recommend it. And acquiring domain expertise requires a certain minimum level of intelligence, which varies from field to field. It's only once you get beyond that minimum that more intelligence doesn't help as much as expertise. Finally, I'm talking about human scale intelligence here, the gap between the village idiot and Einstein is tiny compared to the gap between Einstein and possible superintelligences, so maybe a superintelligence could school any human expert in anything without acquiring any particular domain expertise.
Still, when I hear Eliezer say he thinks he should be able to handle anything on the fly, it strikes me as incredibly foolish. And I worry when I see fellow smart people who seem to think that being very smart and rational gives them grounds to dismiss other people's domain expertise. As Robin Hanson has said:
I was a physics student and then a physics grad student. In that process, I think I assimilated what was the standard worldview of physicists, at least as projected on the students. That worldview was that physicists were great, of course, and physicists could, if they chose to, go out to all those other fields, that all those other people keep mucking up and not making progress on, and they could make a lot faster progress, if progress was possible, but they don’t really want to, because that stuff isn’t nearly as interesting as physics is, so they are staying in physics and making progress there...
Surely you can look at some little patterns but because you can’t experiment on people, or because it’ll be complicated, or whatever it is, it’s just not possible. Partly, that’s because they probably tried for an hour, to see what they could do, and couldn’t get very far. It’s just way too easy to have learned a set of methods, see some hard problem, try it for an hour, or even a day or a week, not get very far, and decide it’s impossible, especially if you can make it clear that your methods definitely won’t work there. You don’t, often, know that there are any other methods to do anything with because you’ve learned only certain methods...
As one of the rare people who have spent a lot of time learning a lot of different methods, I can tell you there are a lot out there. Furthermore, I’ll stick my neck out and say most fields know a lot. Almost all academic fields where there’s lots of articles and stuff published, they know a lot.
(For those who don't know: Robin spent time doing physics, philosophy, and AI before landing in his current field of economics. When he says he's spent a lot of time learning a lot of different methods, it isn't an idle boast.)
Finally, what about the original story that Eliezer says set off his original childhood death spiral around intelligence?:
My parents always used to downplay the value of intelligence. And play up the value of—effort, as recommended by the latest research? No, not effort. Experience. A nicely unattainable hammer with which to smack down a bright young child, to be sure. That was what my parents told me when I questioned the Jewish religion, for example. I tried laying out an argument, and I was told something along the lines of: "Logic has limits, you'll understand when you're older that experience is the important thing, and then you'll see the truth of Judaism." I didn't try again. I made one attempt to question Judaism in school, got slapped down, didn't try again. I've never been a slow learner.
I think concluding experience isn't all that great is the wrong response here. Experience is important. The right response is to ask whether all older, more experienced people see the truth of Judaism. The answer of course is that they don't; a depressing number stick with whatever religion they grew up with (which usually isn't Judaism), a significant number end up non-believers, and a few convert to a new religion. But when almost everyone with a high level relevant experience agrees on something, beware thinking you know better than them based on your superior intelligence and supposed rationality.
Edit: One thing I meant to include when I posted this but forgot: one effect of my experiences is that I tend to see domain expertise where other people see intelligence. See e.g. this old comment by Robin Hanson: are hedge fundies really that smart, or have they simply spent a lot of time learning to seem smart in conversation?