Jeez, what's next? "Optimism Bias and My First Kiss"?
As an avid reader, I have to say I'm enjoying the personal revelations. They have much in common with certain varieties of religious exuberance. In fact the whole sequence has the feel of an Olde Tyme Rationalist Revival.
But butt smacking recollections from summer camp? Surely there has to be a line somewhere.
I know. I can't help it. That was my first step into advanced rationality. I mean, what was I supposed to say? It was an important point and I wanted to get it across.
The truth doesn't always save a person. It especially doesn't always save a person from embarrassment.
This sort of thing intrigues me. It underlines for me that our "technology of the mind" is still very young and unstructured.
Imagine what would happen if this one skill were explicitly taught in schools?
Out of curiosity, if you're still reading LessWrong, has this question resurfaced often since then? Do you think (or anyone think, since this is by no means a closed personal question) progress has been made on this "front"? Is the principle applicable, and is it realistic to have rationality be explicitly taught in public education? Is CFAR exactly what you had in mind back then?
I have been thinking this ever since I started the sequences. Another idea would be to try to produce children's books, and get them distributed as widely as possible.
The hard part is that it's one of those mental skills that can't really be taught. You can tell people about it, but they have to learn it for themselves. Because, even once you know about it intellectually, what it "feels" like when your brain is deliberately not thinking about something is almost certainly a subjective experience that will be different for everyone.
So, like Zen, you'd have to work out a large set of training scenarios that put a person in a situation where it'll happen and then draw their attention to it, and plan on having to run most people through quite a few of them before they grok.
I prefer to see examples like this. There are lots of good reasons for grounding these discussions in the concrete.
An exercise I adopted when I was a child that I did not read anywhere is to recall or reconstruct the thought that led to my current thought, then the thought before that, etc. By examining many such transitions, I discovered some generalizations about the unconscious function that takes thought N to thought N+1.
I do the same, too. (But I had never thought of it as an exercise, just curiosity about “Why the deuce am I thinking about this?”.) I think there was an xkcd comic about that, too, but I can't find it now.
Did you ever actually assert that your reasoning was out of principle, only to suddenly remember the experience through an unrelated pathway and thus the remember the real reason of your refusal later on, contradicting that initial assertion, or was it simply that you had never actually considered your reason of refusal until you were 15, and that it had only felt like you had been making the assumption that you had refused out of principle when you had never asserted that as truth- you had just never considered your reason, and you knew when you were 15 that there was reason to refuse- because it was a negative-sum game.
So, Eliezer. You're interested in the Singularity. And the Singularity will come about by means of self-improving artificial intelligence. If I understand correctly. Meanwhile, you're also interested in self-improvement of the human wetware - hence your contributions to this website. There's an obvious similarity between these two interests, possibly even an overlap.
Took me a while to notice it.
Richard,
By examining many such transitions, I discovered some generalizations about the unconscious function that takes thought N to thought N+1.
I'm very interested, please elaborate!
Eliezer,
You started noticing the value of beating specific biases at age 15, but apparently didn't get the importance of heuristics and biases in general till 2003. Why did that reflective generalisation take so long, when in hindsight it's so vital for your chosen path? That's not meant as criticism - I've done this too. I want to know how to fix it.
Incentive to think about this: doing so might lead you to notice something vital for safe seed AI.
RI, the problem was simply the size of the planet and its scientific literature. I just didn't run into the field of heuristics and biases until that late. I realized it was important the first time I read a mention-in-passing on a webpage - though I didn't realize how much there was to know and how little of it I knew; Emil Gilliam finally mailed me a copy of "Judgment Under Uncertainty".
At the age of fifteen, I realized the importance of defeating biases in general, I just didn't know there was a field that studied them. I knew the importance of rationality in general, but I'd never heard the word "Bayesian". It's a frighteningly big world out there - not like a hunter-gatherer tribe, where you can presume you know everything anyone else knows.
-- and since most writings in psychology are worthless, it is easy to give up on the whole field before one discovers worthwhile writings like "Judgement Under Uncertainty" and "The Moral Animal".
RI, a large part of my motivatation was simply to practice a mental skill: it is a delightful feeling to improve drastically one's ability to observe one's own deliberations. Three decades and a severe bump on the head separate my teenage years from today, and today I am almost completely unable to do this exercise.
BTW, it is my guess that the exercises Eliezer and I describe will confer most of their benefits on exercisers who are still teenagers.
RI, to answer your question: the function that takes thought N into thought N+1 is complex enough that I did not learn anything that could be put into neat sentences, nor do I retain any declarative memories of what I learned except that the deliberation proceeded in a much more "predictable-in-retrospect" manner when I thought about some themes than when I thought about others. E.g., I remember that thinking about my mom produced very opaque chain of thoughts.
The practice Eliezer describes strikes me as of greater potential benefit than the one I describe, but perhaps the one I describe can be accomplished by a greater fraction of teenagers reading these words. Very few individuals are blessed with the delightful hardware that the teenage Eliezer had available for such exercises.
Thanks for your replies,
and since most writings in psychology are worthless, it is easy to give up on the whole field
Fair point. But if it annoys us, think how the rational minds in the field must feel!
...it is a delightful feeling... ...did not learn anything that could be put into neat sentences... ...much more "predictable-in-retrospect"...
Are you sure the model that your mind was using to report its own thought process corresponded to what was really going on under the bonnet? I've barely scratched the surface yet in studying bias and rationality (the 2 books you recommended are queuing on my shelf) but a recurring theme seems to be that introspection about one's own thought processes is often demonstrably unreliable, even among highly intelligent subjects working within their fields of expertise and reporting very high confidence in their introspections.
Eliezer,
Maybe I'm just biased towards cross-fertilization between disciplines. I was just wondering about other vital but relatively obscure knowledge being missed for similar reasons, and if there might be a way to attack the problem. Anyhow, please thank Emil for us, a lot of people are getting stronger thanks to her little gift! :-)
...her little gift...
That's what I get for posting at 1am. Apologies to Emil, who I see is an SIAI donor and therefore one of my favorite people :-)
That's a horrible game. Did you have a name for the game as well? Spank-to-spank?
I remember playing a game in which we would take turns punching each other in the stomach as hard as we could while trying not to flinch, but I viewed it as positive sum. Even though you get punched it's still a lot of fun.
Glad to know it wasn't just my own summer camp which was that deranged.
Might've been fun for you, but was it fun for everyone who was pressured into playing? I'm also slightly skeptical of your after-the-fact judgment that the initiation ritual was fun. But not too skeptical, for different people are different.
The ones for whom it was not fun were the ones in most need of the game.
It's natural to be afraid of pain. It's your body's signal for damage. But pain in that kind of controlled environment, where you know you are fundamentally safe, and no damage will occur, gives you a chance to get over your fear of pain, which in most cases is much more painful than the pain itself.
There are few things that a child will face that is more painful than the fear of pain. I think it's healthy for a child to be gently led out of that fear. When I was young, I was too afraid, and wish that someone had helped me to get past it.
Glad to know it wasn't just my own summer camp which was that deranged.
Far from it. I suspect it's a fairly universal game, either to signal "I am tough", or practice for handling pain or fear.
The version of this game popular in my country involves taking turns kicking a soccer ball at each other from a given distance, usually while the target is facing a wall. Whoever kicks the ball is up next, even if they miss. I assume it is a positive sum game, because kids would play this game of their own volition, though that could be because they don't want to refuse when invited. It was a rather uncommon game, relative to proper games.
I've also seen a game called "mercy" where the fingers of both parties are interlaced and bent backwards until someone gives up, the game called "chicken" involving driving cars at each other (which I've heard a lot about but never seen played), the crushing handshake, truth or dare. However, the most intriguing one is the drinking contest.
We played it with thrown balls, and the target had to stand there until someone missed. But every time someone hit the person the throwing distance was increased by a step.
I totally agree about it being practice for handling pain and finding out what the limits are in a safe manner. You'll see baby animals doing the same thing as they play, slowly ramping up the level of roughness until somebody squawks.
Unfortunately, it's also a way to reinforce an in-group if you can get some out-group players involved. I only played it once since it didn't take me long to notice that, somehow, I was the only one who ever got actually hit with the ball whenever I was involved.
Interesting. As a child I thought I could remember everything, so lying was easy, it's own memory of having lied, and thus distinct from reality. It was only much later that I realized it was even possible for the two to become horribly confused in one's own mind. But did you lie and forget the lie? Or did you just incompletely remember and add the part about negative sums as an explanation of the behavior later on? I think the "sweeping" of our minds' corners is also prone to give us false memories. We think, "well, why did I do that? Oh yeah! Now I remember..." You might have just been lucky to stumble upon what you now believe to be the real memory, but introspection, especially of the past isn't always so accurate.
Are you sure the model that your mind was using to report its own thought process corresponded to what was really going on under the bonnet?
I was very confident that I could reliably retain or reconstruct the last five or so actual thoughts of my conscious deliberation -- and the actual ordering or sequencing of those thoughts. Although recalling or reconstructing a thought sometimes required a few seconds of effort or patience, the reconstruction sure felt like a reliable observation rather than an interpretation or the updating of a model or the use of a learned model. As for extracting generalizations or regularities from those observations, like I said, I can recall making only one (declarative) generalization, but I have no reason to believe that I could not bring the same regularity-noticing, causality-extracting powers to bear on those observations that I could to any observations made by my senses. (When observing a very complex process, the regularity-noticing consists mostly of implicit learning rather than explicit, declarative learning -- or so it seems to me.)
Interesting. Now I'm thinking about my own journey.
Having been raised Christian... and in a very evangelical, pentecostal tradition no less... I wasn't exactly encouraged to think like this early on... great swathes of things were simply to be taken on faith, and doubt simply referred over to the appropriately doctrinal section of apologetics. It did not help that both of my parents hold bachelor's degrees in Christian theology.
Nonetheless... they couldn't entirely shield me from the world. The start of my journey, so far as I can recall, came while studying physics... which I did before high school, incidentally, as I was always a voracious reader. I read about things like Einstein's Theory of Relativity, and Heisenberg's Uncertainty Principle... and after thinking about the implications of them, I realized that I was confused.
"Hang on, this can't be right. God is omniscient, and thus has knowledge of everything, everywhere in the universe, irregardless of such pettiness as the speed of light being the maximum speed for transfer of information. The tools of Science may not be able to determine both the exact position and momentum of a particle, but God must know!"
Of course, my initial reaction was wrong. But I'd spotted a problem in my worldview, and additional study of science seemed to reinforce my nagging doubts. I didn't have a word to explain it, then... indeed, I didn't have the words to fully articulate the problem until I read Atlas Shrugged in college, and thereby learned of the law of non-contradiction. But it was seeing that contradiction that first got me thinking in terms of "either my religious beliefs are true, or a large body of experimentally verified science is true, but not both"... and having realized that, it left me open to changing my mind on the subject later on. Which, fortunately, I did.
The most common Christian answer to that contradiction, when translated into modern parlance, is that God is the hardware on which the universe runs. Not only can he know both the position and speed of a particle at any given time, but he, in fact, must know it at all times or it would cease to exist.
The fact that some philosophers could figure this out over a thousand years ago is impressive. The fact that the majority of "believers" just blink in incomprehension and then go right on thinking of God as just a slightly mutated human who lives in the sky is disheartening. Especially now that we routinely fly above the blue and know that what's "up there," in the physical sense, is just more sky.
The first little heuristic trick toward mental health. It seems to me that children actually used to be taught such things, and weren't left to bring themselves up like wolf boys.
For some reason, this article had my mind wandering to "My Favorite Things". Turns out, there was a reason. Think Julie Andrews, and sing along:
When the dog bites, when the bee stings When I'm feeling sad, I simply remember my favorite things and then I don't feeel so bad!
Weren't children's stories full of such things, once upon a time? Every story had some lesson to be learned. I don't have kids, but my impression is that every story these days aims at imparting a proper attitude, not a useful skill.
Two of the three little pigs got eaten. The grasshopper starved to death. Little Red Ridinghood and her grandmother both got eaten with no miraculous rescue. The boy who cried wolf got eaten, along with all his sheep. The little mermaid didn't get the prince and was cursed to walk the world in agony for the rest of her days. Several other stories, the central "villain" does something wrong (or maybe even just rude or inconsiderate) and the protagonist of the story kills them and all their family and burns their house down.
The stories these days are overly-worried about not scaring children with the fact that the world is a dangerous place and one mistake can be the end of you and everything you care about. The old versions very much wanted to drive that point home.
As a society we've prioritized "feeling safe" over "being safe" when it comes to raising children. Isn't that scary?
I have a similar story.
When I was a senior in highschool, I took microeconomics. Before senior year I never tried in school. Senior year I started to care about intellectual things and started trying. I hated my teacher and openly read the textbook in class instead of listening to her lectures.
When it came time for finals, I had a good grade in the class, some sort of high A, and had already been accepted to college. I figured out that if I don't do the final project, my grade would be a B.
I told this to my teacher. She was so angry at me and said she'd fail me. I said she can't, at least not without going against the grading rubric she had established in our syllabus.
I explained to her that I was frustrated because she was missing the most important concept in all of economics: incentives! How can she design the incentives and then be mad at me for acting accordingly? What kind of economist does that? Moreover, the whole system is set up with bad incentives. She should thank me for exposing them, recognize that I've grasped the important concepts, and give me an A.
At least that's the story I like to tell myself. And other people! It's not the truth though. The truth is I have major public speaking anxiety and didn't want to do the public speaking the final project would entail. That's all it was. I don't even think I grasped the centrality of incentives at the time. I think I learned about that about a year later once I discovered LessWrong.
Well, even that paragraph isn't really the truth, but it's something I tell people sometimes too. The reason I tell people these untrue stories is when I figure the truth doesn't matter much and because the story is just meant to convey a point, not be a description of who I am as a person. This comment itself is probably being made less clear and more awkward by this last paragraph, and I considered not including it for that reason. But things like that are a bit of a slippery slope. I noticed myself starting to believe the fake stories to some extent, and that's probably not worth the price of telling a less awkward story to people who I don't care about.
Based on this one comment I wonder whether you found a way to get round the uncomfortable public speaking thing. Whatever the case, your writing is refreshing, and interesting to read!
That's really awesome to hear, I appreciate the compliment! I don't think I ever got over the public speaking anxiety though. I say "don't think" because it's been such a long time since I've had to do public speaking, so it's possible that it has gone away, but I doubt it.
This sort of thing intrigues me. It underlines for me that our "technology of the mind" is still very young and unstructured.
Imagine what would happen if this one skill were explicitly taught in schools?
The hard part is that it's one of those mental skills that can't really be taught. You can tell people about it, but they have to learn it for themselves. Because, even once you know about it intellectually, what it "feels" like when your brain is deliberately not thinking about something is almost certainly a subjective experience that will be different for everyone.
So, like Zen, you'd have to work out a large set of training scenarios that put a person in a situation where it'll happen and then draw their attention to it, and plan on having to run most people through quite a few of them before they grok.
I remember the exact moment when I began my journey as a rationalist.
It was not while reading Surely You’re Joking, Mr. Feynman or any existing work upon rationality; for these I simply accepted as obvious. The journey begins when you see a great flaw in your existing art, and discover a drive to improve, to create new skills beyond the helpful but inadequate ones you found in books.
In the last moments of my first life, I was fifteen years old, and rehearsing a pleasantly self-righteous memory of a time when I was much younger. My memories this far back are vague; I have a mental image, but I don’t remember how old I was exactly. I think I was six or seven, and that the original event happened during summer camp.
What happened originally was that a camp counselor, a teenage male, got us much younger boys to form a line, and proposed the following game: the boy at the end of the line would crawl through our legs, and we would spank him as he went past, and then it would be the turn of the next eight-year-old boy at the end of the line. (Maybe it’s just that I’ve lost my youthful innocence, but I can’t help but wonder . . .) I refused to play this game, and was told to go sit in the corner.
This memory—of refusing to spank and be spanked—came to symbolize to me that even at this very early age I had refused to take joy in hurting others. That I would not purchase a spank on another’s butt, at the price of a spank on my own; would not pay in hurt for the opportunity to inflict hurt. I had refused to play a negative-sum game.
And then, at the age of fifteen, I suddenly realized that it wasn’t true. I hadn’t refused out of a principled stand against negative-sum games. I found out about the Prisoner’s Dilemma pretty early in life, but not at the age of seven. I’d refused simply because I didn’t want to get hurt, and standing in the corner was an acceptable price to pay for not getting hurt.
More importantly, I realized that I had always known this—that the real memory had always been lurking in a corner of my mind, my mental eye glancing at it for a fraction of a second and then looking away.
In my very first step along the Way, I caught the feeling—generalized over the subjective experience—and said, “So that’s what it feels like to shove an unwanted truth into the corner of my mind! Now I’m going to notice every time I do that, and clean out all my corners!”
This discipline I named singlethink, after Orwell’s doublethink. In doublethink, you forget, and then forget you have forgotten. In singlethink, you notice you are forgetting, and then you remember. You hold only a single non-contradictory thought in your mind at once.
“Singlethink” was the first new rationalist skill I created, which I had not read about in books. I doubt that it is original in the sense of academic priority, but this is thankfully not required.
Oh, and my fifteen-year-old self liked to name things.
The terrifying depths of the confirmation bias go on and on. Not forever, for the brain is of finite complexity, but long enough that it feels like forever. You keep on discovering (or reading about) new mechanisms by which your brain shoves things out of the way.
But my young self swept out quite a few corners with that first broom.