(The Exercise Prize series of posts is the Center for Applied Rationality asking for help inventing exercises that can teach cognitive skills.  The difficulty is coming up with exercises interesting enough, with a high enough hedonic return, that people actually do them and remember them; this often involves standing up and performing actions, or interacting with other people, not just working alone with an exercise booklet and a pencil.  We offer prizes of $50 for any suggestion we decide to test, and $500 for any suggestion we decide to adopt.  This prize also extends to LW meetup activities and good ideas for verifying that a skill has been acquired.  See here for details.)

Exercise Prize:  Be Specific

During YCombinator's Startup School 2011, Paul Graham and Harj Tagger did "office hours" onstage.  One pair of entrepreneurs were doing a matchmaking (dating) startup, and Paul and Harj were trying to figure out what their startup did, exactly - for example, what their startup could do that the existing low-tech solution couldn't.  (Video.)

Harj:  Low-tech like, you know, just like word of mouth, telling someone "hey, you should like, meet up with my friend" or "we're getting drinks, why don't you come along?" Like, what can the software do that's specifically better than that?

Entrepreneur:  I think that our software specifically is providing the better connections for people, um...

Paul: Providing the better connections for people...?

Entrepreneur:  I mean, one way you can think about it, I don't know if this is the right answer, but... there's a lot of things that are happening in real life that they're trying to mimic online, maybe that's not the correct way to...  Look at it like this: to give them an online tool to also do this, like they're already doing in real life, maybe they could reach, uh expand their reach through the online website.

This had been happening with most of the startups Paul and Harj were interrogating - they just could not seem to provide a customer use-case - and I couldn't stand it any more; which is why at this point I whispered audibly enough for a few nearby people to hear, "Be specific!  Be specific!"

A moment later, on stage:

Paul:  Hm.  Not very specific.

I got some strange looks from the people sitting next to me.

I hope this provides some background for my guess that around half of Paul Graham's advantage is based on years of incubator experience, and the other half is unusual rationality skills of the sort that the Center for Modern Rationality is trying to figure out how to teach.  Obviously this is only a very rough conjecture.  But you can see the basis for the hope that - after a fair amount more work - we'll be able to offer a 2-day course for YCombinator entrepreneurs that eliminates 50% of the overhead from their conversations with Paul Graham.

(Also, note how this post starts off with a specific example - an instance of the concrete-abstract writing pattern in which you state the example first and the generalization afterward.  This is one of the most common bits of nonfiction writing advice I dispense:  "Open with the concrete example, not the abstract explanation!")

Theoretical background:

S. I. Hayakawa once gave this illustration of the "ladder of abstraction", and in particular, the difference between going up or down:

"What is meant by the word red?"
"It's a color."
"What's a color?"
"Why, it's a quality things have."
"What's a quality?"


"What is meant by the word red?"
"Well, the next time you see some cars stopped at an intersection, look at the traffic light facing them.  Also, you might go to the fire department and see how their trucks are painted."

"Red is a color" is moving up the ladder; "color" is a supercategory of red.  All things which are red, have colors; but not all things which have colors, are red.  And similarly, if you look at a specific firetruck, that firetruck is a red thing, but there are also many other red things which are not that firetruck.

What is true of one apple may not be true of another apple; suppose apple1 weighs 100 grams and is slightly green in some places, and apple2 weighs 200 grams and is entirely dark-red.  You can say more truths about apple2, like "apple2 is dark red", then you can say that is true of all apples.  (For more on this point see The Virtue of Narrowness.)

Thus, it may be easier to mentally picture "a firetruck" than "something red" - "firetruck" describes a narrower section of Thingspace, so you're less likely to get lost along the way.

S. I. Hayakawa called this the ladder of abstraction.  I'm not sure if understanding the following section will really help with the skill of Being Specific, or help anyone construct exercises for the skill of being specific.  But a better theoretical understanding does sometimes prove useful.  So I will now digress to explain that abstraction isn't really a ladder, but a lattice.

Let's illustrate this using a classic example from the field of machine learning.  Suppose that Days have three properties:

  • Weather: {Sunny, Cloudy, Rainy}
  • Temperature: {Cool, Hot}
  • Timing: {Weekday, Weekend}

And suppose that we've been given some examples of Days on which it was good, or alternatively bad, to play tennis.  For example, the Day {Sunny, Cool, Weekend} was good for playing tennis, but the day {Rainy, Hot, Weekday} was bad for playing tennis.  A classic task in machine learning is to induct, from a set of pre-classified examples like these, a rule describing when it is good to play tennis.

Any proposed rule which can classify all days as good or bad is a concept, in the lingo of machine learning.  "Sunny Days" is a concept; likewise "Sunny Cool Days", and "Days which are either Cool or Sunny".  Each of these is a concept which classifies all 12 possible days either positively or negatively - instances or non-instances of the concept.

There are 212 possible concepts over the 12 possible Days.  Why so many?  Because - for example - there's a concept which only includes the two Days {Sunny+Cool+Weekday} and {Cloudy+Cool+Weekend}}, but classifies all other Days as noninstances.  This is a way of classifying all Days into instances or noninstances, hence a possible concept.  It's not a compact concept, but it's a concept.  Each Day can be classified either positively or negatively - one binary decision per Day - so 212 possible concepts.  (That's why induction is a difficult problem in machine learning.)

The concept "Sunny" is a superconcept of "Sunny and Cool"; it lies above it in the lattice of abstraction, since all days which are "Sunny and Cool" are "Sunny".  "Sunny or Hot" is a supercategory of "Sunny".  "Weekend" is neither a superconcept nor a subconcept of "Sunny".

Concepts form a directed lattice from most general to most specific, with "all Days" at the top (every Day classified as an instance) and "no Days" at the bottom (the concept which classifies every Day as a noninstance).

If you now go back to the problem of telling someone what "red" means, when you say "red is a color", then, even if the listener does happen to know what "color" means, you're still moving upward in the lattice of abstraction.  When you said "color", you were talking about a concept that included all red things, but also many other things that were not red.

"Our software is providing the better connections for people" - the entrepreneur who said that might have had something specific in mind, or they might have just been bluffing or succumbing to wishful thinking.  But they described it using an abstract statement so broad that it included Facebook, or Western Union back when they were sending telegrams.  They might - though this is somewhat optimistic - they might have known themselves what they had in mind; they didn't think of Facebook; so they didn't realize how many other possibilities fit their words.  This is a classic manifestation of the Illusion of Transparency, and it's why we have to keep telling people to navigate the lattice downward.

The skill of Being Specific is the skill of understanding how to navigate the lattice of abstraction.  You can see why this would be a key element of cognition on a par with Bayes's Theorem or consequentialism.

And this is true in practice as well as theory.  When I'm talking to anyone outside the local LW community, I find that a very large amount of my conversation involves repeatedly asking them to be more specific - and if you think that's just me being annoying, watch Paul Graham in the video.

A closely related skill is concreteness, which has to do with nearness-to-sensory-experience or actionability.

According to David Allen's "Getting Things Done", for your brain to stop thinking about an unfinished task, you must (1) know and trust that an external system will remind you to perform that task when it is time to perform it, and (2) have chosen the next action taken at a sufficiently concrete level that your brain is no longer trying to plan it out in the background.  "Contact Luke about dispersing prize awards" is not a sufficiently concrete to-do; it leaves open the question of whether to phone or email, and what exactly to say.  "Read through the comments, gather the LessWrong usernames of everyone who made a suggestion we tried or adopted, and email the list to Luke" is an action item I know how to perform straightforwardly, without my brain trying to plan it in the background.  When you have a trustworthy external system to remind you of what to do, at the time you need to do it - so that the back of your mind isn't worrying about remembering to check the to-do list - and all to-do items have been concretized to the point of being executable without further background planning - then you have, in GTD parlance, "gotten to zero", a state of pure mental blissfulness in which your brain is not worrying about anything except what you're doing right now.

Similarly, for a statement like "Wulky Wilkinsen is a post-utopian" or "Earth gravity pulls at 9.8 meters per second squared" to be falsifiable, it must be concretized - rendered near-to-experience - to a sufficient degree that you can potentially see something and say "Oh, guess the hypothesis was wrong"; you must be able to have an experience which the concretized statement constrains, and which falsifies the theory if the experience is out-of-bounds.

Theoretically:  If you imagine the universe as a huge directed graph of causes and effects - the Great Web of Causality - then "concreteness" is being near enough in the Web to either your sensory inputs or motor outputs that you can directly see the prediction unfold, or directly implement the plan, without much further thought.

"Be Specific" and "Be Concrete" could easily end up being the same unit - they're closely related - and we're happy to entertain exercises for Being Concrete, as well as Being Specific.  Visualizing what your customer literally sees or does after navigating to your site, would've been a good first step toward being able to answer many of Paul Graham's questions.

A possible success criterion:

One question that we spent a lot of time discussing at CMR, was translating our sense of "specific enough" or "concrete enough" into a describable criterion.  (Instead of just a wordless intuition for when something is "too abstract".)

There was an exchange in Paul Graham's office hours that went like this, while interviewing a startup that did metrics - analyzing pageviews, roughly - and the entrepreneur was having great trouble describing what they did that MixPanel didn't.  It went on for a while.  It was painful to watch.

Paul:  I don't get what the difference is.  I still don't get what the difference is.  What's the difference between you and MixPanel?

Entrepreneur:  The difference is - when you have to supplement - they're a view company and we're a platform.  That's what it comes down to.  They're like a view, a reporting company.  If you need something they don't have, a feature - 

Harj:  So what's an example of somewhere you'd use your thing over MixPanel?  Can you give a use-case?

Entrepreneur:  Yeah, I mean, we had revenue on day zero. There's a good reason for um... it's a start up, it's a series A company in the daily deals space.  One we've signed a social game company to -

Harj:  And why do they prefer your thing?

Paul:  That wasn't what Harj was asking.

The problem (from the perspective of our present discussion) is that the Entrepreneur did not understand that Paul and Harj were repeatedly asking him to move downward on the ladder of abstraction.  When the Entrepreneur said "We had revenue on day zero", he was trying to offer confirmation of the abstract statement "We can do things MixPanel can't", but Paul and Harj still had no idea what his startup actually did.[1]

A quick bit of theoretical background:  There's an important difference, in the field of mathematical logic, between models and axioms.  An axiom is something like "All kittens are cute", i.e. "All x: kitten(x)->cute(x)".  A model is a particular universe of objects that includes {Obj #19834, kitten: T, cute: T, color: grey} and {Obj #19835, kitten: F, cute: F, color: striped}, and so on.

Correspondingly, in logical inference, there's a distinction between model-checking and deduction.  Suppose you want to know whether it's true that all positive integers less than 5, when multiplied by 7, are less than 50.  If you prove the general truth that all integers less than 5, times 7, are less than 35, by manipulating the axioms of multiplication and inequality, that's deduction.  If you notice that the only positive integers less than 5 are just {1, 2, 3, 4} and enumerate their products {7, 14, 21, 28}, which are all less than 50, that's model-checking.

My hypothesis about what it means to be "specific enough" or "concrete enough" is that the picture painted is detailed enough to use in model-checking whatever points are being debated.  Paul and Harj don't want to trust you when you state the abstract generalization, "We're better than MixPanel".  They aren't even content with deducing support for this generalization from the further generalization, "We already have customers."  They want a picture of something you do that MixPanel doesn't, which is detailed enough that they can model-check whether you have a competitive advantage.

Not to mention that Paul Graham is probably thinking about a number of other questions:

  • How much would I pay for this product?
  • Is this startup exciting enough that I would tweet about using it?
  • How much resources will it take to develop these features further?

Paul Graham doesn't want you to say, "$50, yes, and twenty engineer-months".  He wants a sufficiently specific picture of (a customer using) your product that he can arrive at his own answers by model-checking.

If Paul Graham is reading this, he's welcome to contradict my interpretation of what was going on in that particular session - but it did seem like a very nice concrete illustration.

That's my guess for what often constitutes "specific enough" - though I'm not sure that's the only thing that ever determines specific-enoughness.

[1]:  The strange part was, near the end of that session, it started to look like this might be an interesting startup; that the Entrepreneur wasn't just bluffing.  Their actual use-case was to let customers easily roll their own code to measure, e.g., the page-viewing behavior of only customers who'd bought more than $200 worth of stuff, which allegedly MixPanel wouldn't let you do.  Which would've been a perfectly good answer if the Entrepreneur had given it at the start of the session, instead of the whole session being about Paul and Harj trying to get at that information.

Five-second-level skill:

The 5SL skill for this problem requires:

  • Trigger:  Recognizing when your words or thoughts are too abstract.
  • Action:  Moving downward in the abstraction lattice, or moving nearer to sense input or motor output; being able to render your thoughts more specific or more concrete.

Both of these are targetable for exercises.

Pain points & Pluses:

• You want Paul Graham to believe your startup is better than MixPanel.  So you say, "My startup is better than MixPanel" - just produce the pure abstract conclusion you want Paul Graham to arrive at.  You keep trying to convince Paul Graham of this statement, saying that you have customers or that you have venture capital, but never actually move downward to the level where Paul Graham could arrive at this conclusion by model-checking.

• You want to describe what your software does, so you say it makes connections between people.  You have something specific in mind, but the words coming out of your mouth are so general that - although you're not thinking of those other cases - they could apply equally well to Facebook or telegraph lines.  Paul Graham has no idea at all what you're trying to describe and is giving you blank looks.

• The worse version - and the reason why Paul Graham doesn't just trust you, even if he thinks you're honest - is the case where you yourself want to believe your startup is better than Facebook, but you can't think of any specific thing your startup does better than Facebook, so you think of other abstract generalizations that seem to support the conclusion, like "We have smarter people" or "We got more funding earlier."  Where fuzzy thinking is motivated, overly abstract thinking is motivated.

• Abstract words can also avoid emotion.  George Orwell:  "Defenceless villages are bombarded from the air, the inhabitants driven out into the countryside, the cattle machine-gunned, the huts set on fire with incendiary bullets: this is called pacification."  Or contrast "Humanity is awful, it'd be better for the planet if we all died" to "Everyone including my little sister is awful, we'd be better off if everyone died including her."  To feel sympathy, we need enough concrete detail that our emotions can model-check the picture and be activated.

• Cognitive-behavioral therapy is the big experimentally supported version of therapy, for anyone not aware of this, bearing very little resemblance to anything Freudian.  CBT talks about using requests for specific details to interrupt thoughts looping around vague but affectively laden centers, like "I am a good husband", "I am a bad husband", or "my roommate is a slob".  How are you a good husband?  How are you a bad husband?  Which specific feature of your roommate are you objecting to?  Taboo the emotionally valent word at the center, like "slob", and replace it with something that's specific enough to be testable, or concrete enough to be acted upon.

•• Contrast also "It bothers me when you leave soda cans on the table" vs. "You're such a slob, stop being such a slob."  Or contrast:  "I'm upset" -> "I'm upset because I think the other person is looking down on me" -> "I'm upset because the person's tone of voice sounds like people who looked down on me in high school".  This is related to the incredibly important skill, search for the historical causes of your thoughts, rather than their justifications.

• Focusing on the specific details of a concrete example, instead of repeating a word or arguing about a category, can interrupt Sneaking in Connotations and Arguing By Definition.

• All the failures of concreteness warned against in the Mysterious Answers sequence, where you go on and on about how Wulky Wilkinsen is a post-utopian without ever once asking or imagining how the world ought to look, and what you yourself should experience, if that were true or alternatively false.

• Visualizing specific examples often improves quality of thought in general - we're often smarter when we're using both model-checking and deduction, visualizing a picture of what we're supposed to be reasoning about, constantly checking our deductive steps against some specific model those deductions are supposed to be true about.  Saith Richard Feynman:

I had a scheme, which I still use today when somebody is explaining something that I'm trying to understand: I keep making up examples. For instance, the mathematicians would come in with a terrific theorem, and they're all excited. As they're telling me the conditions of the theorem, I construct something which fits all the conditions. You know, you have a set (one ball) - disjoint (two halls). Then the balls turn colors, grow hairs, or whatever, in my head as they put more conditions on. Finally they state the theorem, which is some dumb thing about the ball which isn't true for my hairy green ball thing, so I say, "False!"

 If it's true, they get all excited, and I let them go on for a while. Then I point out my counterexample.

"Oh. We forgot to tell you that it's Class 2 Hausdorff homomorphic."

"Well, then," I say, "It's trivial! It's trivial!"

• Being specific helps notice and call bluffs, should you be mischievously inclined.

"Beware, demon!" he intoned hollowly.  "I am not without defenses."
"Oh yeah?  Name three."
-- Robert Asprin, Another Fine Myth

Wannabe executive:  "I will improve communications between employees and management."
Me:  "Can you give me a specific example of how you would do that?"

Known exercises for this skill:

In our previous Rationality Camps, Anna found that her attempt to teach a unit on "Being Specific" didn't seem to work.  Her central exercise was picking a category and asking people to name examples.

This isn't to say that the Camps were unsuccessful at teaching the skill.  Attendees picked it up, not from the explicit unit, but from all the instructors having to repeatedly ask the attendees to be more specific, and then having to ask them again, while being specific themselves, until the attendees picked up the rhythm by example and feedback.

Given our present teaching technology, this skill seems transmissible from master to apprentice, but not yet replicable by exercises.  That's why we're turning it over to you.

New Comment
295 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

her attempt to teach a unit on "Being Specific" didn't seem to work.

How specifically did it not work?

(ETA: I should probably add I'm not being mischievous here; "doesn't work" is a trigger phrase for me, born out of extensive experience of dealing with useless bug reports. It systematically unpacks into at least two questions, "what behavior were you expecting" and "what did you get instead".)

An example of this that will be familiar to any programmer, and was taught to me in grade school, is "give orders to a malicious idiot." The teacher has the students write down the algorithm for a simple task, like "sharpen a pencil," with a wooden pencil and an old crank-operated sharpener as the props.

Typically, people begin with something like "stick the pencil into the sharpener, then turn the crank," which the teacher will do by ineffectually pushing the side of the pencil against the sharpener while turning the crank. The students revise to "stick the end of the pencil into the hole in the sharpener, then turn the crank," which the teacher will do by sticking the eraser into sharpener. (There are, if I remember correctly, four or five different features you can require the pencil-sharpening algorithm have, like which end of the pencil to stick into what part of the sharpener, which way to turn the crank, to hold the pencil still so it doesn't just spin with the crank or fall out if the sharpener is oriented poorly.)

(This will be familiar to programmers because going from the basic algorithm to code requires a level of detail that can't be faked.)

I was reminded of something similar by AspiringKnitter's post below. There is an event in Science Olympiad called Write It Do It. One person is given a constructed object made out of LEGO, K'Nex, or similar. They write a set of instructions for how to reproduce the object. These are then given to a teammate who hasn't seen the original object, who must use the instructions to reconstruct the original object. Seems fairly simple to adapt to a group setting - you could just split the group into two rooms and have them first write their own instructions and then try to follow the instructions of a partner in the other room.

This exercise and malicious idiot exercise differ in the "when" and "by whom". With a malicious idiot, your errors are pointed out immediately and by somebody else. When writing instructions, your errors don't come to light until your partner's object doesn't look like yours, and neither of you might notice until that point. It's important to notice a lack of specificity both in others (so they don't lead you astray) and in yourself (so you don't lead yourself astray), so it would probably be useful to do both kinds of exercises.

There's a lower-overhead version of the LEGO exercise involving pen and paper: person A draws a design on a piece of paper and hands it to person B, who writes instructions for how to reproduce that shape and hands them to person C, who follows them. Then compare A's output to C's.

Naturally, this can be done in parallel with N people, all of whom start out as As and end up as Cs.

Of course, this kind of depends on A not knowing what's coming, since otherwise A just draws a circle or something.

This game is particularly fun when chained; A draws, B describes, C draws, D describes, and so on. Then you see how the shape transformed over time.
There's actually an online game called "Doodle or Die" for playing this. It being an online game, however, there are a disgustingly large number of players who break the chain (willfully or non).
We've played this at meetups a few times. It hammered in the illusion of transparency pretty well. (Puppy Trampoline -> Drawing -> If you jump on a dog you make it stronger).
I remember that! I think the biggest obstacle to clarity in the game is actually the rarity of artistic skill, not the vagueness of the written descriptions, though.
I've played a similar game in person - I think it was Telestrations. You get a word from a stack of cards, and try to draw that word. The next player guesses which word you were trying to draw, and the next player tries to draw that word (and so on). Fun party game.
Except that the aim of telephone pictionary is to produce hilariously incongruous lists of phrases and pictures, and the aim of this game is, well, the opposite. Erm... posted this in the wrong thread, then "retracted" it -didn't actually know what that button did. Oh well...
If you reload, you can delete a retracted comment.
This is the party game called "Eat Poop You Cat" (pronounced "I'pupiukat") or "Telephone Pictionary".
Except that the aim of telephone pictionary is to produce hilariously incongruous lists of phrases and pictures, and the aim of this game is, well, the opposite.

Once a year, an acquaintance of mine gets his first-year programming class to tell him how to make a peanut butter and jelly sandwich. Even more knobs. :)

The best question I ever encountered during an interview for a Technical Support position was to describe either that or tying your shoes. It's a great test of whether a prospective employee will be able to actually communicate troubleshooting concepts to the caller on the other end of the line, since obviously they can't use anything but words to do so :)

I think this is a great idea! One addition I think would be useful is that (after a demo), have people get into small groups and take turns being the "malicious idiot" (instead of just the teacher playing this role). This will allow them to think of the issue from the OTHER side. (and be more kinetically interactive)

Darn it, this was the first thing I thought of, and now I can't get any credit for it! See also: How to cook scrambled eggs
... Then the malicious idiot stabs you in the eye with the pencil. Oh, the malicious idiot was supposed to follow orders and only follow orders? Why didn't you say so?!
Because I love setting other people up for jokes.
I guess the malicious idiot is not suppossed to be creative, but lazy. They should use the simplest possible explanation -- only the simplicity is not measured by common sense, but by something like Solomonoff prior.
No, it's maliciousness, but very specifically aimed maliciousness. They don't want to hurt you, they just want to demonstrate that you are bad at giving directions.
I don't think this a good restriction. Consider the fact that Hanlon's Razor is even a thing: This suggests that people often mistake stupidity for malice. So given that in these examples, your opponent probably does secretly understand what you're communicating (most of us know deep down how to sharpen a pencil), it might be necessary to have malice/creativity play the part of inferential distance. Otherwise you may learn to anticipate an unrealistically rational audience, one which never comes in with incorrect preconceived ideas, or lacks the necessary technical vocabulary, or seems to practice selective hearing, etc. In short, original seeing is the exception, not the rule, so the opponent should be at least slightly hostile in his/her interpretations to account for this.

Idea One: Monday/Tuesday Game

On Monday, your proposition is true. On Tuesday, your proposition is false. Tell me a story about each of the days so I can see how they are different. Don't just list the differences (because you're already not doing that well). Start with "I wake up" so you start concrete and move on in that vein, naming the parts of your day that are identical as well as those that are different.

Idea Two: Sabotage Game

You're definitely right, but unfortunately there's a malevolent actor who wants to make you look a fool. And even more unfortunately, he's creative and has a lot of resources. What does he sabotage? What evidence does he counterfeit? (This way you're identifying the examples or the proofs by looking at them as structurally crucial.)

I really like this and wish I'd seen it earlier. Good idea.

Idea: train the skill "ask for examples" instead, which seems easier to train, and bootstraps you into being more specific. I am not usually confused about my own thoughts, but I am often confused when others are trying to explain their ideas.

Example: explaining a startup idea to my friend today, I said "...and the viral strategy for this payments app could be loaning money to people". He was confused and asked for an example, so I said "let's say we are at a coffee shop. I have no cash, you have the payment app, and I want to borrow money for coffee. You tell me to download the app, so you can loan me the money. Then I can get my coffee, but to pay you back, I have to complete the signup process." -- here, I was incorrectly assuming my friend had all the same context I did with respect to viral strategies.

Exercise: after explaining the virtues of asking for examples, start to move onto another topic. A confederate in the audience yells "can you give an example?" Everyone giggles, the instructor says "I'm glad you asked!" and the exercise is explained: Students pair off and start telling stories to each other, intentionally leaving ... (read more)

A friend of mine makes these stickers:

At the risk of escalating the Meta War, I think "be specific" and "be concrete" are themselves too general and abstract to engender good exercises. They look more like "do algebra" than "factor a polynomial". Not that you wouldn't get some interesting responses if you said, "We need ideas for teaching how to do algebra," but most of them probably wouldn't make students better at factoring a polynomial-- analogously, I like the "teach me to sharpen a pencil" game, and it would make a fun and striking activity, but I'm not sure it would help students learn to explain their business plan better in an interview. If you want students to communicate judgements and opinions better, teach them to do that.

In this case, I would unpack "specific" into two parts: concrete and relevant. To make a statement more concrete, you talk about how qualities can be measured or observed ("Yellow is a color" becomes "Yellow is the color of a dandelion" or "yellow is the color of the emission spectrum of sodium"). To make it more relevant, you relate it to a goal or higher-level question ("These scissor... (read more)

Argh. I'm reminding myself that Retroactive Rewards Rage is a cognitive fallacy. Is there a formal name for it? I bet you could induce it in chimps.


Abstraction Telephone

Divide into at least 4 groups, of minimum size 1 and maximum size maybe 5. Each group gets a different short passage. They collaborate to translate the passage their choice of either "one rung up," making it all more abstract, or "one rung down," making it all more specific. Group N then passes their translated passage only, not the original, to Group N+1 modulo the number of groups. Then each group performs the same operation, then passes it to the next group in line. I think two iterations will be enough to get something entertainingly mangled, so then each group in turn performs the passage they've been handed for the audience. The remaining people may try to guess what the original passage was.

Example (2/3rds stolen from George Orwell):

Group 1 gets:

"I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance hap... (read more)

6Eliezer Yudkowsky
We'll totally do retroactive awards for anything we try from Check Consequentialism.
Oh, thanks!

I think you should go with Vaniver's idea. (Edit: Vaniver now has multiple ideas up. I mean the one about giving orders to malicious idiots. Completely off-topic: that's also a useful way to explain tasks to people with Asperger's Syndrome or other neurological oddities that cause executive dysfunction.)

I also think this reminds me of something (fiction) writers talk about a lot: they've hit on the way people won't sympathize with "a billion people died/starved/were tortured/experienced dust specks in their eyes" but will sympathize with "Alice was mobbed by dust specks and blinded" and will sympathize even better if you give some specific details about how it felt. And then they go on to talk about how to make Alice someone the reader cares about and how to craft sentences and other stuff that's relevant to them but irrelevant here.

But maybe something like making up a character and talking someone through xyr experience using the product step by step, in the kind of detail a novelist would use to describe the climactic fight scene.

Another idea that occurred to me is some sort of exercise where two people would pair up. One would have to do a novel task or navig... (read more)

Upvoted for that last paragraph.
Yeah, seconding the "blind obstacle course" exercise, although you don't usually need to make it more difficult by not letting the person giving instructions watch (mostly because it's difficult just to walk in a straight line without visual cues, let alone execute precise turns). It's a common leadership/working as a group game and people usually need to watch other attempts go wrong 3 or 4 times before hitting a useful level of specificity.
(Summary: Orienteering with the navigation and movement separate. This exercise requires both people to be specific to not get lost, and it can be extended by adding in a race aspect (trying to be specific under pressure!).) The last paragraph made me think of "MoboGoGlobo", which is an orienteering event where there are two participants: one doing the navigation who guides the other via phone. (I'm not sure how familiar with orienteering many people are, so I'll give a quick intro) When participating in an orienteering event one normally has to visit a series of markers in a predefined order as quickly as possible (or not, if one isn't feeling like racing, entirely up to the participant). One has a detailed map (although, importantly, it rarely has street names) that indicates the location of these and the order to visit them in, e.g. the two maps here (the pink (or purple) triangle is the start location and the numbered circles are the markers). When competing, one uses all sorts of clues to make sure one is going in the right direction and one isn't lost (and to become unlost), like most obviously the shape of nearby buildings, the topography (e.g. a steep hill), a fork in a track or stream, or more subtly things, like a bend in a track or the position of a power line on the next hill. (Conventionally, one also has a compass, which one uses to orient oneself correctly.) In this exercise you need two people: one with the map ("navigator"), the other actually on the territory that corresponds to the map ("runner", this doesn't imply that one needs to run though): it requires the navigator to describe exactly where to go ("go past the building" < "go to the left of the building that has a round canopy outside"), and also for the runner to describe what they see so that the navigator can keep track ("There is a line of trees" < "A line of trees starts just to my left and goes directly away"). There are multiple levels of specificity too: the best teams will have
I typically use a permalink to refer to comments that aren't upthread. (Thanks for the recommendation, by the way!)

Exercise - filing bug reports. Have attendees use software that sucks (that they provide, e.g. on their mobile device, or that you provide, e.g. over the Web), or recall occasions where they used software that sucked. Ask them to describe the problem they encountered in enough detail that someone else can reproduce it, or deduce something useful such as a workaround.

"It needs to be about 20% cooler."

I, for one, am thrilled to see Eliezer writing another Sequence. :)

Also, in my quest to make these all relatable to improv games:

More Specific (it's not a very good video, sorry), is a game in which two people act out a scene, with the audience (or moderator) occasionally demanding that they be "More Specific"


B: Good job finishing your paper on time!
C: Yeah, I had the hardest time finding my references...
Audience: More specific!
C: My kid thought that my reference books would be better with all the pages ripped out, and I couldn't find the pages with my quotes

I'm thinking that I actually might have seen this game mentioned on LW before, but I didn't a quick search and didn't see it.

Spitball (not a vid) is a game in which "Players take a mundane object suggested by the audience and elaborately detail it", generally using "Yes, and..." technique.

An improv-y style warm-up game that I just thought up, upon which I bestow the name "Something". In this game, everyone gets in a circle. The starting person (or moderator) comes up with a sentence that has a lot of "somethings" and "things" in it. Each person makes it more and more specific.

1: Did you something the thing ... (read more)

Unrelated to the post, but I'm not sure where else to suggest rationality exercises. So I'd like to revive an idea I saw here a while back called 'What Did You See?' (I can't take any prize if it's selected because it's not mine). I think it would be a wonderful game for developing curiosity and noticing specifics. But above all its purpose is learning that you can learn, which I think even in the rationality community is an important lesson that helps to reignite the inquisitive spark.

At home there was a game that all the parents played with their children. It was called, What Did You See? Mara was about Dann’s age when she was first called into her father’s room one evening, where he sat in his big carved and coloured chair. He said to her, ‘And now we are going to play a game. What was the thing you liked best today?’

At first she chattered: ‘I played with my cousin . . . I was out with Shera in the garden . . . I made a stone house.’ And then he had said, ‘Tell me about the house.’ And she said, ‘I made a house of the stones that come from the river bed.’ And he said, ‘Now tell me about the stones.’ And she said, ‘They were mostly smooth stones, but some were sharp and had dif

... (read more)
I realized that the movie was "It's a Wonderful Life" within the first paragraph. Consider adjusting your estimates of readers. Also, an imdb review that gave me the plot of a movie would not be a good review. It would be a synopsis. That review told me that the acting was good and the story heartwarming. I don't think that it is a good subject for criticism of specificity.
Are you replying to the right comment?
No, no I'm not. A thousand pardons.

Exercise: incentivize both teacher and student participants in Vaniver's "malicious idiot" exercise. Give the student points when she is successfully more specific, and give the teacher points when he finds a new way to misinterpret the student instructions.

Example: how to brush your teeth?

S: hold your toothbrush

T: (picks up toothbrush with teeth) (1 point)

S : hold your toothbrush between your thumb and fingers of your right hand (1 point)

T: (makes a fist, puts toothbrush on outside of fingers) (1 point)

S: argh. Like this! (picks it up as example) (2 points for being concrete)

T: (mimics perfectly)

S: great, now... Brush your teeth. (laughter)

I could go on, but hopefully you get the point.

I think this is the best exercise posted so far. Unlike the other exercises, it can be explained to ordinary middle schoolers in 90 seconds, it engages both visual and tactile senses, it is competitive, it is devoid of perverse incentives, and it will usually be fun to play and funny to watch.
Not so, you're better of improving in small increments than doing the best you can immediately.
That depends on the scoring system. If the judge grade exponentially for better answers, then small increments are a loosing choice.

Exercise - fetch something from the kitchen. This happens all the time at home and drives me crazy. "Get me the dentist's papers, they're in the drawer." What drawer, in what room, what do the papers look like, are they in some sort of container... Reproducing this in a residential training setting opens up interesting possibilities. For instance, while giving attendees a tour or taking them from place to place tell them (separately) to pay attention to a specific item ("this is your target for the exercise after lunch") in a non-central location. The next day, pair people up, have them describe the target to each other, fetch targets, debrief.


Several comments mention guessing games. Here's my variant:

One "communicator", one or many askers.

The communicator starts by describing something in the most general terms they can think of.

Each round, the asker(s) can either ask a question, or guess. The question has to be about an attribute of the thing, not its name, and they can't say "Is it X?" or "What is it?". The communicator answers, vaguely if the question is vague, getting specific if they're backed into a corner.

When they guess, the game is over. If they guess wrong, they lose. Then you try to figure out what your next question should have been.

If they guess right, they win and get congratulated.

"It's a way of moving things from place to place."

"What kinds of things?"

"Oh, things you want moved."

"How big can these things get?"

"No more than six feet high, a couple feet wide, less than a foot long."

"Are they inert and durable?"


"Are they fragile?"

"Depends on your standards."

"Would they break if you dropped them from a foot?"


"From ten feet?"


"... (read more)

The answerer is being cooperative here though. This is how I would have answered: "It's a way of doing stuff to stuff" "What kinds of stuff?" "Stuff that's like some stuff but unlike other stuff" "How big is the stuff?" "Around the size of other stuff" "Is the stuff inert and durable?" "As much as that sort of stuff can be" ...
Yes, but sufficiently specific questions should be anle to corner any uncooperative answerer playing in good faith. You don't have to accept that answer to "how big," for example. You can ask, is the stuff organized into discrete items? If so, are they bigger than a cubic mile? If not, are they smaller than a cubic foot? And so on.
Very strongly reminds me of this as well. I wonder what sort of exercise might combine the two.

Exercise: Stay in the entrepreneurship domain and channel PG. Pretty much everyone has startup ideas, right? Actually apply PG's algorithm to the participants' real startup ideas.

(I think his algorithm is something like: do I understand the core idea? Is it something people want? How do you know? Is it something people will pay for? How do you know? What are the obvious flaws? Why are you the team to do this? Why is now a good time for this? Is it working? How do you know? What insights/surprises?)

Example: I love Magic the Gathering and I want to have a place for people to talk about how awesome the decks are--

T: be specific!

A site to talk Magic strategy. (+1 conciseness)

T: how do you know people want this?

Well, people love magic and talk about it all the time.

T: I'm not convinced.

Well, I talked to a famous magic player (+1 specificity) and he told me this story about how he invented this deck back in the day on the starcity forums, and he couldn't have done it without the community [lincoln note: this is totally made up], so we want to enable that to happen more. (too general, but the teacher doesn't press yet, instead noting the obvious flaws for later discussion)

T: and people w... (read more)

Company mission statements are notoriously abstract and might make a good starting place. If someone didn't know anything about a company and they went and read the mission statement, they probably wouldn't have a much better idea of what the company actually did.

For example, if (stereotypical) Grandpa asked you what Google was and you replied, "they organize the world’s information and make it universally accessible and useful" you probably wouldn't do much to help him understand what Google is (despite that being one of the best mission stateme... (read more)

One way sub-skill of being specific is learning to focus on details, instead of big picture.

For example, if a manager is vague they focus on the big picture and tell their employee "You need to stop arriving late."
If a manager is specific, they focus on details and tell their employee: "In the past week, you have been more than 15 minutes late two times. If you are running more than 10 minutes late you have to call. If you are more than 15 minutes late 3 times in a month, you will get written up, etc."

I think a fun way to teach how to ... (read more)

Exercise - refining descriptions.

Get three or four people sitting together. Place a large group of different items (30-50 small cheap plastic toys of the sort readily available in bulk from Oriental Trading Company, for instance) in the center of the group.

The exercise is to narrow down to a single toy by adding one detail at a time to a description. The description begins with the word "thingy," "toy," or a similarly vague word. Players take turns adding a single detail to the description, repeating it each time. (A detail is usually a... (read more)

A variation: Again, a large group of different items (I was thinking abstract images of colors and shapes, but toys is a good idea too) is visible to the group. One of the objects is selected and then each person writes a description that selects that object alone out of the set. The goal is to write the shortest description that can pick that particular object. Once everyone is done, answers are compared, and violations are sought: if one of the other objects fulfills all the requirements of someone's description, they are disqualified. Whoever has the shortest description that describes the chosen object and only the chosen object wins a point, and the game is repeated with another object.
This adds a competitive/fun element (looking for violations in other descriptions), may widen ideas on how to describe something, and trains conciseness as well as specificity.
(I like the idea, but I think the last few paragraphs could do with some editing: your meaning has been obscured :( )
Good grief, what happened to my post? Thanks for the heads up.
A second variation: Once again, a large group of different items is the set. Half the group leaves the room, and the other half selects one of the items. Each person left in the room writes the shortest description of the chosen item that selects it out of the group. The other members come back in and are each given one of the written descriptions and a stopwatch. They read the descriptions and then try to visually find the chosen item without giving obvious indications to the other searchers. Once a searcher thinks he knows which item is being described, he stops his stopwatch to indicate how long it took him. After every searcher believes they have found the correct item, the written descriptions and stopwatch times can be compared as above. This has the added bonus of providing information on what kind of descriptions are the most helpful in finding things.

Exercise: What Was That All About?

Players get samples of writing from various internet sources - randomly chosen movie reviews from IMDB, news stories from Huffington Post, blog posts from Wordpress, Wikipedia articles, etc.

Player A gets to block out 5% of the words in the sample. Player B then tries to guess the topic the sample discusses.

For example, here's a semi-randomly chosen IMDB review - the first one I grabbed off the site. It got 137 "helpful" votes out of 161 voters, so it's perceived as a good review. It's of a famous movie. I've bloc... (read more)

I realized that the movie was "It's a Wonderful Life" within the first paragraph. Consider adjusting your estimates of readers. Also, an imdb review that gave me the plot of a movie would not be a good review. It would be a synopsis. That review told me that the acting was good and the story heartwarming. I don't think that it is a good subject for criticism of specificity.
Thanks for letting me know you found it out so quickly. By specificity for the review, I didn't mean that it should summarize the plot. Instead, when some general statement is made, there should be some connection to the movie that supports it. Jimmy Stewart has boyish charm? When? What scenes? What about them? Contrast to Roger Ebert's review. An excerpt: This is a specific example supporting his statement at the beginning of the paragraph that, ""It's a Wonderful Life" is not just a heart-warming "message picture.""
This clears up your point wonderfully.

Why is "be specific" a hard skill to teach?

I think it is because being specific is not really the problem, and by labeling it as such we force ourselves into a dead-end which does not contain a solution to the real problem. The real problem is achieving communication. By 'achieving communication', I mean that concepts in one mind are reproduced with good fidelity in another. By good fidelity, I mean that 90% (arbitrary threshold) of assertions based on my model will be confirmed as true by yours.

There are many different ways that the fidelity c... (read more)

I suspect the Socratic method (the old one, not the bland one) fits under this heading- "put forth a proposition, and I'll demolish you with your own statements."
Sadly, "Communicate well" isn't quite as simple of a skill.

For moderately tech-savvy people who are not programmers: Write pseudocode to sort a list of numbers. Have a human, perhaps the moderator, execute the steps on a blackboard with a particular list. Repeat until you get the expected results. For extra credit, drill down on steps such as "exchange two numbers".

Ask for ways in which 2012 laws are better than 1912 laws. (Year is arbitrary.) Drill down on abstractions; for example, if "unions have more power" is given as an answer, ask for specific ways in which this is an improvement. For a... (read more)

Exercise: Interview (or "be Paul Graham")

Example: (participants A and B)
A: I am a gardener.
B: What is a gardener?
A: It's my job.
B: No, what does a gardener do?
A: I maintain gardens.
B: What is an example of a task that you have to do?


2 participants act in roles of the Interviewer and Interviewee. The Interviewer asks questions to be answered honestly, ideally that would tend towards abstract answers like "What do you do for a living?" and "What did you study in school?". The Interviewer repeatedly asks for mo... (read more)

When I did more job interviews, I was fond of the question "What does the job you'd most want here look like, in terms of what you'd actually be doing on a typical day?"
Inspired by people's failure to grasp at the obvious solutions to Harry's problem after Chapter 80, I posted an obvious exercise in case everyone's failed to notice it.
I posit that this is well more obvious than any other 'obvious' solutions to Harry's problem.
I'm not sure if that makes people more or less likely to notice it.
The "it is my job" response is a bit hard to interpret. Is the interviewee supposed to be intentionally obtuse? The interviewer's motivation is easy to maintain, but the interviewee's motivation might become challenging if this is just a conversation with the goal of specificity and external topic control. If I know that my interlocutor is supposed to ask me questions to make me more and more specific, I might jump the gun, leading off with "I am a human employed in the care of plants selected for their decorative or nutritive value, and at present I care for carrots, beets, and daisies" rather than "I am a gardener."

An idea of an exercise, which actually involves some popular culture - which may make it more interesting for those involved. Agree on a set of books/movies/series/etc that can be used before starting, which participants are reasonably familiar with. Postulate a broad category of characters, e.g. "Good politicians". Brainstorm a list of characters who fit this category and put the list onto a piece of paper or a whiteboard. Take turns in trying to narrow down the list by making the category more specific and crossing some characters off until onl... (read more)

In psychology, the term "high-level construal" or "abstract construal" means thinking in supercategories (mammals), and the term "low-level construal" or "concrete construal" means thinking in subcategories (poodles). Being in a high level construal helps you focus on goals, and being in a low level construal helps you focus on methods and specifics, IIRC.

When testing construal effects, one way psychologists induce low-level construals is to ask an iterative set of "How" questions.

For example:
Subject- I wan... (read more)

Notice that the entrepreneur's failure to convey his idea to PG was not, evidently, a failure of Thought - as you footnote, his system was actually pretty well-thought-out. Instead it was a failure of Communication. This distinction is important and shouldn't be blurred. Communication requires two parties, and a failure of Communication can't always be definitely attributed to one side or the other; maybe PG just wasn't asking the right questions. Or maybe he didn't care about understanding the idea at all but just wanted to probe the entrepreneur's ability to remain steady under pressure, which ability is obviously advantageous to would-be-CEOs.

I'm not convinced enough that being specific IS a good thing to do, frankly. Maybe there's something else buried within the concept that we're called "specific" that really is valuable... but when I think about "being specific" and applying this to all the various cases, it very quickly leads me to doing things that are completely useless. It collapses, basically, into casuistry. At maximum specificity, I have a list of observed cases and no way of linking them together to make sense out of them.

"What is red?" becomes "We... (read more)

Perhaps the better advice would be "be AWARE of how specific you are being, and in which direction you should move to facilitate communication" - sometimes you're giving a bunch of overly-specific examples without tying them together, and sometimes you're being overly general without giving someone some concrete examples to test their understanding.

My roommate and I often sit on the couch and discuss ideas for our shared D&D campaign. We exercise this skill a lot - two very common questions are: "can you give me some specific examples?" [be more specific] and "I don't get what ties these examples together" [what's the big picture / be more general]

Possibly an exercise that involves going through a quick dialog, and the audience calling out "be more specific" / "what's the big picture?" as appropriate at each step (or holding up colored flags, etc.). Another very simple exercise here, but it helps the audience calibrate to the idea, engages them a bit.

I don't think breaking people in to pair would be beneficial for this, unless you want to focus on "seeing where you are on the ladder, and WHICH direction you need to move." You seem to want to focus just on "moving down, being more specific", which seems fine to me. This just helps them to see the ladder itself, and realize that it moves BOTH ways.

You can then transition by saying "Okay, now we're going to focus specifically on techniques for moving DOWN the ladder, for being more SPECIFIC..."

I have begun noticing that i do this(moving up and down the ladder of abstraction) in all my communications. I realize i am doing it by reflex/habit. Trying to observe conditions that trigger a move up or down. So far there seems to be a correlation between the level of uncertainty i feel about an answer and the level of abstraction at which i verbalize my answer. Infact, i have found that the more uncertain i am about i tend to take the descriptive,detailed examples route.
I agree-- as far as I can tell, a lot of people have no idea how to summarize. On the other hand, there really are people who have trouble being specific.

A further exercise (maybe especially for children/beginners) — each student is to make up a new word. The made-up word must refer to something (real or imaginary) that does not have any specific name in the English language (or in whatever language is the working language of the group. (Examples of possible new words: "Flonk — that portion of the back of the human hand which does not contain any of the fingers" "Wedlaw — a term for the kinship relation between two people whose former spouses have married ea... (read more)

Huh, it's like Zendo except in reverse - the goal is create a guessable pattern rather than an obscure one. Goodness, I would so much rather play Reverse Zendo, too! I might actually take this idea and run with it! :)
Ewwww... if you have masters trying to make obscure patterns, then they're doing Zendo wrong. Patterns simply are obscure; there's little helping it.

This exercise is not for being specific, but just a general rationality-skill exercise that I think is useful.

Trivial Deduction

In every conversation, we hear hundreds of statements. Each of these implies many others - some directly, through definitions and linguistic rules that border on the tautological, some in combination with background knowledge, and some indirectly through multi-step inferences. Because the implications of each statement are too numerous to handle, we apply a strong filter to what reaches our attention: a statement reached by inferen... (read more)

I like this idea. I would say, though, that it teaches "how to make a specific statement", rather than "noticing WHEN you need to be more specific." If you're running in to groups that have trouble understanding what a specific statement is, or coming up with their own, this would be a very useful precursor exercise, though. It's funny to realize that telling the entrepreneur to "be more specific" doesn't work, because you weren't specific about what "specific" means! :)

A simple exercise, borrowed from giraffe language / non-violent commenucation. Describe what happened in a way, even your worst enemy would have to agree with. This means sticking to what you saw, heard etc., without lumping things together or including judgements.

So, if your friend and you were to meet at the cafe at 13.00, and he showed up at 13.05, there's not much else you can say about that situation. You can't deduce a motive for his being late, and it wouldn't be wise to lump this being-late together with other being-lates.

Exercise based on this: 4 ... (read more)

This means sticking to what you saw, heard etc., without lumping things together or including judgements.

So, if your friend and you were to meet at the cafe at 13.00, and he showed up at 13.05, there's not much else you can say about that situation. You can't deduce a motive for his being late, and it wouldn't be wise to lump this being-late together with other being-lates.

Go further. The friend showed up when the clock on the wall said 13:05; or my watch said 13:05; or whatever. After all, the friend's watch may be five minutes slow; or yours may be fast.

I suspect the bigger source of disagreement here, though, is whether 13:05 counts as "being late" for a social meeting booked for 13:00. This turns out to be extremely culturally dependent. Some cultures (and some people) value on-the-dot punctuality much more than others. To some, five minutes is a rounding error, and describing it as "being late" would be the equivalent of getting out your protractor to determine if you've been given a fair slice of pie: requiring that level of precision from someone indicates that you're either a crank, or looking for an excuse to have a disagreement with them.

So going from "we agreed to meet at 13:00; he showed up at 13:05" to "he is late" counts as a "judgement", too.

0Eliezer Yudkowsky
Is that A, B, C, D exercise standard from non-violent communication?
Sorry! Only just now saw this message! I learned this exercise from one of the giraffe language weekends I've attended, and it goes well with the first part of giraffe language, which is describing without judgement. That's all I know.

It seems like a common situation where this skill would be useful and is often lacking is where one person is an expert or semi-expert on a subject, but lacks conscious awareness of their criteria for expert decision-making. A concrete example of this that I have experienced is trying to explain to someone how I know confidently and in an instant something like "that tree is a kind of maple" or "that bird is a robin". Asking me this forces me to step back and start pointing out all the information I'm taking into account, which usuall... (read more)

"Read through the comments, gather the LessWrong usernames of everyone who made a suggestion we tried or adopted, and email the list to Luke"

It's pretty easy to just let Luke do everything isn't it? (No snark meant; I noticed this tendency in myself when we were housemates and started actively trying to fight it.)


It seems possible that when people take personality tests, they just write down their perceptions of themselves. A much better way would be to think of specific examples (of times when they were on time or late for an appointm... (read more)

This gets people thinking of specifics, but would it contrast being specific with failing to be specific, and make the students want to be specific in the future? I think that the students need that contrast just to appreciate what the issue is, and they need to see what they're doing as something that could apply to a broad set of situations in order to find occasions to behave this way in the future. I suppose you could contrast your test with a personality test that doesn't use specifics, and that could supply the contrast. How would you supply the applicability? I might be overestimating people's tendency to compartmentalize, but I doubt it. Once, when my parents were visiting my apartment before I got a microwave, my mother wanted to reheat some food in the ordinary oven. She asked what temperature I thought she should use; I didn't reheat food in the oven very often, but I suggested 300 degrees. It took me several seconds more before it occurred to me: "Mom, you reheat food in the toaster oven at home all of the time. What temperature do you use then?" Then she knew what temperature she should use. But she didn't automatically bridge the gap from "toaster oven" to "real oven," and I almost didn't, either. We're looking at a bigger gap, here, and a very strong tendency to be not-specific.
Beating compartmentalization is almost an impossible mandate. My exercise already calls for people to think of specific things they did recently in their lives. I doubt many exercises can do better than that. In terms of application, I think people are already massively curious about who they are and how they fit in (and this might apply especially strongly to people who aren't the sort to read LW). Just improving folks' self-evaluations could be seen as a pretty big benefit. http://lesswrong.com/lw/8gv/the_curse_of_identity/ Giving participants two different Big Five tests, and only telling them to think of specific examples on the second one, could work well. But it also opens the door to an underwhelming conclusion if people's abstract self-evaluations actually do tend to be fairly accurate.
Actually, I want to take some of my criticism back. It seems to me that there are several instrumental goals that would help with the terminal goal of getting people to routinely be specific at useful times in the future. No one exercise has to encompass all of the instrumental goals. The list I see right now is: 1) Make people better at being specific. 2) Get people to appreciate the value of being specific. 3) Get people to recognize situations where they or other people aren't being specific. 4) Get people to react negatively to a lack of specifics. 5) Make it occur to people to be specific. 6) Show/get people to think of contexts to apply their new skill of being specific. 7) Get people to be specific as a habit, without thinking about it. Your exercise could help with #1, and also #2 if the contrast between personality test results strikes the students as significant. Mine is intended to help with #4, and with good scenarios, could help with #3 and #6. If the scenario involves something practical, it could also help with #2. Feel free to add to my list.
8) Get people to recognize when other people want them to be more (or less) specific. 9) Get people to recognize when they are being specific about the wrong subject.
In getting them to be specific in the present, it's hard to ask for better. In getting them to be specific in the future, I'm not sure, and the point is their future behavior, right? Of course, my own exercise might be considered a cop-out in this regard; it doesn't get them to be specific in the present, even, and its main goal is to get them to simply be frustrated with a lack of specifics in the present and future. Yeah, I can see that. But that's helpful in improving their self-evaluations, not in being specific as an ongoing habit. Still useful, but I'm not sure it helps with this goal. True, but we could probably bruise it a bit. It could help even to do something as simple as telling the students three other situations they could apply the same approach to. They'd have to be fairly similar to the exercise, or else it wouldn't establish a strong enough connection in the students' heads, but I do think that talking about closely-related applications could help. With the mission statement exercise, for instance, you could point out that the same approach could help them recognize the need for specifics and the range of possible specifics in 1) descriptions of courses in college catalogs, 2) the kinds of goals that institutions set for projects, and 3) political speeches. (Any help making these three examples more specific would be gratefully appreciated.) Maybe what we need is a series of related exercises that lend themselves to being applied in related but different situations, to push at the boundaries of compartmentalization.
This risks running into salience issues. Events which lead to failure or loss of status tend to be a lot more salient than those which lead to success, and while that by itself could still give you a reasonable if upsetting metric, I don't see any obvious way to control for differences in how this tendency manifests, or for differences in salience between domains (which could paradoxically lead to overweighting failures in a domain the testee considers personally important). Differences in general recall ability would also weight the results, but you could probably control for that with some extra effort.
You could probably fix these problems by specifically asking for recent examples, in the last few weeks. Also, note I might've picked an unusually upsetting personality question. If upsetting participants is a significant issue, choosing questions and personality metrics carefully could ameliorate things. Also note that accurate personality assessment is not the primary goal here. In theory, it could be a good exercise even if it doesn't provide accurate personality assessment. In fact, you could even discard scientific pretense altogether, and just have participants complete one of those "Which Harry Potter character are you?" type quizzes, but thinking of specific examples.

Exercise (preferably attempted by participants before the lesson):

Another variation on an old classic. Split the group into triads, and each group is given a bunch of words in a hat (a box may also be used, or a hollowed out rock of some sort). The words, preferably, are about half-way "up" the abstraction lattice; "red" is good, "Steve Irwin" and "Concepts" are not.

Each person in a triad rotates between three roles: guesser, hinter, or observer. The hinter has to get the guesser to say the word within a short time ... (read more)

If CMR mini-camp participants learned the skill through exposure, then perhaps an incentivized game executed in organically occurring scenarios that rewards those who recognize and do not practice non-specificity, would do the trick.

I'm thinking of a point-based game. It would occur either during a specified block of time, or on a specified day; everyone playing would begin with an equal number of points. During this specified period players would earn points pointing out abstract, non-specific utterances of other players' (including utterances operati... (read more)

2Eliezer Yudkowsky
Hm. I especially note the concept of handing out some sort of non-monetary gift at the end of the session to someone. I wonder if that would be productive or counterproductive...
Are you wondering after the productivity of the gift idea in particular or the productivity of the concept as a whole? If it's the gift idea, then I don't see how it wouldn't be productive; a desirous prize would ensure all players apply themselves to the game, and dedicate themselves to learning the skill in the short term. If the game proceeds as expected, then all involved would have had a jolly good time; memories linked to a strong emotion are more memorable (learned this somewhere - no citation I can recall), so the positive fun emotion tied to the game, and thus the skill, will insure its memorability. The idea as a whole, however, could indeed be quite counterproductive. Though I assume there would be some down time during the mini-camp? walking between sessions, meal times, pre-sleep socializing? The game might fit well during those periods, however it could just as well hinder socialization through added pressure and unnecessary competition.
That particular element seems like it would incentivize campers to spend the period hyper-aware of their own and others' specificity, which seems counterproductive to me. The goal is an increase in the specificity of statements made casually, which could be entirely unrelated. Extending the period to say, a week, might work to prevent this- at that point it would be a long term incentive rather than a prize.
I was actually thinking of the week-long mini-camp when writing this. The idea would have to be thoroughly tested before implementation, in order to find the right balance to strike when presenting the game: the balance between encouraging a keen sense of how well you evince the meaning of your communications, while still discouraging anti-social hypersensitivity to, and subsequent criticizing of, non-specificity in others.

I've been reading the answers and trying to put words into what I want to say. Ideally people will experience not just being more specific, but experience that when they're more specific, they immedaitely communicate more effectively.

For instance, think of three or four topics people probably have an opinion on, starting with innocuous (do you like movie X) and going on to controvertial (what do you think of abortion). Either have a list in advance, or ask people for examples. Perhaps have a shortlist and let people choose, or suggest something else if the... (read more)

I like this, because it forces the audience to come up with specific statements, but it doesn't seem to teach them to recognize WHEN they need to be more specific. I'd say it's a very good precursor, to help them see what a specific statement is, and why it's useful. It's actually my favorite from this whole thread for that, so I do think it's a really cool idea! :) (I'm finding it neat how often this thread is identifying, for me, things that ought to be taught BEFORE you even get in to the core 5-second-skill of "recognizing when to be more specific". It reminds me of Eliezer's comments on the Sequences growing exponentially as he realized he needed to establish X before going on to Y, and then realizing he'd also need Q and K)

Here's a possible exercise. It's about making a set of specifications that meet a pre-defined level of abstraction, not about moving between levels of abstraction.

Toy Designer

"When I was just a wee little lad full of hope and joy My father homeward came one night and gave to me a toy A wonder to behold it was with many colours bright And the moment I laid eyes on it it became my hearts delight It went zip when it moved and bop when it stopped and whirr when it stood still I never knew just what it was and I guess I never will."

You are a toy desi... (read more)

A caveat - "specific" is not the same as "detailed". An exercise which teaches the latter may fail to hit the former.

"Detailed" is a listing of all your customers with a testimonial from each of them, many of which will be redundant, others irrelevant ("pretty home page design"). "Specific" picks out the particular thing that serves your customer better than the competition.

Being able to recognize that a given example is particularly illustrative - an examplar - is a subskill.

Exercise: Add as many qualifiers as you can that do not make your statement irrelevant or false.

For example:

My startup is better than MixPanel

My startup is better than MixPanel at making revenue on day zero

My startup is better than MixPanel at making revenue on day zero when the economy is down

Well, never mind, that didn't work.


The Center for Modern Rationality is now offering prizes for suggested exercises:

$50 for each exercise promising enough that we test it during a Saturday session. A $500 prize for any exercise which actually seems to work (as in, we decide to adopt it into the unit after testing).

Hmm. I'd understood that there was some pretty convincing evidence that offering cash incentives like this is counterproductive - it decreases creativity and effectiveness of troubleshooting. I don't have scholarly cites handy (I know there was a CMU/LSE study) but this is popularised in Dan Pink's "Drive". (There's a TED talk and a great RSAnimate video on this.)

I was interpreting this not as "do this because we will pay you", but rather as a way of signalling how much CMR cares about this being done well and quickly.
In the RSAnimate talk, Pink cited research that found that once people were doing the task (presumably under time pressure), higher incentives produced worse results. That's different from using an incentive to get people to come do the task at all.
I noticed a similar flaw in some studies Dan Areily was citing to prove that incentive pay didn't work, and e-mailed him about it. He didn't have any counterarguments when he responded to me, but he did keep citing the studies IIRC...
4Eliezer Yudkowsky
Not offering money wasn't working, and if there are a few creative money-positive people reading this who try to pick up the free $500 bills, hopefully that's enough.
Mm, the studies looked at where people were assigned specific tasks and then given incentives for speed/volume results. It may be that it works differently when used as open-ended bait to motivate people to self-assign tasks.

Continuing with the "adapt a classic" suggestions:

Surely there's some way to adapt charades to this. You give someone an example of a complicated concept, and have them try to communicate that concept in as few examples as possible. We have similar problems with the twenty questions suggestion, though: a lot of specificity depends on deep knowledge of the subject matter. If you get a concept you or the guessers have never heard of, then you're dead in the water and that'll be frustrating.

The skill goals appear to be mostly "articulate the k... (read more)

This activity seems like it would tie in well with a unit on hypothesis and experiment generation as well- it reminds me of the 2-4-6 test. Perhaps have two different scoring rules: when trying to teach specificity, give points for getting your partner to guess; when teaching how to find the right hypotheses and tests, give points for guessing correctly.

Zach Weinersmith describes Professor Liar. One player gets a field of expertise randomly drawn from two lists of 100 elements (the professor); the other players (the examiners) ask the professor questions, trying to expose that the professor is not in fact an expert on Feminist Mustache Theory. It seems like it could be readily adapted to focus on the specificity aspect of things, rather than the "convincingly bullshit" aspect of things.


This has obviously been a fun thought-exercise for many of you, but I think what's at heart here, the "skill" we're eager to develop via exercise, is just plain old good communication. In the entrepreneur example, the failure is not his inability to BE more specific, when asked. The failure is that he doesn't already KNOW he hasn't been specific enough. So, the skill we need is not how to muster specificity when Paul Graham asks for it. What we want is simply to be able to communicate abstract ideas clearly in the first place.

All these consci... (read more)

Here’s a fairly simple one for thinking concretely about the abstraction lattice. Mine an encyclopedia article for topical words (i.e. omit “the” and “and” and their ilk – also omit duplicates, should be fairly easy to program for). Place each word on an index card and have the students arrange them on a large flat surface in order of abstraction – I’d have some large amusing goalposts at either end, but this is not strictly necessary. It should probably be acceptable for some words to be judged equally concrete.

Because this is a collaborative exercise, st... (read more)

I'd suggest a game - I Don't Wanna.

The two sides are shown an end state considered to be the goal. One side writes directions to accomplish the goal, including whatever constraints he wants. His opponent tries to fail to accomplish the task while still following the directions an fulfilling the requirements.

The second player should always be able to win in absolute terms. His real goal is to be minimally obtuse - find the minimal distance between our usual priors and the priors he has to assume so that the directions don't complete the task.

For any of the ... (read more)

Okay, so split into sets of 2 people (or, split into 2 teams, or even dynamic teams could work). Person A asks a simple personal question about person B (such as "do you have a girlfriend?" or "do you have a college degree?" or "do you prefer dogs or cats?"). Person B then tries to answer like the people in the video did, by telling an abstract related story, or by answering a different question contained within or related to the question (like "well, dolphins are really my favorite animal" or "college degrees... (read more)

Exercise idea that may simultaneously teach this and help illustrate notion of Locating the Hypothesis: A prize/treasure hunt in which you make them make you be specific. ("you", of course, being understood to mean whoever's teaching them.)

ie, hide some prize (money, for example) somewhere in the building or such. Tell them you will answer their questions about where the prize is, but they might have to work a bit to ask the right questions.

You: "Ask me where the prize is." Them: "Where is the prize?" You: "Somewhere. You... (read more)

Upvoted. This helps teach the skill of "noticing WHEN you need a more specific answer", especially if you set the expectation of somewhat tricky, literal answers (think "giving orders to a malicious idiot") to badly phrased questions. You can also pretty easily split the group in to pairs - have one person pick an object in the room, then the other person has to ask questions until they can physically touch the object to confirm they have the right one in mind. I'd also set up the room with a lot of similar objects - have ten of the same identical vase in the room, for example. Try to involve distinguishing factors people might not normally think of. "Is it the vase on the left or right of the bookshelf?" is a novel question to come up with - it creates fun "aha!" moments instead of just using scripted questions.

I'm confused. Don't you already have your answer? Attendees picked it up by instructors asking attendees to be more specific about things they were discussing. Just turn that into an exercise: ask some questions about a subject that is suitably fuzzy in most people's heads (economics, philosophy, future prediction). Give feedback. Repeat until they get it.

And if you want something more specific:

Ask questions like, "Is a 'weak dollar' bad?" People will default to their cached thoughts. "Of course it's bad. It's weak, it should be strong."... (read more)

This is related to the incredibly important skill, search for the historical causes of your thoughts, rather than their justifications.

Isn't this not recommended by CBT? Everything I've read has been present-focused or forward-focused, whereas Freudian therapy is typically past-focused ("ok, we've figured out what you should do next time" vs. "ok, we've figured out who you should blame").

CBT focusses on the immediate causes, not the long-standing causes. "I'm feeling anxious because I've just got an email from my boss and it makes me worry that he's angry", not "I'm feeling anxious because of my troubled relationship with my nursery carers".

Right. The full example he gave is: The arrows implied a progression to me, and the mention of "historical causes" in that context seems like "original causes" rather than "examples of this occurring in the past."
There's at least a recent past aspect to CBT. If you say "Everyone at work hates me", you'll be asked for specific evidence that specific people (and how many of them compared to all the people at your job) hate you.
I think the bigger difference between CBT and psychoanalysis is something like, CBT: "Your feelings are the residue of your thoughts, many of which are totally wrong and should be countered by your therapist and you because human brains are horribly biased." vs, Psychoanalysis: "Your feelings are a true reflection of what an awful, corrupt, contemptible, morally bankrupt human being you are. As your therapist, I will agree with and validate anything you believe about yourself since anything you report about yourself must be true by definition." CBT still works with specific past instances of your emotions to chart feelings into thoughts. It's good to do that so you can see clearly that thoughts always proceeded your feelings about a matter.... and also to see what the content of the thoughts are if they are, sneaky, "automatic" thoughts. For example, "Jill made me sad." might be examined and reframed as "My automatic thought that hearing I was wrong about what day the garbage was picked up made me think: I'm wrong, therefore, I'm stupid, therefore, I'm worthless, therefore I'm sad. Those were all my highly-optimized and compressed thoughts which executed so fast... in such well-worn pathways... that I didn't even notice them. So my thoughts about that made me feel sad, not Jill."

Further notes on the Prize:

"Figure out a way to X" is advice, not a prize-eligible suggestion - if your comment is "Figure out a way to X..." and someone else replies with a suggested way to actually do X, they get a prize and you don't. (This may sound harsh, but we already have lots of goals - we don't need help coming up with goals - we need exercises that actually achieve those goals.)

Please don't overlook the value of including at least one sample problem or sample use-case! If it's clear how to implement your suggestion on our e... (read more)

Please don't overlook the value of including at least one sample problem or sample use-case!

The irony of having to spell that out on this post is killing me.

The irony of having to spell that out on this post is killing me.

How specifically is it killing you? :D

Softly, with his song.
That's what I was thinking, more or less. A typology of the more commonly needed sorts of specificity would probably be very useful.

The first thing that comes to mind is "write erotic fiction", but that has obvious social problems.

When it comes to specificity erotic non-fiction is even better, but the social problems are greater.
Why specifically "erotic", and then why "fiction"? What's wrong with "write about what you had for lunch" or "write about your commute to get here"? I think anyone who can write a page on those has a decent sense of how to be specific (although this does NOT train the 5-second-skill of noticing WHEN you need to be specific)
I don't think it'd necessarily need to be erotic as such, but the "show, don't tell" concept (a subset of "be specific") is, if not quite specific to fiction, at least a lot easier to screw up there. If you're writing about your lunch or your commute to work, you've already got a sense of the salient details and probably won't be tempted to hide behind coarse descriptions of your emotional state; neither's necessarily true if you're writing about some fictional character.
1Swimmer963 (Miranda Dixon-Luinenburg)
I would say 'fiction' because you're forced to come up with details yourself, and 'erotic' because that's where most people, including lots of authors, shy away from specific details...the memory of painful making yourself write very explicitly would be a more memorable lesson than writing about something innocuous like, I don't know, two siblings playing Lego.

Exercise/Game: Elevator-Pitch Descriptions

For two players. (P1 and P2)
Each player tries to describe a specific image for the other player to identify out of a larger set of images in 30 seconds.


  • Two different sets of images, and clones of these sets.
  • Something to stop P1 and P2 from seeing each other’s images. (e.g. a battle-ship style setup)
  • Clock/stop-watch that can take the time in seconds

Constructing the image sets:

The sets of images need to be of a moderate size (say 20 images) and the images themselves need to depict similar things. For exam... (read more)

I did not understand the distinction being made between a ladder and a lattice. Is the idea that the lattice is multi-dimensional? If so why was Eliezer still talking about the top and the bottom rather than some point and the origin?

He's referring to the mathematical formalism). Sets of elements, ordered by the subset relationship, are a good example of a lattice. Lattices have a specific "top" and "bottom". Among the subsets of the letters in the alphabet, the empty set is "bottom", and the entire alphabet is "top". The particular point of calling it a "lattice" rather than a ladder is that there are many ways to be more specific; of these, not all are more or less specific than each other. (Formally: you can have a < c and b < c such that neither a < b, a = b, nor a > b.)
Syntax note: Link is broken due to not escaping the closing parenthesis in the URL.
argle bargle. Fixed! Thanks.
Thank you! I was not aware of that concept.

(These are not relevant to the skill of the week. I misinterpreted the instructions. I'll leave this comment up, though, because I enjoyed these games/exercises and others might as well.)

1) Play Wits and Wagers. This is an unoriginal suggestion and probably not eligible for a reward, but nevertheless it's a fun, socially rewarding, accessible, large collection of Fermi questions about which Bayesian updates based on vague real-world knowledge can be readily applied.

Variation 1 Add the additional step that every player must explain at least one reason th... (read more)

How about this:

People are divided into pairs. Say A and B are in one pair. A gets a map of something that's fairly complex but not too complex. For example, an apartment with a sufficiently large number of rooms. A's task is to describe this to B. Once A and B are both satisfied with the description, B is asked questions about the place the map represented. Here are examples of questions that could be asked:

How many left-turns do you need to make to go from the master bed room to the kitchen?

Which one is the washroom nearest to the game room?

You are sittin... (read more)

Exercise - characterization. Form triads: one is the moderator, one the target, one the observer. Moderator asks observer "name one thing that struck you about 's character". Observer goes, for instance: "she's funny". Moderator asks for specifics: "what particular things have you observed about to make you say that?". Alternately, ask moderator to pick up on particular observations ("she has this way of pushing her glasses up her nose") and move in the other direction ("what does that tell you about her, if anything"). After a few minutes, switch roles.

This could be a good icebreaker, or a useful adjunct in a module on social skills.

This kind of close personal scrutiny could make some people feel very uncomfortable. (I don't think that's a reason to not do it though.)

It seems to me like there's a twenty-questions style rule-based game waiting to be developed here. That is, rather than asking twenty questions about an object ("is it an animal?") you ask them about a concept.

There are a few challenges- the first is that natural divisions about concepts are unfamiliar, they're probably fuzzy instead of binary, and you need fairly deep conceptual knowledge of the thing in question to find it. For example, if the concept is "Athenian direct democracy," and the first question is "does it relate to hi... (read more)

Within the domain of building-a-system, paper prototyping/wireframing teaches people to be specific with their ideas. It's only helpful when your ideas are "I want there to be this kind of thing" and then putting it on paper creates the specifics in your head.

I thought of a game for this called "Functionality Telephone"

So you divide people into pairs, and in each pair one person is the Manager, and the other person is the Designer. The Manager recieves a card with some sort of functionality pritned on it, like "Can hold a gram of water without leaking", or "Can have pebbles thrown at it and remain standing", or some other easily testable function. It will also have some taboo words, like (for the water example above) water, waterproof, spill, leak, etc. The Designer will have a ... (read more)

Have you tried playing Corrupt a Wish? No one has mentioned it by name yet.

I once played a mage in a Live Action Role Playing adventure, and cast a spell to summon a water elemental, to fight on the side of the party against the monsters opposing us.

One of the GMs appeared, appropriately clothed, and proceeded to give the fastest lesson ever on being specific. Because, the way he played it, the water elemental would obey commands, but was malicious, and where possible would misinterpret the command, or do things I'd not yet specifically forbidden it to do.

elemental appears, moves to nearest party member and starts hitting her
Me:... (read more)

"Open with the concrete example, not the abstract explanation!"

Whether concrete-first or abstract-first works better may well be a cognitive style that varies quite a bit with culture.

Kamentz & Womser-Hacker, 2003, Defining Culture-Bound User Characteristics as a Starting-Point for the Design of Adaptive Learning Systems

Nitpick: "You can say more truths about apple2, like "apple2 is dark red", then you can say that is true of all apples."

The thirteenth word of this sentence should be 'than' rather than 'then'.

During our discussion about Be Specific, majus had a good point:

Be Specific seems like a subskill of a more generic skill: fixing ontology problems. Going down a level helps you discover that you're talking about very different things when you use the word "connection"- if one person is thinking "interpersonal relationships," and the other person is thinking "communication mediated by an online application," they're not going to understand each other very well. Specific examples don't fix the ontological models, though, alth

... (read more)

Given our present teaching technology, this skill seems transmissible from master to apprentice, but not yet replicable by exercises.

Transmission may be a lot of how specificity has to be taught, since it's about comparing what's in your mind that you're trying to communicate to what's actually being received by someone else's mind, and the lesson is clearer if it's something you actually want to communicate-- that is, not an exercise.

Computer programming has lots of replicable tasks, and with any kind of meaningful feedback it's quite good at "be precise," which is the abstract version of "be specific."
That doesn't seem like the kind of precision most people have trouble with, though teaching it explicitly might help people who have trouble getting started on programming. Do you think knowing how to program helps people be sufficiently explicit about other subjects?
Probably not, because of compartmentalization. I wonder, though, if there is some sort of similar activity that: * Doesn't require expert instructors (some feedback is obvious or automatic) * Deals with less mathematical and more "real world" problems * Builds skills that transfer well to other domains.
Management, which is at times remarkably similar to the "malicious idiot" game...

Action exercise:

Instructor provides a set of somewhat vague tasks to a few groups or individuals ("Draw me a tire swing" / "Write me a poem about fish" / "Tell me about [yourself | your selves]"). The instructor has a specific interpretation in mind, in advance ("An old tire from a dump truck, tied via blue-painted rope to hang horizontal to the ground, 3 feet from the ground, from a large branch of a willow tree" / "A haiku about salmon and how they are unpleasant when used as projectiles" / "Your fa... (read more)

I like the idea of "provide abstract instructions, and leave it up to the students to realize that this is too general, and that they need to ask for more information." I would say, however, that usually when you need more information, you're aware of that need. In Eliezer's example, Paul can automatically recognize that he needs more information, because he doesn't have enough information for his own task. The very simple example would be to split in to pairs: a describer and a picker. The describer is given a picture of the piece they need, and a quick description to read (print these ahead of time!): i.e. they might need a blue 2x8x1 block, but they're told to ask for "a long block". The picker then has to realize they don't have sufficient information, and ask to clarify. It's teaching a VERY basic level of "realize I need more information, and then ask for it" - after the first example, every student will REALIZE what they need to be asking (size + color), so there's minimal sense of discovery/novelty. I think you would only be able to spend 5-10 minutes on it before adults got bored, but you could probably go a bit longer dealing with young kids. I chose Legos specifically because they don't have a ton of variables, and it's very quick to hold up a piece and get told "no", which I think is a big help here. You don't want someone to spend 5 minutes drawing a picture only to discover they got it wrong, because that creates a big negative, a lot of frustration that you don't want in a quick-and-basic exercise like this. Save frustration for the bigger challenges, when you want to illustrate the cost of failing at this skill :) This exercise would mostly serve to prime the audience's thinking in the right direction, and as a very mild "success spiral". You would definitely need to lead-in to a more complex and challenging example, to keep the audience engaged

Here's a game that might be a little more vivid, and easier to set up, than some of the games that are strictly verbal games. Call it "Tech Support."

There are two players, separated by a screen or barrier so they can't see each other, point, or gesture, only communicate by words.

One player is the Customer. They are trying to figure out how to use or perform some operation on a physical object. (Preferably one they actually don't know how to use!)

The other player is the Expert. They are trying to help the Customer use the object.

A simple musical i... (read more)

I've played that game, using various shaped blocks that the Customer has to assemble in a specific pattern. It's great. There's also the variation with an Intermediary, and the Expert and Customer can only communicate with the Intermediary, who moves back and forth between rooms and can't write anything down.

Hold up an image in front of the group, and ask "What do you see?"

Allow the participants to exhaust themselves trying to describe it. Then, start narrowing in. Ask them to describe specific parts of the image.Ask them to detail everything that is red. Everything that seems to be moving. Everything touching the ground. Everything smaller than a human. Ask them to clarify and detail descriptions. The more questions, and the more detailed the questions, the more information they'll be able to extract from the image.

Also, an improv game called... (read more)

I agree with fubarobfusco's refinements — maybe as intermediate/advanced levels, to add after the basic exercise.

2Eliezer Yudkowsky
It looks like the original what-was-being-agreed-with got deleted by the edit!
I like this. Some refinements: * Make at least one statement about yourself. * Make at least one statement a moral claim (e.g. "eating meat is wrong"). * Distinguish the contradiction of the sentence (e.g. "some religions are not ancient") from the contrary ("no religions are ancient"). * Identify a relevant base rate that the subject of your statement can be compared with (e.g. "organic food is as nutritious as conventionally grown food"). * Break the statement down into at least three distinct refinements (e.g. "organic food contains more vitamins", "organic food contains fewer insecticides", "people who switch to an organic diet report health benefits"). * Describe specific tests for those refined statements (e.g. "we could measure the Vitamin A, C, and E content of these strawberries").

i'm involved with a startup. there's so much well-intentioned bullshit and it's the founders who harm themselves more than any user or any investor.

i watched the video, and felt something was wrong, and then i read your article, you dissected it mercilessly, and nailed it precisely.

precision is hard. you know, until i started studying math, i didn't even know what "be precise" really means.
laughs That is too true :)

Easy excercise on the 5-second level: ask the question "as opposed to what?" both loud, and when constructing what you'd like to tell. An easy trigger to remember is qualifiers -they're usually a mark of motivated abstraction-switch.

Medium-level excercise: take one of your life failures at any level, and dismantle it via root cause analysis:

"The business failed." "Why?"

"We failed to nail down the unit economics tightly before scaling up marketing" "why?"

"No one was dedicated to look over all the 6 piec... (read more)

Suggested exercise: faced with a non-specific statement, like "our software makes connections between people" or "this person is a slob", come up with a specific story: say, about a particular customer using the software in a particular way, with the result that he gained two friends that he wouldn't have without the software. Adding extra details to the story, like character development, might help to hit the right level of specificity.

There are, of course, a near-infinite number of possible different stories, with different specific ... (read more)

I am borrowing an old acting game for this one, and modifying it slightly. I am calling it "which word." The rules are very simple, and this is a fairly fun warm up exercise.

Base: The other person replies with a word that is either a superclass or a subclass of the given word. Using words in a different sense is encouraged.

Options for increased difficulty include

Forced:: Each person must go up twice and then down twice, repeating endlessly

Time Limit:: People must respond with a word of their own in a given number of seconds. Feel free to make it ... (read more)

Would you mind providing an example, please, or explaining the original acting game and your alterations to it? Thank you.
1: dog 2: mammal 1: cat 2: feline 1: animal 2: flea 1: flyer 2: pilot 1: human 2: living 1: breathing 2: breather

Hi! I've been lurking for a bit.

It looks to me like one thing that would help would be to get the people you're teaching to get frustrated enough to spontaneously say "Be specific!" on their own. If you can get them to associate a feeling of frustration with certain situations, the emotional reaction could reinforce the cognitive skills they're developing.

Specific scenarios can be based off of actual conversations like the one Eliezer presented in his post. Here's an example, based on Eliezer's example:

Sample Exercise:

The student must decide... (read more)

I'd be a little leery of embarrassing those guys even more than they've already been embarrassed.
Hmm. I hadn't envisioned them being present, but I may have misunderstood the ongoing nature of these things. If they aren't present, key details might be changed, and so on, to preserve anonymity. I was also imagining that this exercise would be done privately, rather than on a stage in front of lots of people -- although that would be flexible. Reusing earlier exercises isn't essential to my idea, so that could be changed, and made-up dialogue used instead. However... I think that one of the issues with all of these exercises is going to be making sure that the students mentally connect these exercises to other areas of their lives, and apply the skill in real life. A lot of the ideas suggested here sound good, but they sound too much like games, and not enough like people actually sound when they fail to be specific in real life. I liked the "mission statement" idea for the same reason.
I meant to avoid embarrassing the real-life people whose start up wasn't mixpanel by making their embarrassing dialogue with Paul Graham into a standard rationality exercise. Not sure whether that was what you intended or not. What you think of my exercise? ;-)
That was kind of my intention, but I had imagined that they wouldn't be the only ones. (Misery loves company?) I was thinking that you might do dialogues like in Eliezer's example with a lot of people (privately), as one rationality exercise, then use that to put together a large set of scenarios. Remember, students will probably need scenarios that are different enough from their own lives for them to recognize the lack of specifics. Eliezer's example might not work at all well for someone involved in an online company, and/or a startup, because it might seem normal rather than frustration-provoking. I don't know; maybe other people here have a strong enough idea of what people actually say in these situations, and can write realistic dialogue from scratch. I'd still expect transcripts to worth consulting as part of the writing process, though. I think it would lead to more accurate personality tests, among other things. ;) Seriously, I realized after taking one personality test that I would have responded completely differently to a question if it had said "walking" rather than "driving," and the question was about something entirely unrelated to modes of transportation. (It was something like comfort in trying new things, or comfort with visual maps vs. verbal directions.) Oops?

The following exercise is inspired by the game Taboo.

Pair everyone off, giving one member of the pair a card that tells them something they must describe so that the other person will guess what it is. Include a list of taboo words that prevent them from going up the lattice, forcing them to be more specific in their descriptions.

e.g. New York Yankees Taboo words: “Yankees,” “New York,” “NYC,” “Baseball,” “Sport,” “National Pastime”

The description giver has to be more specific, saying something like “a collection of athletes, including Derek Jeter and Alex... (read more)

Potential problem with this: The game will devolve into associations rather than real specificity. For example, for the phrase "New York Yankees" I might say just "Derek Jeter" and most people would be able to guess correctly, but I haven't really practiced the skill of being specific. You'd have to specifically penalize answers like that to prevent the game from devolving into Hasbro's Taboo.

Have you thought about making one of the skills of the week "think with system two instead of system one"?

Empirically, reading anything related to rationality seems to level up some global "rationality skill" parameter in my brain, and I suspect this is because I'm learning to think using system two more and more.

But you can see the basis for the hope that - after a fair amount more work - we'll be able to offer a 2-day course for YCombinator entrepreneurs that eliminates 50% of the overhead from their conversations with Paul Graham.

That sounds like an awesome idea!

They aren't the only incubator in the Valley either.

Can I submit this to Hacker News with an inflammatory headline based on this quote, or is it too soon?