The "Prisoner's Dilemma" refers to a game theory problem developed in the 1950's. Two prisoners are taken and interrogated separately. If either of them confesses and betrays the other person - "defecting" - they'll receive a reduced sentence, and their partner will get a greater sentence. However, if both defect, then they'll both receive higher sentences than if neither of them confessed.
This brings the prisoner to a strange problem. The best solution individually is to defect. But if both take the individually best solution, then they'll be worst off overall. This has wide ranging implications for international relations, negotiation, politics, and many other fields.
Members of LessWrong are incredibly smart people who tend to like game theory, and debate and explore and try to understand problems like this. But, does knowing game theory actually make you more effective in real life?
I think the answer is yes, with a caveat - you need the basic social skills to implement your game theory solution. The worst-case scenario in an interrogation would be to "defect by accident" - meaning that you'd just blurt out something stupidly because you didn't think it through before speaking. This might result in you and your partner both receiving higher sentences... a very bad situation. Game theory doesn't take over until basic skill conditions are met, so that you could actually execute any plan you come up with.
The Purpose of This Post: I think many smart people "defect" by accident. I don't mean in serious situations like a police investigation. I mean in casual, everyday situations, where they tweak and upset people around them by accident, due to a lack of reflection of desired outcomes.
Rationalists should win. Defecting by accident frequently results in losing. Let's examine this phenomenon, and ideally work to improve it.
Contents Of This Post
- I'll define "defecting by accident."
- I'll explain a common outcome of defecting by accident.
- I'll give some recent, mild examples of accidental defections.
- I'll give examples of how to turn accidental defections into cooperation.
- I'll give some examples of how this can make you more successful at your goals.
- I'll list some books I recommend if you decide to learn more on the topic.
I know a lot of people who were nerds in school, and they all tell the same story: there is a strong correlation between being smart and being a nerd, and an even stronger inverse correlation between being a nerd and being popular. Being smart seems to make you unpopular.[...]The key to this mystery is to rephrase the question slightly. Why don't smart kids make themselves popular? If they're so smart, why don't they figure out how popularity works and beat the system, just as they do for standardized tests?[...]So if intelligence in itself is not a factor in popularity, why are smart kids so consistently unpopular? The answer, I think, is that they don't really want to be popular.If someone had told me that at the time, I would have laughed at him. Being unpopular in school makes kids miserable, some of them so miserable that they commit suicide. Telling me that I didn't want to be popular would have seemed like telling someone dying of thirst in a desert that he didn't want a glass of water. Of course I wanted to be popular.But in fact I didn't, not enough. There was something else I wanted more: to be smart.
From when I was still forced to attend, I remember our synagogue's annual fundraising appeal. It was a simple enough format, if I recall correctly. The rabbi and the treasurer talked about the shul's expenses and how vital this annual fundraise was, and then the synagogue's members called out their pledges from their seats.
Let me tell you about a different annual fundraising appeal. One that I ran, in fact; during the early years of a nonprofit organization that may not be named. One difference was that the appeal was conducted over the Internet. And another difference was that the audience was largely drawn from the atheist/libertarian/technophile/sf-fan/early-adopter/programmer/etc crowd. (To point in the rough direction of an empirical cluster in personspace. If you understood the phrase "empirical cluster in personspace" then you know who I'm talking about.)
I crafted the fundraising appeal with care. By my nature I'm too proud to ask other people for help; but I've gotten over around 60% of that reluctance over the years. The nonprofit needed money and was growing too slowly, so I put some force and poetry into that year's annual appeal. I sent it out to several mailing lists that covered most of our potential support base.
And almost immediately, people started posting to the mailing lists about why they weren't going to donate. Some of them raised basic questions about the nonprofit's philosophy and mission. Others talked about their brilliant ideas for all the other sources that the nonprofit could get funding from, instead of them. (They didn't volunteer to contact any of those sources themselves, they just had ideas for how we could do it.)
Now you might say, "Well, maybe your mission and philosophy did have basic problems - you wouldn't want tocensor that discussion, would you?"
Hold on to that thought.
Because people were donating. We started getting donations right away, via Paypal. We even got congratulatory notes saying how the appeal had finally gotten them to start moving. A donation of $111.11 was accompanied by a message saying, "I decided to give **** a little bit more. One more hundred, one more ten, one more single, one more dime, and one more penny. All may not be for one, but this one is trying to be for all."
But none of those donors posted their agreement to the mailing list. Not one.
So far as any of those donors knew, they were alone. And when they tuned in the next day, they discovered not thanks, but arguments for why they shouldn't have donated. The criticisms, the justifications for not donating - only those were displayed proudly in the open.
As though the treasurer had finished his annual appeal, and everyone not making a pledge had proudly stood up to call out justifications for refusing; while those making pledges whispered them quietly, so that no one could hear.
Indeed, that's a problem. Eliezer continues:
"It is dangerous to be half a rationalist."
Our culture puts all the emphasis on heroic disagreement and heroic defiance, and none on heroic agreement or heroic group consensus. We signal our superior intelligence and our membership in the nonconformist community by inventing clever objections to others' arguments. Perhaps that is why the atheist/libertarian/technophile/sf-fan/Silicon-Valley/programmer/early-adopter crowd stays marginalized, losing battles with less nonconformist factions in larger society. No, we're not losing because we're so superior, we're losing because our exclusively individualist traditions sabotage our ability to cooperate.
On Being Pedantic, Sarcastic, Disagreeable, Non-Complimentary, and Otherwise Defecting by Accident
You might not realize it, but in almost all of human civilization it's considered insulting to just point out something wrong someone is doing without any preface, softening, or making it clear why you're doing it.
It's taken for granted in some blunt, "say it like it is" communities, but it's usually taken as a personal attack and a sign of animosity in, oh, 90%+ of the rest of civilization.
In these so-called "normal people's societies," correcting them in front of their peers will be perceived as trying to lower them and make them look stupid. Thus, they'll likely want to retaliate against you, or at least not cooperate with you.
Now, there's a time and place to do this anyways. Sometimes there's an emergency, and you don't have time to take care of people's feelings, and just need to get something done. But surfing the internet is not that time.
I'm going to take some example replies from a recent post I made to illustrate this. There's always a risk in doing this of not being objective, but I think it's worth it because (1) I tend to read every reply to me and carefully reflect on it for a moment, (2) I understand exactly my first reactions to these comments, and (3) I won't have to rehash criticisms of another person. Take a grain of salt with you since I'm looking at replies to myself originally, but I think I can give you some good examples.
The first thing I want to do is take a second to mention that almost everyone in the entire world gets emotionally invested in things they create, and are also a little insecure about their creations. It's extraordinarily rare that people don't care what others' think of their writing, science, or art.
Criticism has good and bad points. Great critics are rare, but they actually make works of creation even in critique. A great critic can give background, context, and highlight a number of relevant mainstream and obscure works through history that the piece they're critiquing reminds them of.
Good critique is an art of creation in and of itself. But bad critique - just blind "that's wrong" without explaining why - tends to be construed as a hostile action and not accomplish much, other than signalling that "heroic disagreement" that Eliezer talks about.
I recently wrote a post titled, "Nahh, that wouldn't work". I thought about it for around a week, then it took me about two hours to think it through, draw up key examples on paper, choose the most suitable, edit, and post it. It was generally well-received here on LW and on my blog.
I'll show you three comments on there, and how I believe they could be subtly tweaked.
> I wizened up,
I don't think that's the word you want to use, unless you're talking about how you finally lost those 20 pounds by not drinking anymore.
FWIW, I think posts like this are more valuable the more they include real-world examples; it's kind of odd to read a post which says I had theory A of the world but now I hold theory B, without reading about the actual observations. It would be like reading a history of quantum mechanics or relativity with all mentions of things like the laser or double-slit experiment or Edding or Michelson-Morley removed.
An interesting start, but I would rather see this in Discussion -- it's not fully adapted yet, I think...
Now, I spend a lot of time around analytical people, so I take no offense at this. But I believe these are good examples of what I'd call "accidental defection" - this is the kind of thing that produces a negative reaction in the person you're talking to, perhaps without you even noticing.
#1 is kind of clever pointing out a spelling error. But you have to realize, in normal society that's going to upset and make hostile the person you're addressing. Whether you mean to or not, it comes across as, "I'm demonstrating that I'm more clever than you."
There's a few ways it could be done differently. For instance, an email that says, "Hey Sebastian, I wanted to give you a heads up. I saw your recent post, but you spelled "wisen" as "wizen" - easy spelling error to make, since they're uncommonly used words, but I thought you should know. "Wizen" means for things to dry up and lose water. Cheers and best wishes."
That would point out the error (if that's the main goal), and also engender a feeling of gratitude in whoever received it (me, in this case). Then I would have written back, "Hey, thanks... I don't worry about spelling too much, but yeah that one's embarrassing, I'll fix it. Much appreciated. Anyways, what are you working on? How can I help?"
I know that's how I'd have written back, because that's how I generally write back to someone who tries to help me out. Mutual goodwill, it's a virtuous cycle.
Just pointing out someone is wrong in a clever way usually engenders bad will and makes them dislike you. The thing is, I know that's not the intention of anyone here - hence, "defecting by accident." Analytical people often don't even realize they're showing someone up when they do it.
I'm not particularly bothered. I get the intent behind it. But normal people are going to be ultra-hostile if you do it to them. There's other ways, if you feel the need to point it out publicly. You could "soften" it by praising first - "Hey, some interesting points in this one... I've thought about a similar bias of not considering outcomes if I don't like what it'd mean by the world. By the way, you probably didn't mean wizen there..." - or even just saying, "I think you meant 'wisen' instead of 'wizen'" - with links to the dictionary, maybe. Any of those would go over better with the original author/presenter whom you're pointing out the error to.
Let's look at point #2. "FWIW, I think posts like this are more valuable the more they include real-world examples; it's kind of odd to read a post which says I had theory A of the world but now I hold theory B, without reading about the actual observations."
This is something which makes people trying to help or create shake their head. See, it's potentially a good point. But after someone takes some time to create something and give it away for free, then hearing, "Your work would be more valuable if you did (xyz) instead. Your way is kind of odd."
People generally don't like that.
Again, it's trivially easy to write that differently. Something like, "Thanks for the post. I was wondering, you mentioned (claim X), but I wonder if you have any examples of claim X so I can understand it better?"
That one has - gratitude, no unnecessary criticism, explains your motivation. All of which are good social skill points, especially the last one as written about in Cialdini's "Influence" - give a reason why.
#3 - "An interesting start, but I would rather see this in Discussion -- it's not fully adapted yet, I think..."
The difference between complaining and constructive work is looking for solutions. So, "There's some good stuff in here, but I think we could adapt it more. One thing I was thinking is (main point)."
Becoming More Self-Aware and Strategic; Some Practical Social Guidelines
From Anna Salamon's "Humans Are Not Automatically Strategic" -
But there are clearly also heuristics that would be useful to goal-achievement (or that would be part of what it means to “have goals” at all) that we do not automatically carry out. We do not automatically:
- (a) Ask ourselves what we’re trying to achieve;
- (b) Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress;
- (c) Find ourselves strongly, intrinsically curious about information that would help us achieve our goal;
- Generally, be gracious and thankful. This goes immensely far. Things like starting a reply with, "Thanks for this" or "Thanks for sharing these insights."
- Praising someone makes it more likely they'll accept your criticisms. "I thought your point A was excellent, however point B..."
- If you're going to disagree, summarize the person's main argument beforehand - this has a few positive effects. First, it forces you make sure you actually understand. Second, if the author has a different main point and wasn't clear, that comes out. Third, it shows some respect that you actually took the time to read and understand the post. So you could write, "I know your main argument is A, but I wanted to explore your minor point X."
- If you think something is wrong, give an explanation of what would be correct and better. "I enjoyed this post a lot - thanks for that - but one thing that's tough for me is that all the examples are about martial arts, and I don't really understand martial arts so much. Maybe next time you could provide some examples from other fields? For instance, I remember reading you're an accountant and you write poetry, maybe some examples from there?"
- If you point out something is wrong, do your best to make the mistake-maker not feel stupid. This makes them massively appreciate that. "Hey, you got your math on example X wrong... I think it actually works to 11.7. Anyways, I only recognize that because I made that mistake dozens of times myself, it's a common one to make, just wanted to point it out."
- Explain why you care about a point. This has a few positive effects. First, it lets the author cater a reply to exactly what you want. Second, you'd be amazed at how many people assume evil intent and worst-possible motives - it neutralizes that. Third, it forces you to think through how you'd like things to be, which is again good. "Hey man, I really liked this post, but I wonder if you could have split it into pieces and made it a three-parter? I ask because I surf the web from work, and I can only read in 10 minute chunks... longer posts are harder for me to get through, and I like reading your writing."
- Consider correcting someone privately while praising them publicly. This combination has been observed to engender loyalty and good feelings throughout history. I recently read an example of a samurai encouraging lords to do this from the early 1700's book "Hagakure." It works.
- Consider dropping it altogether if it's not a big deal. This about learning to prioritize - I had someone comment on my site thinking mistakenly that The Richest Man in Babylon and The Greatest Salesman in the World were by the same author. It wasn't, but who cares? It makes no difference. It's not worth pointing it out - almost everyone has an aversion to being corrected, so only do it if there's actually tangible gain. Otherwise, go do something more important and not engender the potential bad will.
- Influence: The Psychology of Persuasion
- How to Win Friends and Influence People
- Crucial Conversations: Tools For Talking When the Stakes Are High
- Any and all books on negotiation you can find (I'd recommend you read at least five highly-reviewed books on negotiation to get different points of you)
Edit: Lots of comments on this. 130 and counting. The most common criticism seems to be that adding fluff is a waste of time, insincere, and reduces signal:noise ratio. I'd encourage you to actually try it instead of just guessing - a quick word of thanks or encouragement before criticizing creates a more friendly, cooperative environment and works well. It doesn't take very long, and it doesn't detract from S:N ratio much, if at all.
Don't just guess here. Try it out for a month. I think you'll be amazed at how differently people react to you, and the uptake on your suggestions and feedback and ability to convince and teach people. Of course, you can construct examples of going overboard and it being silly. But that's not required - just try to make everything 10% more gracious, and watch how much your effectiveness increases.