Rationality Market Research

by Raemon 8y14th Jul 201186 comments


Several weeks ago, the NYC  Rationality Meetup Group began discussing outreach, both for  rationality in general and the group in particular. A lot of interesting problems were brought up. Should we be targeting the average person, or sticking to the cluster of personality-types that Less Wrong already attracts? How quickly should we introduce people to our community? What are the most effective ways to spread the idea of  rationality, and what are the most effective ways of actually encouraging people to undertake rational actions?

Those are all complex questions with complex answers, which are beyond the scope of this post. I ended up focusing on the question: "Is ' Rationality' the word we want to use when we're pitching ourselves?" I do not think it's worthwhile to try and change the central meme of the Less Wrong community, but it's not obvious that the new, realspace communities forming need to use the same central meme. 

This begat a simpler question: "What does the average person think of when they hear the word ' Rationality?' What positive or negative connotations does it have?" Do they think of straw vulcans and robots? Do they think of effective programmers or businessmen? Armed with this knowledge, we can craft a rationalist pitch that is likely to be effective at the average person, either by challenging their conception of  rationality or by bypassing keywords that might set off memetic immune systems.

This question has an empirical answer. A few weeks ago I made some effort to answer it. I did not get a huge array of data, but I got enough that I thought I should share it, and I'd encourage others to go out and find their own data points. Ideally someone would make a website that somehow sorts that data (and in the process hopefully get a more structured experimental setup, since mine was rather freeform.)

I work in a tall office building in NYC. Each day, I ride an elevator up to the 30th floor. At least some of those times, I find myself alone with people for 30 seconds. I started asking those people what they thought about " Rationality." My first encounter went like this:



Subject: Female Asian Approx 30
Me: "Excuse me, I was wondering, when I say the word " Rationality" what images and feelings immediately come to mind?
Woman: (Neutral but curious expression) " Rationality?"
Me: If someone were to describe themselves as a Rationalist, what would you assume about them?
Woman: Well Rationalism sounds like a particular philosophy I haven't heard of, but  Rationality just sounds like... (doesn't use a word but has an expression that seemed to indicate " Rationality is common sense, everyone uses that")
Me: That's pretty much it. A rationalist is just someone trying their best to act Rational.
Woman: "Oh." (Elevator opens, she departed with an expression suggesting neutral-postive-ish feelings, confused as to why the question was a big deal)

Over the next few subjects, I adjusted my starter-question to get to the point faster. I'm including all the data for completeness sake, but if other people want to help with this, I recommend using the question I eventually settled on. The rest of my encounters went as follows (if the conversation ends abruptly, assume the elevator door opened and the person stepped out). Afterwards I'll share some thoughts.

Subject: Female, 70s, White
(This is my conversation with my Grandmother, a musician and therapist who was helping me write a Rational Humanist song. We'd just had a lengthy conversation about my beliefs. I asked her to pretend we hadn't had that conversation and answer the question)
Me: What do you think of when you hear the word  Rationality? What would you think of someone who identified as a Rationalist?
Her: I mostly think of clients [in Therapy] who try too hard to distance themselves from their emotions. 
Me: Does it remind you of "Spock" from Star Trek?
Her: He's not really what comes to my mind, but I think people who obsess over  rationality are out of touch with their emotions.
(In this particular case I had the time to discuss the issue in detail and clarify that  rationality can mean being more, not less, in touch with your emotions)

Subject: Male, early 30s (Italian? There's a cluster of ethnicities I have trouble distinguishing between. Not sure if it matters anyway)
(The man had a slight smile on his face most of the time, with a mixture of complex expressive details. I know just enough facial recognition to know that I had no idea what he was thinking the whole time)
Me: "I'm just curious, if someone were to describe themselves as a Rationalist to you, what thoughts and images would come to your mind:
Man: "Rationalist?" 
Me: "Yeah. What stereotypes would come to mind?"
Man: "Hmm. Pragmatic. But a little naive."
Me: Interesting. What do you mean?
Man: "I just don't think you can realistically expect people to act rationally."

Subject: Male, early 40s, White
Me: "I'm just curious, if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
Man: "Heh. Normal. Clear headed. Rational. I dunno. Why? Someone call you that?
Me: "Heh, no. Well, yeah. I'm part of a group whose looking into branding issues and getting a sense of general reactions to the word."
Man: "Yeah, well anyone who doesn't like the word is being irrational."

Subject: Woman, 50s, White
Me: "I'm just curious, if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
Woman: "Hmm. Conservative. Practical."
Me: "Conservative. Do you mean politically?"
Woman. "Yeah. Well, in every way really. Politically conservative, practically conservative."
Woman exits.
I didn't have time to figure out whether for her, "conservative" was a word with positive or negative implications. (My guess was positive, with 65% probability).

Subject: Man, 60s-70s, White
Me: "I'm just curious, if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
Man. "I don't know. I  think when people rationalize they end up slipping down a path that ends up making bad decisions."
Me: "Do you see a difference between  Rationality and Rationalization?"
I can't remember exactly how he responded but it basically amounted to "no." On one hand, this guy is probably out of our demographic bracket, but I expect this particular problem to come up a lot. 

Subject: Female, late 20s, White
Me: "I'm just curious, if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
Woman: "I can't answer that."
30 seconds of awkward silence.

Subject: Woman, Approx 30s, White

(Standard intro: "what stereotypes come to mind")

Woman: "Realistic"
Me: "Oh?"
Woman: "Yeah. Why?"
Me: "We're doing some  market  research and trying to see what gut reactions we'd get if we used a particular name. Like some people hear the world Rationalist and they think 'cold and emotionless'"
Woman, "No, I don't think that at all. Just able to look realistically at things."
(As a rule of thumb I'm not sure I should explain myself in as much detail, especially not leaving people with the specific phrase "cold and emotionless" in case they weren't already thinking it. But in this case it worked out)

 Male, 50ish
Man: "Someone who likes to break things down to a [ground?] level" (can't recall his exact phrasing).
Me: "And is that a good or bad thing?"
Man: Could be either. Some people break things down just to be an asshole. Other people actually are trying to get something done."
Me: "Ah"
Man: "And sometimes you don't want someone to break things down, sometimes you want someone who doesn't lose track [of the big picture?]" (can't remember exact phrasing)
Me: Well  Rationality doesn't necessarily mean you lose track of the big picture....
(door opens, guy gets off, I feel like I messed up a bit there.)

Closing Thoughts
I felt like I had a lot of data, and that I had overall gotten a fairly positive response. I wasn't sure  Rationality was the most effective word for outreach, but I thought that the benefits of "reclaiming" the word as something with positive connotations outweighed the lukewarm responses I sometimes got.
Then I looked back and realized I'd only actually talked to 9 people. Which is not a lot. I stopped doing it for a while but I'll try and get some more data. I invite others to either help with this particular question, or to start forming other questions and getting some feedback on them. I did my questioning in one specific building, which I think had a decent cross section of people, but was still heavily tilted towards upper-middle-class, middle-aged white people, working in central Manhattan. We'll want more variety than that.
I'm sure many of you spend time standing in lines, waiting in elevators or otherwise hanging around other people in awkward silence. This is an opportunity to take that time and convert it into useful information. Most of those people will never see you again, so there's little risk of losing status.
If you are someone who shies away from social encounters, this would probably be a good experience for you. It requires a bit of courage, but it's a fear you can overcome, and helps develop the skill of initiating conversations with strangers. (It does NOT necessarily teach you good conversation techniques, since you're asking specific questions that do not make good "traditional" smalltalk, but if you have trouble initiating in the first place, it's probably more important to work on that than to worry about specific ways of conversing).
So far I've deliberately avoided asking people when there was more than one person there. Partly this was because was still a little scared and the bigger groups were even more intimidating. Partly because I was concerned about the group influencing the individual responses. (This is easy to avoid if you're in an elevator, less easy if you're waiting in a line. Not sure what I recommend there). 
Some things this was lacking:
1) A control group. "Positive" responses may have just been polite, regardless of the subject matter. At this point I don't think the effort and consequences of a control group are really worth it, but we should at least acknowledge the issue. 
2) More consistent followup statements/questions, so that all the data is based on similar input.
3) Later on, a variety of DIFFERENT statements/questions, so we can see how much has to do with  rationality and how much has to do with our phrasing. Also, just come up with new questions in general.
4) It's been suggested that I record people's responses with a hidden camera, then ask permission to use it after the fact. I'm not sure how I feel about that. It'd definitely be useful to have the real responses rather than my recollections of them, but this requires a bit more courage than I feel like I have. Hidden microphone might be more workable (I recently discovered that iPhones have pretty awesome voice memo capabilities. Which makes sense, given that audio-input is their original function). This has the added benefit of giving you feedback on your communication skills as well.
5) I mentioned before: it'd be great if someone could set up a website designed to sort this data, and it'd be even better if it was designed to handle a variety of related topics as well. Obviously the free form responses don't lend themselves well to sorting, but I think we can at least sort by:
a) demographics of the subject
b) subjective impression of how "positive" the subject was towards  rationality
c) somehow cluster responses by keywords

If the conversation has time to go anywhere, it'll be important to have an actual, good, positive definition of rationality to give to people. (Using a particular phrase can tie in with point 2 above, so we have an easier time comparing conversations). The new intro on the front page of Less Wrong is pretty good but a little too long. My recommended definition for a short, casual conversation: 
"Rationality is the study of making good decisions. We're trying to improve our understanding of how the world works and what we can do to achieve our goals."
Does anyone know if I set up a Google Docs form, can I make it public? 
If so I can actually get a repository for the data up pretty soon, but it leaves me with a new question: Outside of a America, what's the typical breakdowns for racial background? (i.e. in America you generally say "African American" which is just silly in a potentially international audience, or "Black" which is accurate but has come to sound a bit unprofessional)
Edit 2: 
I created a Google Docs form. It should be available here.  Let me know if you have any comments/critiques.