It's definitely a good idea to do this.
But the way you've set about doing it isn't going to produce any worthwhile data.
I'm no expert on branding and market research, but I'm pretty sure that the best practice in the field isn't having conversations with 9 non-random strangers in a lift (asking different leading questions each time) then bunging it in Google Docs and getting other people to add more haphazard data in the hope that someone will make a website that sorts it all out.
First you need to define the question you're asking. Exactly which sub-population are you interested in? You start off asking about "the average person"'s attitude to rationality, suggesting that maybe you want to gauge attitudes across the whole (US?) population. But then you decide that the 60+ man is "outside our demographic bracket", although your 70+ grandmother apparently isn't.
Either way, the set of [people who work in your office building plus your grandmother] might not constitute a representative sample of the population of the USA, let alone everyone in the world. Getting people who frequent Less Wrong to ask people they cross paths with isn't going to be a representative ...
Action can be way worse than inaction, if what you end up doing is misleading yourself or doing harm to your cause.
I don't think what you've done is necessarily misleading or harmful, as long as you don't consider it anything more than incomplete, qualitative research into the range of responses the word "rationality" gets from random people.
But you really, really need to decide what the point of this exercise is. Are you trying to gather useful data, or make people feel more positive about rationality, or just get comfortable talking to random people? It kind of seems like at the moment, you mainly want to find post-hoc reasons why the exercise was "useful".
Here's my suggestion: if you're trying to do a survey, decide on your demographic(s) of interest. Get everyone on Less Wrong to ask around until they find a sympathiser who works in a branding/marketing survey organisation, and can slip in an extra question in a survey, asking how people respond to the term "rationality".
Failing that, collaboratively draw up a proper survey protocol and get Less Wrongers to administer it to a random sample of a people. Think it through before you do it: e.g. stopp...
Just a note about analyzing this sort of data: almost anything you do can affect responses one way or another. The inflection of your voice, facial expressions, etc, let alone who is asking the subject. I've helped run an economic experiment where we recruited subjects in a shopping mall. Both the person who recruited the subject (we had two recruiters) and the person who read the instructions (we had two instructors as well, distinct from the recruiters) were significantly associated with the response, both more so than any other covariate.
I don't bring this up to say that what your doing is worthless, just to say be careful what conclusions you draw, especially if you don't have a standardized procedure across all subjects.
There's a skill I learned in a creative writing class once: the ability to shut up completely and accept feedback. If someone said they felt your character was unsympathetic, you weren't allowed to explain the deeper aspects of his/her personality you were hinting at, or accuse the reader of missing the point. The reason being that in general authors don't get a chance to argue with their readers: your writing stands or falls on its own merit. So hearing a person's unmassaged reaction was useful and valuable feedback that you could use to actually improve the text and do better next time. Trying to change their mind was a waste of everybody's time.
The only reason I bother relating the whole anecdote is that I think it generalizes to a useful life skill. My specific point is just that I think it would be good to keep separate the attempt to discover a person's natural reaction to the word 'rationality' and the attempt to change their conception of it. Obviously changing a person's mind about rationality is something we'll want to get better at eventually, but we should first focus on learning what our starting point is. Resist the urge to defend rationality, and focus on getting as much useful information as possible about just what we're up against.
Lastly, as others have noted, props for actually going out and getting data. Arguing is fun, but empiricism is a much greater ally.
What if the question were phrased "I'm doing some market research. What thoughts come to mind when you hear the word 'rationality'?". Honest and to the point. To me the phrase "I'm just curious" leads immediately to suspicion (i.e. You're not 'just curious', you have a reason for asking. What is that reason? ... (drifting thoughts of your reasons for asking)). "If someone were to describe themselves as a Rationalist...", my first thought, "You mean yourself, don't you?". "what stereotypes would come to your mind about that person?" immediately puts me in a negative mindset since I already associate the word 'stereotype' with negative characteristics of a certain group. I know 'stereotype' doesn't always mean 'negative stereotype' but, to me, it signals an inaccurate map warning, and I think most people first associate that word with something like "Asians can't drive" instead of "Asians are really good at math".
When I tried to explain Less Wrong to my father (60ish white man), his response was, "I bet a lot of those people don't have cars." From which I think he meant that "rationalists" meant "people who are eccentric in the same ways you are".
I'm curious how varying the "opener" will affect the responses. Here are a few off the top of my head:
Opener #1: "Hey, what would you think of someone who calls themselves a rationalist? I was interviewing this guy the other day, and I asked him what words he'd use to describe himself, and that was the first thing out of his mouth. I've never heard that response before, and don't know what to think."
Opener #2: "Hey, so, I met this guy the other day, and he was going on and on about how he was a huge rationalist. That's not really something I've seen anyone do. What would you make of someone like that?"
Opener #3: "So, I have a friend who's thinking about joining/starting a 'Rationality Club.' I don't really know what to say about that, other than that it sounds a bit [pause] unique. What kind of people would you expect at a 'Rationality club?'"
These are designed to do several things. First, by describing a specific incident, it puts them in near mode, and gets them to imagine what they'd actually think. Second, they have a livelier and more conversational phrasing, which should help in getting people to open up. Third, by distancing yourself from the label, it makes them free to not be polite to the word.
These are completely untested, though I might give them a try tonight.
This seems like a good approach to get data on what the actual connotations are. There do seem to be two small concerns: First, if a stranger came up to me and asked what I thought about when I heard a certain word and that word was sufficiently abstract, I'd assign a very high probability that it was some religious or ideological group trying to missionize. That could alter how people respond, although I'm not sure exactly how it will do so. The other issue is that if someone asks about a specific word, they probably have a positive view of that word, so desire for politeness might cause one to be more positive when describing what you think.
This is pretty cool stuff. Upvoted for multiple reasons.
If I may please make a suggestion: I'd avoid suggesting the answers to your subjects. If they don't mention Spock (or some other stereotype you're prepared to knock down), you don't need to mention Spock. Just listen to what they actually are saying, even (especially?) when it's something you didn't predict.
Also, you probably already know this, but at least two of your cases kinda felt like that so I'll say it anyway: avoid being a Jehova's Witness. Or at least a clumsy JW. By this I mean, in my exper...
Each day, I ride an elevator up to the 30th floor. At least some of those times, I find myself alone with people for 30 seconds. I started asking those people what they thought about " Rationality."
I never engage strangers in situations in which the stranger cannot escape because their reactions in that situation are much less helpful than otherwise, but perhaps you are much better at putting people at their ease than I am. ADDED. Replace "engage" above with "initiate conversations with".
For what it's worth, although I used to have the impression that folks in the LW cluster were especially impressive, I don't have that impression any more. My impression is that the LW cluster represents a very small fraction of the folks who have our median level of innate intelligence and rationality--seems likely that you could do just as well picking random proficient computer programmers, and much better by, say, picking computer programmers who work for tech companies with highly selective hiring processes.
I also think it's worthwhile for us on LW to try to find the best quality people we can to infect with our rationality meme.
I do it in a roundabout way. I've gotten far fewer people started off by trying to teach them about biases directly than by saying "dude check out this fanfic it's awesome HARRY'S A SCIENTIST," or by showing them Crab Canon from Godel Escher Bach and telling them the rest of the book's just as much fun, or by showing them the game of life. Once they're hooked on these marvelous new ways of thinking, I show them the blog.
I think I'll start doing this.
And regarding #4, I think that a phone recording audio in a shirt pocket to aid later transcription wouldn't be amiss. Maybe putting the audio up online would, but in the news/reporting field it's pretty standard practice to have audio going while interviewing.
Also, if you [have/know somone with] a fairly "official-looking" camera with an externally attachable interview-style microphone, then you can go around and do some street spots with less awkwardness. When people see somone talking to a person on the street like that, they get the impression it's one-on-one. And if you approach someone, they'll be more likely to be comfortable with an interview-style conversation.
Here's two more samples for you. Both subjects are Indian, male, late twenties, highly educated and fluent in English. These are chat transcripts (used with permission):
Roshni: if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
X:Logical ?
Roshni: anything else?
X: Analytical ?
Roshni: would that be positive or negative?
X: Grey area... positive in some case - but in extreme cases, it can be negative
Roshni: if some one person identified that way, which part of the spectrum would you place ...
Subject: Female, late 20s, White
Me: "I'm just curious, if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
Woman: "I can't answer that."
30 seconds of awkward silence.
This is comedy gold.
This is a pretty good idea, actually. Before I started reading posts on Less Wrong, I didn't have a negative view of rationality, and saw it as "common sense," or "logic." I did see it as ignoring the emotional bits and focusing wholly on the facts, though. (I became interested in Less Wrong after reading through Harry Potter and the MOR, finding the theories etc. in it fascinating, and eventually meandering over to this site, which I have been browsing for several months now.)
I'll be on campus in about a month and a half and I'll make...
Well, I'll ask my dad.
.. he said the his associations with rationality are, 'being correct, logic, having wisdom, science and knowledge'. I confirmed 'no negative connotations?' and he said, 'no'.
A friend said something similar, that he associates the word with logic, being scientific,making empirical observations ... but especially logic. No negative associations. .
Is it just me, or is the word "rationality" highlighted with a yellow background for this post, and only this post? I'm finding it a very distracting effect, and I'm not seeing it anyplace but this one post.
(Firefox 5.0 and Windows XP, in case it's relevant. I'm assuming it's post formatting, but no one else seems to have commented on it, and it seems sufficiently odd to merit mention)
I've already made that comment somewhere else a few times, but public outreach would best be handled by incorporating rationality "classes/lessons" into the standard curriculum of schools. All other public outreach initiatives pale in terms of reach and effectiveness in comparison to making it "mandatory" for students. School is the holy grail of public outreach.
Getting something into the curriculum isn't easy obviously, but it can be done and one easy way in which I can see this implemented is by making a series of well-presented video...
Late to this discussion, but I wonder, have you considered how "Rational" is defined under the Myers-Briggs system?
I score in MBTI as an INTJ -- Rational Mastermind -- so I inherently understand the meaning, but there may be other personality types who express a high degree of rationality.
Have any of you taken the test? I'm sure there must be a preponderance of INTJs here.
The form seems to be broken at the City field. There is an open parenthesis, but nothing inside it and no closing parenthesis.
Several weeks ago, the NYC Rationality Meetup Group began discussing outreach, both for rationality in general and the group in particular. A lot of interesting problems were brought up. Should we be targeting the average person, or sticking to the cluster of personality-types that Less Wrong already attracts? How quickly should we introduce people to our community? What are the most effective ways to spread the idea of rationality, and what are the most effective ways of actually encouraging people to undertake rational actions?
Those are all complex questions with complex answers, which are beyond the scope of this post. I ended up focusing on the question: "Is ' Rationality' the word we want to use when we're pitching ourselves?" I do not think it's worthwhile to try and change the central meme of the Less Wrong community, but it's not obvious that the new, realspace communities forming need to use the same central meme.
This begat a simpler question: "What does the average person think of when they hear the word ' Rationality?' What positive or negative connotations does it have?" Do they think of straw vulcans and robots? Do they think of effective programmers or businessmen? Armed with this knowledge, we can craft a rationalist pitch that is likely to be effective at the average person, either by challenging their conception of rationality or by bypassing keywords that might set off memetic immune systems.
This question has an empirical answer. A few weeks ago I made some effort to answer it. I did not get a huge array of data, but I got enough that I thought I should share it, and I'd encourage others to go out and find their own data points. Ideally someone would make a website that somehow sorts that data (and in the process hopefully get a more structured experimental setup, since mine was rather freeform.)
I work in a tall office building in NYC. Each day, I ride an elevator up to the 30th floor. At least some of those times, I find myself alone with people for 30 seconds. I started asking those people what they thought about " Rationality." My first encounter went like this: