We could do a group personality test on LW with 200 questions on personality and then hunt for correlations.
If this were to happen, here are some scales possibly worth including:
The first two links are from a public-domain pool of personality scales. Shorter versions of the Big Five are also available there. The original dark triad scale is free to use. I'm not exactly sure about the copyright status of items described in academic papers, but are presumably covered under fair use.
Implicit-association tests are handy for identifying things you might not be willing to admit to yourself.
There is a Harvard page full of such tests, ranging from guns, to race and all kinds of stuff.
Still, the value of this information should be low as I see no obvious, easy way to remove the association - assuming it is invalid in the first place.
An interesting question would be whether there are areas where we don't know ourselves well as rationalists and whether we can use those tests in those areas.
Guns and race are popular topics but we don't profit much from understanding our own positions on those questions much better.
Having a implicit-association test that put people into categories of Aristotelianism, Anton-Wilsonism and Bayesianism would be fun. Having people who think that they are Bayesians score as Aristotelians would be an opportunity for growth.
An interesting question would be whether there are areas where we don't know ourselves well as rationalists and whether we can use those tests in those areas.
The obvious thing would be to test negative associations with people who do not explicitly subscribe to LW style rationality. Or associations with low IQ people. I am certain that there will be quite some surprises to some people around here.
Guns and race are popular topics but we don't profit much from understanding our own positions on those questions much better.
What would be some topics that could, in your opinion, be fruitful?
What would be some topics that could, in your opinion, be fruitful?
I added on via an edit to my last question before I saw an answer. But it's not an easy question.
As far as I understand CFAR has some techniques that they teach on their bootcamps that are effective and change the minds of the person who goes to the bootcamp in a good way.
That change of mindset could be measured with an implicit-association test. Of course knowing what those changes happen to be means knowing the basics of rationality and when you followed what I wrote lately, I consider knowing the basics to be hard.
Suddenly having a data driven tool that tells good rationalists from bad rationalists also would make things uncomfortable for a bunch of people, because it's deeper to their core than a test telling them whether the are implicit racists.
It a lot more fundamental than the basic LW consensus where we assume from each other that we are good rationalists. Data has power. Talking about the value of the scientific method as the only true frame for reality is noble. It makes it easy to signal to be a good rationalist. Identifying good rationalists from bad ones via data driven implicit reasoning tests would be walking the talk instead of just talking it.
I don't think it's my role to say what makes a good rationalist but I think we can agree that the stuff CFAR does makes someone a better rationalist. If someone has better suggestions that could be tested I'm also happy.
The LW survey has a few rationalist testing questions.
In aggregate I think you can learn something from those questions but I don't think they provide you a way to judge individual people well.
Answering those questions well also has a lot to do with whether you are exposed to them beforehand.
Guns and race are popular topics but we don't profit much from understanding our own positions on those questions much better.
We all like to imagine we're not racist. But knowing how racist one is can help optimize one's happiness when e.g. choosing where to buy a house.
So the idea is to use your test results to help optimize happiness through overtly racist behavior? This strikes me as a bad idea that could be misapplied widely.
Also, Steven Pinker suggests in Better Angels of Our Nature that rationality/the enlightenment has lead to a decline in racism/nationalism. I buy his argument, and think that it would be a much better to apply rationality toward that end than to use implicit biases to justify or guide behavior. Maybe I'm missing something, but this idea doesn't seem to fit with this site, which started out as Overcoming Bias, not "justify behavior caused/influenced by implicit biases."
I think the idea is more that you will realize that you may be irrationally discounting a house based on its neighbors -- not that you will concede extra ground to your prejudice.
you will realize that you may be irrationally discounting a house based on its neighbors
This seems like a good example of a rationalist winning to me.
Perhaps I've misinterpreted what Imm said above, but I think he was sort of saying the opposite: "The test shows I have a strong implicit bias against [name minority], so I should move to an all-white [or name your in-group here] neighborhood to be happier. In this situation, it seems like you are using knowledge of your bias to increase your irrational discounting of a house or neighborhood.
It was kind of ambiguous. BUT the sort of implicit association that this test measures is the kind that actual exposure would tend to diminish, so there's not much point in avoiding.
Rationalists should win, and feeling vaguely unhappy as you go about your life but not knowing why is not winning. The point is to overcome bias, not to pretend it isn't there (and believing one isn't biased when one is is one of the more pernicious and common biases). Sure, if you have a technique for magically making people not racist then that's better than a technique for figuring out how racist they are, but in the absence of the former the latter is useful.
if you have a technique for magically making people not racist
Pinker points to some non-magical causes: greater commerce, greater literacy, larger political and military coalitions. Essentially, greater exposure allowing people to view other people as useful (instrumentally in many cases), rational beings. He points to how much progress inter-racial relationships and gay relationships have made in acceptance; it wasn't due to people moving away from the people they didn't like. The relevant point is that people don't feel those biases as strongly any longer.
The point is to overcome bias, not to pretend it isn't there
I really like the way you put this.
Rationalists should win
I guess I think deliberately acting on implicit racist bias doesn't seem like a win for a particular rationalist or rationalism generally
The Finnish military uses personality tests on everyone to look for the leader types amongst their conscripts. Everyone with half a brain could game them either to shorten their stay or to get picked as a leader candidate. It's amazing how these kinds of useless testing rituals stick.
Everyone with half a brain could game them either to shorten their stay or to get picked as a leader candidate.
Maybe that's the test.
I talked to other people who gamed the test and they usually got they wanted, which happened to be shortened stay for all of them. I myself didn't pick just the wrong answers, but tried to randomly pick some right answers as well. Considering that none of the low-mid rank officers seemed to be the sharpest tools in the box, maybe you're right :)
Perhaps the psychologist who designed the test was a pacifist.
Regarding your thought about subtle personality tests: I find the attachment styles model a fairly powerful predictor of personality, and particularly suitable for dating. It seems to me that people fall into one of the four attachment styles quite reliably, it is fairly easy to know which one and the correlates of each style are easy to remember.
This is interesting, thanks for the link. The model seems to be partly based on one's assessment of self-worth as compared to their partner. Based on this I'd expect to see a person exhibit different styles depending on who they're dating, though this effect could be diminished by acclimation. This might account for some portion of the 20-30% of people who change styles. Is this explored in the literature? Or maybe I'm misunderstanding and the self- and other-assessments are purely positive or negative and not at all comparative.
Also I tried searching for frequencies of the four styles in the population and wasn't able to find anything. Do you know if that's available anywhere? Or even better, how the styles are distributed (are there strong clusters or are most people borderline as with MBTI?).
No remotely reliable (as in, not astrology) test I have ever seen has revealed genuinely surprising information for a moderately self-aware person, outside of ranking.
Given that most of astrology is about giving someone very vague statements that can be interpreted in any way you want, if you do get surprised by an astrology test of yourself that might mean something.
An example of a more interesting/subtle way to potentially conduct a personality test would be to use question like OKTrends' "do you like beer?" which clusters strongly with "do you have sex on the first date,"
As far as I understand most of the personality tests that get used for job interviews are of that nature. Someone who wants to hire a candidate wouldn't use a test that's easily faked by the person taking the interview.
Those tests seem to be good enough that companies are willing to pay money for them but of course that doesn't mean that they are more than cargo cult practices.
if you do get surprised by an astrology test of yourself that might mean something.
It might mean you're unusual. Did you have something particular in mind?
One of my university professor once told me that if you have a strong reaction to either the sentence: "I'm a normal human." or "I'm not a normal human." that means that you probably have an issue worth addressing in detail.
An astrology test might tell you: "One the one hand you are a human like everybody else, but on the other hand there something were unique about you."
If the statement that the test says that you are a normal human like everybody else triggers you, that has meaning. If you get an angry reaction, where you say: "No, there no way in which I'm like other people." that's a topic worth further exploration.
Just to be clear, I don't have meaningful personal experience with astrology I'm extrapolating from other personal experience and general knowledge.
If the statement that the test says that you are a normal human like everybody else triggers you, that has meaning.
I wouldn't read too much into such a reaction. It seems to be a fairly common thing, resulting in the creation of a uniqueness-seeking scale in psychology. There is some support for a "need for uniqueness" as a human universal, with a review here.
From my notes on the Handbook of Positive Psychology:
As predicted, the students who were told that they were mod- erately similar to other respondents reported more positive moods than did those students who were told that they were either highly sim- ilar or highly dissimilar to other respondents. (page 415)
The establishment of a sense of uniqueness is emotionally satisfying to individuals. Moreover, it is necessary for our psychological welfare. (page 423)
And here's some just-for-fun trivia:
Specifically, evidence of a higher than usual need for uniqueness has been found among (a) women with unusual first names (Zweigenhaft, 1981); (b) women whose nearest sibling is male rather than female (Chrenka, 1983); (c) students who are firstborn or only children versus latter born (Fromkin, Williams, & Dipboye, 1973); and (d) children of interfaith marriages (Grossman, 1990). (page 416)
It seems to be a fairly common thing
The fact that something is common doesn't mean that it's healthy. In general people who are strongly triggered by the assertion that they are normal might engage in behavior that cuts them off from their fellow humans in order to feel more unique.
There are a bunch of people who make uncommon clothing choices and then complain when they draw attention by strangers when walking in public. When I personally walk around in Vibrams (which I got 3 years ago) I do it welcoming attention by strangers.
I have no issues with practicing my dance turns in public while waiting for a train which is not standard behavior but I don't do it out of a desire to prove that I'm unique, practicing the dance turns is about practicing the dance turns regardless of who's looking.
If you engage in a bunch activity to either prove that your normal or to prove that you are special, than you aren't doing what's most beneficial for your other goals.
I liked Gallup's Strength test. It seemed somewhat less susceptible than the average to confirming prior assumptions, and at least for me it gave some interesting thought-provoking results. I'm not sure it gave me any genuinely new insights, but it certainly sharpened previously vaguely-formed feelings into something I could more readily understand.
As an aside, the different strengths all represent different ways of looking at and appreciating the world. My reaction upon reading half of the strengths were "wow, there are people who think like that?", while to the other half my reaction was, "wow, there are people who don't think like that?". For me that was the greatest antidote to the Typical Mind Fallacy that I've ever come across.
Actually the most useful thing I've gotten out of the test was as a tool for gaining insight into others. I asked several people whom I work with to take the test, and then we went through the results together to see how exactly each strength applied to them in particular. The results were definitely eye-opening.
For example, one of my employees had the Arranger strength as a leading strength. Amongst other things, that implied that she enjoyed juggling the demands of many different projects all going on at once. I asked her if this was accurate and she replied that not only was it true, but it also caused her to be bored out of her mind in her old job where there were only ever one or two things going on. I can't even begin to identify with this (I'm the opposite - I like one or two projects and more is way too much), and I would never ever have thought to ask if she wanted yet more things on her plate. Yet that's exactly what she wanted and she was very excited when I handed over several additional projects to her. And here I thought (Typical Mind Fallacy!) that I was overloading her with what I had already given her.
As far as I know, the parameters from quantified psychology that seem to have the highest descriptive value by far are g-heavy IQ and OCEAN. I do not know how stuff like EQ or different types of mental disorder tests fit into that framework, but here are some forms that psychologists can use. If there is an anonymous test, as ChristianKI suggests, I'd be willing to take it.
I wonder what the value of extensive psychological testing (IQ, OCEAN, disorders) is relative to physical tests (weight, height, blood pressure, the usual preventative measures) in terms of costs and benefits. Quite a number of people are not self-aware enough to spot the first signs of disorders themselves and to take appropriate action. Such testing, assuming high predictive value, could help with that.
I find the Love Languages test, http://www.5lovelanguages.com/ , despite sounding a bit odd, to be useful. It rates whether you express and feel affection more strongly from Physical Touch, Words of Affirmation, Quality Time, Acts of Service or Gifts.
Mainly useful because while the results made sense, they were not the ones I tend to think of and present as most important (or that I would like to be most important to me) - so they gave me a bit of insight into what emotionally felt best, rather than what I would like to appreciate most. Similarly it's useful to have partners try it, as it may be different from what they ask for or think is important.
Several people I know have said Myers-Briggs is surprisingly accurate for them but I don't know if they've done anything with that information
Several people I know have said Astrology is surprisingly accurate.
You can't really depend on recommendations for this sort of thing, since there is a force (vagueness of the reports) which explains away the evidence of the recommendations.
If you happen to read your horoscope, or your Myers-Briggs personality type, or any similar sort of thing, and find that it fits quite well for you, I can recommend selecting a few others, not intended for you, and see if you can make them fit you as well. You can also use this technique with a credulous friend, by reading them the 'wrong' one.
For me this works well to undo the 'magic' effect. But then that's just the sort of shenanigans you'd expect from a truth-seeking Sagittarius or 'Teacher' ENFJ.*
I use my results as a checklist when I'm disagreeing with someone to see why we might be seeing things differently. It seems to help, although I haven't tested rigorously.
From what I've heard, the Myers-Briggs test is fairly accurate, but even the paid MBTI evangelists say that reading through detailed descriptions of the various types to find the one which feels most familiar is better. The detailed description part is important, because the details (about a page per type, IIRC) go into both the strong and weak points, providing pressure to reject descriptions which have flaws that feel inaccurate.
From what I remember, the corporate paid evangelist visits generally do type testing three ways: Written test, description-based given by your peers, and description-based self-report. The first is for calibration, the third is the main assigner, and the second is basically a conversation-starter; the idea is to discuss ways in which people feel different internally from how they're perceived.
What did the mean with accuracy? The test told them accurately what they thought of themselves? To the extend that a test tells you something new that derivative from what you think of yourself the test should seem inaccurate.
I've tried it and I'm not sure what surprisingly accurate would mean as the questions were precisely of the sort you knew what they were asking for. It seems it could be useful for finding like minded people with the same four letter label though.
The Myers-Briggs test comes along with a theory of how the four axes interact with each other. Those interactions can be accurate or inaccurate.
These 'date marketing value' tests meet your criteria of offering surprising and valuable information. Not necessarily pleasant or true information, but surprising and valuable information yes.
heartiste.wordpress.com/dating-market-value-test-for-men/
heartiste.wordpress.com/dating-market-value-test-for-women/
Knowing what the Less Wrong wiki calls the dark arts allows one to either practice or avoid them. That is one way information that is not true can be valuable. Another way is to 'fake it til you make it' - to act confident (for example) if you know acting confident will help you achieve a goal even if your confidence is low. Valuable but not true information. See also a great deal of writing at LW and elsewhere about signaling.
Have you ever taken a personality quiz/test that helped you have valuable insights? If so, what were the tests and how were they useful?
The only useful ones I've found all yielded the same type of insight. They showed me where I stand relative to others, which is can be genuinely useful since representative samples of large populations can be hard to come by. This includes IQ tests and tests for mental disorders (in my experience, people are usually aware that they are, for example, smarter than the average (although the Dunning-Kruger effect might complicate this) or have some intrusive thoughts and compulsive rituals, but might be surprised to find that they are three standard deviations above the norm or that their symptoms are sufficiently severe to be considered OCD).
No remotely reliable (as in, not astrology) test I have ever seen has revealed genuinely surprising information for a moderately self-aware person, outside of ranking. Furthermore, they rarely gather personality data in a remotely subtle or non-transparent way ("do you like spending lots of time with large groups of people?" "yes..." "surprise, you're an extrovert!"), and thus seem super susceptible to test-takers' attempts to confirm a desired identity.
An example of a more interesting/subtle way to potentially conduct a personality test would be to use question like OKTrends' "do you like beer?" which clusters strongly with "do you have sex on the first date," and, potentially, sexual openness. Such results might be harder for manipulate (consciously or unconsciously) and could assist with deeper self-awareness.
Edited because the first link was broken.