2012 Survey Results

Hi Yvain,

please state a definite end date next year. Filling out the survey didn't have a really high priority for me, but knowing that I had "about a month" made me put it off. Had I known that the last possible day was the 26th of November, I probably would have fit it in sometime in between other stuff.

6John_Maxwell13y

Hm, could it be that the longer survey format this time around cut down on the number of responses as well?

-1[anonymous]13y

So, what "cut down on the number of responses"?

7Ralith13y

I presume that John's indended implication was that community growth was greater than indicated by the number of respondents.

[-]jimrandomh13y410

The calibration question is an n=1 sample on one of the two important axes (those axes being who's answering, and what question they're answering). Give a question that's harder than it looks, and people will come out overconfident on average; give a question that's easier than it looks, and they'll come out underconfident on average. Getting rid of this effect requires a pool of questions, so that it'll average out.

[-]Morendil13y150

Yep. (Or as Yvain suggests, give a question which is likely to be answered with a bias in a particular direction.)

It's not clear what you can conclude from the fact that 17% of all people who answered a single question at 50% confidence got it right, but you can't conclude from it that if you asked one of these people a hundred binary questions and they answered "yes" at 50% confidence, that person would only get 17% right. The latter is what would deserve to be called "atrocious"; I don't believe the adjective applies to the results observed in the survey.

I'm not even sure that you can draw the conclusion "not everyone in the sample is perfectly calibrated" from these results. Well, the people who were 100% sure they were wrong, and happened to be correct, are definitely not perfectly calibrated; but I'm not sure what we can say of the rest.

7CarlShulman13y

I have often pondered this problem with respect to some of the traditional heuristics and biases studies, e.g. the "above-average driver" effect. If people consult their experiences of subjective difficulty at doing a task, and then guess they are above average for the ones that feel easy, and below average for the ones that feel hard, this will to some degree track their actual particular strengths and weaknesses. Plausibly a heuristic along these lines gives overall better predictions than guessing "I am average" about everything. However, if we focus in on activities that happen to be unusually easy-feeling or hard-feeling in general, then we can make the heuristics look bad by only showing their successes and not their failures. Although the name "heuristics and biases" does reflect this notion: we have heuristics because they usually work, but they produce biases in some cases as an acceptable loss.

2steven046113y

I would agree that this explains the apparent atrocious calibration. It's worth an edit to the main post. No reason to beat ourselves up needlessly. People were answering different questions in the sense that they each had an interval of their own choosing to assign a probability to, but obviously different people's performance here was going to be strongly correlated. Bayes just happens to be the kind of guy who was born surprisingly early. If everyone had literally been asked to assign a probability to the exact same proposition, like "Bayes was born before 1750" or "this coin will come up heads", that would have been a more extreme case. We'd have found that events that people predicted with probability x% actually happened either 0% or 100% of the time, and it wouldn't mean people were infinitely badly calibrated.

-1A1987dM13y

All of that also applies to the year calibration questions in previous surveys and yet people did much better in those.

8steven046113y

Because they weren't about events that occurred surprisingly early.

0[anonymous]13y

Yes, and this is probably worth an edit to the original post. For a more extreme example, consider what would happen if you asked a large group of people to assess the probability that the same coin would come up heads. You'd find that events that people said would happen 50% of the time happened either 0% or 100% of the time, but it would be wrong to conclude they were atrociously calibrated.

[-]gwern13y370

I previously mentioned that item non-response might be a good measure of Conscientiousness. Before doing anything fancy with non-response, I first checked that there was a correlation with the questionnaire reports. The correlation is zero:

R> lwc <- subset(lw, !is.na(as.integer(as.character(BigFiveC))))
R> missing_answers <- apply(lwc, 1, function(x) sum(sapply(x, function(y) is.na(y) || as.character(y)==" ")))
R> cor.test(as.integer(as.character(lwc$BigFiveC)), missing_answers)

    Pearson's product-moment correlation

data:  as.integer(as.character(lwc$BigFiveC)) and missing_answers
t = -0.0061, df = 421, p-value = 0.9952
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 -0.09564  0.09505
sample estimates:
       cor
-0.0002954
# visualize to see if we made some mistake somewhere
R> plot(as.integer(as.character(lwc$BigFiveC)), missing_answers)

I am completely surprised. The results in the economics paper looked great and the rationale is very plausible. Yet... The 2 sets of data here have the right ranges, there's plenty of variation in both dimension, I'm sure I'm catching most of the item non-responses or N... (read more)

[-]Kindly13y220

There is a correlation of 0.13 between non-responses and N.

Of course, there's also a correlation of -0.13 between C and the random number generator.

[-]A1987dM13y160

People who had seen the RNG give a large number were primed to feel unusually reckless when taking the Big 5 test. Duh. (Just kidding.)

6NancyLebovitz13y

Were you expecting that people with high C would or wouldn't skip questions? I can see arguments either way. Conscientious people might skip questions they don't have answers to or that they aren't willing to put the time into to give a good answer, or they might put in the work to have answers they consider good to as many questions as possible. Is it feasible to compare wrong sort of answer with C? Is it possible that the test for C wasn't very good?

8gwern13y

Wouldn't; that was the claim of the linked paper. Not really, if it wasn't caught by the no-answer check or the NA check. As I said, it came out as expected for LW as a whole, and it did correlate with income once the CS salaries were removed... Hard to know what ground-truth there could be to check the scores against.

4Vaniver13y

I am also surprised by this. I wonder about the effect of "I'm taking this survey so I don't have to go to bed / do work / etc.," but I wouldn't have expected that to be as large as the diligence effect. Also, perhaps look at nonresponse by section? I seem to recall the C part being after the personality test, which might be having some selection effects.

1gwern13y

What do you mean? I can't compare non-response with anyone who didn't supply a C score, and there were plenty of questions to non-response on after the personality test section.

3Vaniver13y

It seems to me that other survey non-response may be uncorrelated with C once you condition on taking a long personality survey, especially if the personality survey doesn't allow nonresponse. (I seem to recall taking all of the optional surveys and considering the personality one the most boring. I don't know how much that generalizes to other people.) The first way that comes to mind to gather information for this is to compare the nonresponse of people who supplied personality scores and people who didn't, but that isn't a full test unless you can come up with another way to link the nonresponse to C. I was thinking it might help to break down the responses by section, and seeing if nonresponse to particular sections was correlated with C, but the result could only be that some sections are anticorrelated if a few are correlated. So that probably won't get you anything.

1gwern13y

Why would the strong correlation go away after adding a floor? That would simply restrict the range... if that were true, we'd expect to see a cutoff for all C scores but in fact we see plenty of very low C scores being reported. Yes. You'd expect, by definition, that people who answered the personality questions would have fewer non-responses than the people who didn't... That's pretty obvious and true: R> lwc <- subset(lw, !is.na(as.integer(as.character(BigFiveC)))) R> missing_answers1 <- apply(lwc, 1, function(x) sum(sapply(x, function(y) is.na(y) || as.character(y)==" "))) R> lwnc <- subset(lw, is.na(as.integer(as.character(BigFiveC)))) R> missing_answers2 <- apply(lwnc, 1, function(x) sum(sapply(x, function(y) is.na(y) || as.character(y)==" "))) R> t.test(missing_answers1, missing_answers2) Welch Two Sample t-test data: missing_answers1 and missing_answers2 t = -25.19, df = 806.8, p-value < 2.2e-16 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -18.77 -16.05 sample estimates: mean of x mean of y 9.719 27.129

[-]A1987dM13y360

I really have no idea what went so wrong [with the question about Bayes' birth year]

Note also that in the last two surveys the mean and median answers were approximately correct, whereas this time even the first quartile answer was too late by almost a decade. So it's not just a matter of overconfidence -- there also was a systematic error. Note that Essay Towards Solving a Problem in the Doctrine of Chances was published posthumously when Bayes would have been 62; if people estimated the year it was published and assumed that he had been approximately in his thirties (as I did), that would explain half of the systematic bias.

7Sam_Jaques13y

To expand on this: Confidence intervals that are accurate for multiple judgements by the same person may be accurate for the same judgement made by multiple people. Normally, we can group everyone's responses and measure how many people were actually right when they said they were 70% sure. This should average out to 70% is because the error is caused by independent variations in each person's estimate. If there's a systematic error, then even if we all accounted for the systematic error in our confidence levels, we would all still fail at the same time if there was an error.

2Alejandro113y

I had a vaguely right idea for the year of publication, and didn't know it was posthumous, but assumed that it was published in his middle-to-old age and so got the question right.

1Cakoluchiam13y

This question was biased against people who don't believe in history. For my answer, which was wildly wrong, I guesstimated by interpolating backward using the rate of technological and cultural advance in various cultures throughout my lifetime, the dependency of such advances on Bayesian and related logics, with an adjustment for known wars and persecution of scientists and an assumption that Bayes lived in the western world. I should have realized that my confidence on estimates of each of these (except the last) was not very good and that I really shouldn't have had any more than marginal confidence in my answer, but I was hoping that the sheer number of assumptions I made would approach statistical mean of my confidences and that the overestimates would counterbalance the underestimates. The real lesson I learned from this exercise is that I shouldn't have such high confidence in my ability to produce and compound a statistically significant number of assumptions with associated confidence levels.

0Manfred13y

Have you read Malcolm Gladwell - Blink? It's a fun book that doesn't take too long, which hella makes up for the occasional failure of rigor. Anyhow, the conclusion is that even on hard problems, expert-trusted models will still have very few parameters. And those parameters don't have to be the same things you'd use if you were a perfect reasoner - what's important is that you can use it as an indicator.

0magfrump13y

I personally had error bars of 75 years on my confidence and was 74 years off. I'm not sure if I translated that correctly into percent certainty of being within 20 years of correct, but I felt okay about the result. This might be another source of systematic error?

[-]Epiphany13y340

On IQ Accuracy:

As Yvain says, "people have been pretty quick to ridicule this survey's intelligence numbers as completely useless and impossible and so on" because if they're true, it means that the average LessWronger is gifted. Yvain added a few questions to the 2012 survey, including the ACT and SAT questions and the Myers-Briggs personality type question that I requested (I'll explain why this is interesting), and that give us a few other things to check against, which has made the figures more believable. The ridicule may be an example of the "virtuous doubt" that Luke warns about in Overconfident Pessimism, so it makes sense to "consider the opposite":

The distribution of Myers-Briggs personality types on LessWrong replicates the Mensa pattern. This is remarkable since the patterns of personality types here are, in many significant ways, the exact opposite of what you'd find in the regular population. For instance, the introverted rationalists and idealists are each about 1% of the population. Here, they are the majority and it's the artisans and guardians who are relegated to 1% or less of our population.

Mensa's personality test results we... (read more)

[-]erratio13y100

Alternate possibility: The distribution of personality types in Mensa/LW relative to everyone else is an artifact produced by self-identified smart people trying to signal their intelligence by answering 'yes' to traits that sound like the traits they ought to have.

eg. I know that a number of the T/F questions are along the lines of "I use logic to make decisions (Y/N)", which is a no-brainer if you're trying to signal intelligence.

A hypothetical way to get around this would be to have your partner/family member/best friend next to you as you take the test, ready to call you out when your self-assessment diverges from your actual behaviour ("hold on, what about that time you decided not to go to the concert of [band you love] because you were angry about an unrelated thing?")

5Epiphany13y

Ok, it's possible that all of the following happened: * Most of the 1000 people decided to lie about their IQ on the LessWrong survey. * Most of the liars realized that their personality test results were going to be compared with Mensa's personality type results, and it dawned on them that this would bring their IQ lie into question. * Most of the liars decided that instead of simply skipping the personality test question, or taking it to experience the enjoyment of finding out their type, they were going to fudge the personality test results, too. * Most of the liars actually had the patience to do an additional 72 questions specifically for the purpose of continuing to support a lie when they had just slogged through 100 questions. * Most of the liars did all of that extra work (Researching the IQ correlation with the SAT and the ACT and fudging 72 personality type questions) when it would have been so much easier to put their real IQ in the box, or simply skip the IQ question completely because it is not required. * Most of the liars succeeded in fudging their personality types. This is, of course, possible, but it it is likely to be more complicated than it at first seems. They'd have to be lucky that enough of the questions give away their intelligence correlation in the wording (we haven't verified that). They'd have to have enough of an understanding of what intelligent people are like that they'd choose the right ones. Questions like these are likely to confuse a non-gifted person trying to guess which answers will make them look gifted: "You are more interested in a general idea than in the details of its realization" (Do intelligent people like ideas or details more?) "Strict observance of the established rules is likely to prevent a good outcome" (Either could be the smarter answer, depending who you ask.) "You believe the best decision is one that can be easily changed" (It's smart to leave your options open, but it's also more intelle

1private_messaging12y

"Lie" is a strawman. One could report an estimate, mis-remember, report the other "IQ" (mental age / chronological age metric), or one may have took any one of entirely faulty online tests that report IQ as high to increase the referral rate (some are bad enough to produce >100 if the answers are filled in at random).

1Epiphany12y

This would be a good point in the event that we were not discussing IQ scores generated by an IQ test selected by Yvain, which many people took at the same time as filling out the survey. This method (and timing) rules out problems due to relying on estimates alone, most of the potential for mis-remembering, (neither of which should be assumed to be likely to result in an average score that's 30 points too high, as mistakes like these could go in either direction), and, assuming that the IQ test Yvain selected was pretty good, it also rules out the problem of the test being seriously skewed. If you would like to continue this line of argument, one effective method of producing doubt would be to go to the specific IQ test in question, fill out all of the answers randomly, and report the IQ that it produces. If you want to generate a full-on update regarding those particular test results, complete with Yvain being likely to refrain from recommending this source during his next survey, write a script that fills out the test randomly and reports the results so that multiple people can run it and see for themselves what average IQ the test produces after a large number of trials. You may want to check to see whether Yvain or Gwern or someone has already done this before going to the trouble. Also, there really were people whose concern it was that people were lying on the survey. Your "lie is a strawman" perception appears to have been formed due to not having read the (admittedly massive number of) comments on this.

0private_messaging12y

Look. People misremember (and remember the largest value, and so on) in the way most favourable to themselves. While mistakes can of course go in either direction, they don't actually go in either direction. If you ask men to report their penis size (quite literally), they over-estimate; if you ask them to measure, they still overestimate but not by as much. This sort of error is absolutely the norm in any surveys. More so here, as the calibration (on Bayes date of birth question at least) was comparatively very bad. The situation is anything but symmetric, given that the results are rather far from the mean, on a Gaussian. Furthermore, given the interest in self improvement, people here are likely to have tried to improve their test scores by practice, which would have considerably lower effect on iqtest.dk unless you practice specifically the Raven's matrices. The low scores on iqtest.dk are particularly interesting in light that the scores on the latter are a result of better assignment of priors / processing of probabilities (as, fundamentally, one needs to pick the choice which results in simplest - highest probability - overall pattern. If one is overconfident about the pattern they see being the best, one's score is lowered, so poor calibration will hurt that test more).

1Epiphany12y

I intuit that this is likely to be a popular view among sceptics, but I do not recall ever being presented with research that supports this by anyone. To avoid the lure of "undiscriminating scepticism", I am requesting to see the evidence of this. I agree that, for numerous reasons, self-reported IQ scores, SAT scores, ACT scores and any other scores are likely to have some amount of error, and I think it's likely for the room for error to be pretty big. On that we agree. An average thirty points higher than normal seems to me to be quite a lot more than "pretty big". That's the difference between an IQ in the normal range and an IQ large enough to qualify for every definition of gifted. To use your metaphor, that's like having a 6-incher and saying it's 12. I can see guys unconsciously saying it's 7 if it's 6, or maybe even 8. But I have a hard time believing that most of these people have let their imaginations run so far away with them as to accidentally believe that they're Mensa level gifted when they're average. I'd bet that there was a significant amount of error, but not an average of 30 points. If you agree with those two, then whether we agree over all just depends on what specific belief we're each supporting. I think these beliefs are supported: * The SAT, ACT, self-reported IQ and / or iqtest.dk scores found on the survey are not likely to be highly accurate. * Despite inaccuracies, it's very likely that the average LessWrong member has an IQ above average - in other words, I don't think that the scores reported on the survey are so inaccurate that I should believe that most LessWrongers actually have just an average IQ. * LessWrong is (considering a variety of pieces of evidence, not just the survey) likely to have more gifted people than you'd find by random chance. Do we agree on those three beliefs? If not, then please phrase the belief(s) you want to support.

9CCC12y

Even if every self-reported IQ is exactly correct, the average of the self-reported IQ values can still be (and likely will still be) higher than the average of the readership's IQ values. Consider two readers, Tom and Jim. Tom does an IQ test, and gets a result of 110. Jim does an IQ test, and gets a result of 90. Tom and Jim are both given the option to fill in a survey, which asks (among other questions) what their IQ is. Neither Tom nor Jim intend to lie. However, Jim seems significantly more likely to decide not to participate; while Tom may decide to fill in the survey as a minor sort of showing off. This effect will skew the average upwards. Perhaps not 30 points upwards... but it's an additional source of bias, independent of any bias in individual reported values.

5Vaniver12y

I remember looking into this when I looked at the survey data. There were only a handful of people who reported two-digit IQs, which is consistent with both the concealment hypothesis and the high average intelligence hypothesis. If you assume that nonresponders have an IQ of 100 on average the average IQ across everyone drops down to 112. (I think this is assumption is mostly useful for demonstrative purposes; I suspect that the prevalence of people with two-digit IQs on LW is lower than in the general population.) (You could do some more complicated stuff if you had a functional form for concealment that you wanted to predict, but it's not obvious to me that IQs on LW actually follow a normal distribution, which would make it hard to separate out the oddities of concealment with the oddities of the LW population.)

0Epiphany12y

Ah! Good point! Karma for you! Now I will think about whether there is a way to figure out the truth despite this. Ideas?

3CCC12y

Hmmm. Tricky. * Select a random sampling of people (such as by picking names from the phonebook). Ask each person whether they would like to fill in a survey which asks, among other things, for their IQ. If a sufficiently large, representative sample is taken, the average IQ of the sample is likely to be 100 (confirm if possible). Compare this to the average reported IQ, in order to get an idea of the size of the bias. * Select a random sampling of lesswrongers, and ask them for their IQs. If they all respond, this should cut out the self-selection bias (though the odds are that at least some of them won't respond, putting us back at square one). It's probably also worth noting that this is a known problem in statistics which is not easy to compensate for.

2somervta12y

There's also the selection effect of only getting answers from "people who , when asked, can actually name their IQ".

3satt12y

As one of the sceptics, I might as well mention a specific feature of the self-reported IQs that made me pretty sure they're inflated. (Even before I noticed this feature, I expected the IQs to be inflated because, well, they're self-reported. Note that I'm not saying people must be consciously lying, though I wouldn't rule it out. Also, I agree with your three bullet points but still find an average LW IQ of 138-139 implausibly high.) The survey has data on education level as well as IQ. Education level correlates well with IQ, so if the self-reported IQ & education data are accurate, the subsample of LWers who reported having a "high school" level of education (or less) should have a much lower average IQ. But in fact the mean IQ of the 34% of LWers with a high school education or less was 136.5, only 2.2 points less than the overall mean. There is a pretty obvious bias in that calculation: a lot of LWers are young and haven't had time to complete their education, however high their IQs. This stacks the deck in my favour because it means the high-school-or-less group includes a lot of people who are going to get degrees but haven't yet, which could exaggerate the IQ of the high-school-or-less group. I can account for this bias by looking only at the people who said they were ≥29 years old. Among that older group, only 13% had a high school education or less...but the mean IQ of that 13% was even higher* at 139.6, almost equal to the mean IQ of 140.0 for older LWers in general. The sample sizes aren't huge but I think they're too big to explain this near-equality away as statistical noise. So IQ or education level or age was systematically misreported, and the most likely candidate is IQ, 'cause almost everyone knows their age & education level, and nerds probably have more incentive to lie on a survey about their IQ than about their age or education level. ---------------------------------------- * Assuming people start university at age 18, take 3 years to g

5gwern12y

And I suspect if you look at the American population for that age cohort, you'll find a lot higher a percentage than 13% which have a "high school education or less"... All you've shown is that of the highschool-educated populace, LW attracts the most intelligent end, the people who are the dropouts for whatever reason. Which for high-IQ people is not that uncommon (and one reason the generic education/IQ correlation isn't close to unity). LW filters for IQ and so only smart highschool dropouts bother to hang out here? Hardly a daring or special pleading sort of suggestion. And if we take your reasoning at face-value that the general population-wide IQ/education correlate must hold here, it would suggest that there would be hardly any autodidacts on LW (clearly not the case), such as our leading 'high school education or less' member, Eliezer Yudkowsky.

2satt12y

Right, but even among LWers I'd still expect the dropouts to have a lower average IQ if all that's going on here is selection by IQ. Sketch the diagram. Put down an x-axis (representing education) and a y-axis (IQ). Put a big slanted ellipse over the x-axis to represent everyone aged 29+. Now (crudely, granted) model the selection by IQ by cutting horizontally through the ellipse somewhere above its centroid. Then split the sample that's above the horizontal line by drawing a vertical line. That's the boundary between the high-school-or-less group and everyone else. Forget about everyone below the horizontal line because they're winnowed out. That leaves group A (the high-IQ people with less education) and group B (the high-IQ people with more). Even with the filtering, group A is visibly going to have a lower average IQ than B. So even though A comprises "the most intelligent end" of the less educated group, there remains a lingering correlation between education level and IQ in the high-IQ sample; A scores less than B. The correlation won't be as strong as the general population-wide correlation you refer to, but an attenuated correlation is still a correlation.

4Nornagest12y

It seems implausible to me that education level would screen off the same parts of the IQ distribution in LW as it does in the general population, at least at its lower levels. It's not too unreasonable to expect LWers with PhDs to have higher IQs than the local mean, but anyone dropping out of high school or declining to enter college because they dislike intellectual pursuits, say, seems quite unlikely to appreciate what we tend to talk about here.

1satt12y

Upvoted. If I repeat the exercise for the PhD holders, I find they have a mean IQ of 146.5 in the older subsample, compared to 140.0 for the whole older subsample, which is consistent with what you wrote.

0EHeller12y

How significant is that difference?

0satt12y

I did a back-of-the-R-session guesstimate before I posted and got a two-tailed p-value of roughly 0.1, so not significant by the usual standard, but I figured that was suggestive enough. Doing it properly, I should really compare the PhD holders' IQ to the IQ of the non-PhD holders (so the samples are disjoint). Of the survey responses that reported an IQ score and an age of 29+, 13 were from people with PhDs (mean IQ 146.5, SD 14.8) and 135 were from people without (mean IQ 139.3, SD 14.3). Doing a t-test I get t = 1.68 with 14.2 degrees of freedom, giving p = 0.115.

0Nornagest12y

It's a third of a SD and change (assuming a 15-point SD, which is the modern standard), which isn't too shabby; comparable, for example, with the IQ difference between managerial and professional workers. Much smaller than the difference between the general population and PhDs within it, though; that's around 25 points.

1EHeller12y

I was really asking about sample size, as I was too lazy to grab the raw data.

0private_messaging12y

Yes, and even without particular expectation of inflation, once you see IQs that are very high, you can be quite sure IQs tend to be inflated simply because of the prior being the bell curve. Any time I see "undiscriminating scepticism" mentioned, it's a plea to simply ignore necessarily low priors when evidence is too weak to change conclusions. Of course, it's not true "undiscriminating scepticism". If LW undergone psychologist-administered IQ testing and that were the results, and then there was a lot of scepticism, you could claim that there's some excessive scepticism. But as it is, rational processing of probabilities is not going to discriminate that much based on self reported data.

2private_messaging12y

Sceptics in that case, I suppose, being anyone who actually does the most basic "Bayesian" reasoning, such as starting with a Gaussian prior when you should (and understanding how an imperfect correlation between self reported IQ and actual IQ would work on that prior, i.e. regression towards the mean when you are measuring by proxy). I picture there's a certain level of Dunning Kruger effect at play, whereby those least capable of probabilistic reasoning would think themselves most capable (further evidenced by calibration; even though the question may have been to blame, I'm pretty sure most people believed that a bad question couldn't have that much of an impact). Wikipedia to the rescue, given that a lot of stuff is behind the paywall... http://en.wikipedia.org/wiki/Illusory_superiority#IQ "The disparity between actual IQ and perceived IQ has also been noted between genders by British psychologist Adrian Furnham, in whose work there was a suggestion that, on average, men are more likely to overestimate their intelligence by 5 points, while women are more likely to underestimate their IQ by a similar margin." and more amusingly http://en.wikipedia.org/wiki/Human_penis_size#Erect_length Just about any internet forum would select for people owning a computer and having an internet connection and thus cut off the poor, mentally disabled, and so on, improving the average. So when you state it this way - mere "above average" - it is a set of completely unremarkable beliefs. It'd be interesting to check how common are advanced degrees among white Americans with actual IQ of 138 and above, but I can't find any info.

1Vaniver12y

This was one of the things I checked when I looked into the IQ results from the survey here and here. One of the things I thought was particularly interesting was that there was a positive correlation between self-reported IQ and iqtest.dk (which is still self-reported, and could have been lied on, but hopefully this is only deliberate lies, rather than fuzzy memory effects) among posters and a negative correlation among lurkers. This comment might also be interesting. I endorse Epiphany's three potential explanations, and would quantify the last one: I strongly suspect the average IQ of LWers is at least one standard deviation above the norm. I would be skeptical of the claim that it's two standard deviations above the norm, given the data we have.

2private_messaging12y

Wow, that's quite interesting - that's some serious Dunning-Kruger. Scatterplot could be of interest. Thing to keep in mind is that even given a prior that errors can go either way equally, when you have obtained a result far from the mean, you must expect that errors (including systematic errors) were predominantly in that direction. Other issue is that in a 1000 people, about 1 will have an IQ of >=146 or so , while something around 10 will have fairly severe narcissism (and this is not just your garden variety of overestimating oneself, but the level where it interferes with normal functioning). Self reported IQ of 146 is thus not really a good sign overall. Interestingly some people do not understand that and go on how the others "punish" them for making poorly supported statements of exceptionality, while it is merely a matter of correct probabilistic reasoning. The actual data is even worse than what comparisons of prevalence would suggest - 25% of people put themselves in the top 1% in some circumstances. Yes, average of 115 would be possible.

3Vaniver12y

The actual data is linked in the post near the end. If you drop three of the lurkers- who self-reported 180, 162, and 156 but scored 102, 108, and 107- then the correlation is positive (but small). (Both samples look like trapezoids, which is kind of interesting, but might be explained by people using different standard deviations.)

0[anonymous]12y

That sounds pretty high to me. I haven't looked into narcissism as such, but I remember seeing similar numbers for antisocial personality disorder when I was looking into that, which surprised me; the confusion went away, however, when I noticed that I was looking at the prevalence in therapy rather than the general population. Something similar, perhaps?

0ygert12y

You know, people do lie to themselves. It's a sad but true (and well known around here) fact about human psychology that humans have surprisingly bad models of themselves. It is simply true that if you asked a bunch of people selected at random about their (self-reported) IQ scores, you would get an average of more than 100. One would hope that LessWrongers are good enough at detecting bias in order to mostly dodge that bullet, but the evidence of whether or not we actually are that good at it is scarce at best.

2Epiphany12y

Your unintentional lie explanation does not explain how the SAT scores ended up so closely synchronised to the IQ scores - as we know, one common sign of a lie is that the details do not add up. Synchronising one's SAT scores to the same level as one's IQ scores would most likely require conscious effort, making the discrepancy obvious to the LessWrong members who took the survey. If you would argue that they were likely to have chosen corresponding SAT scores in some way that did not require them to become consciously aware of discrepancies in order to synchronize the scores, how would you support the argument that they synched them on accident? If not, then would you support the argument that LessWrong members consciously lied about it? Linda Silverman, a giftedness researcher, has observed that parents are actually pretty decent at assessing their child's intellectual abilities despite the obvious cause for bias. "In this study, 84% of the children whose parents indicated that they fit three-fourths of the characteristics tested above 120 IQ. " (An unpublished study, unfortunately.) http://www.gifteddevelopment.com/PDF_files/scalersrch.pdf This isn't exactly the same as managing knowledge of one's own intellectual abilities, but if it would seem to you that parents would most likely be hideously biased when assessing their children's intellectual abilities even though, according to a giftedness researcher, this is probably not the case, then should you probably also consider that your concern that most LessWrong members are likely to subconsciously falsify their own IQ scores by a whopping 30 points (if that is your perception) may be far less likely to be a problem than you thought?

8NonComposMentis13y

Scores on standardized tests like SAT and ACT can be improved via hard work and lots of practice -- there are abundant practice books out there for such tests. It is entirely conceivable that those self-reported IQs were generated via comparing scores on these standardized tests against IQ-conversion charts. I.e., with very hard work, the apparent IQs are in the 130+ range according to these standardised tests; but when it comes to tests that measure your native intelligence (e.g., iqtest.dk), the scores are significantly lower. In future years, it would be advisable for the questionnaire to ask participants how much time they spent in total to prepare for tests such as SAT and ACT -- and even then you might not get honest answers. That brings me to the point of lying... Not necessarily true. If the survey results show that LWers generally have IQs in the gifted range, then it allows LWers to signal their intelligence to others just by identifying themselves as LWers. People would assume that you probably have an IQ in the gifted range if you tell them that you read LW. In this case, everyone has an incentive to fudge the numbers. erratio has also pointed out that participants might have answered those personality tests untruthfully in order to signal intelligence, so I shan't belabour the point here.

3Epiphany13y

Ok, now here is a motive! I still find it difficult to believe that: 1. Most of 1000 people care so much about status that they're willing to prioritize it over truth, especially since this is LessWrong where we gather around the theme of rationality. If there's anyplace you'd think it would be unlikely to find a lot of people lying about things on a survey, it's here. 2. The people who take the survey know that their IQ contribution is going to be watered down by the 1000 other people taking the survey. Unless they have collaborated by PM and have made a pact to fudge their IQ test figures, these frequently math oriented people must know that fudging their IQ figure is going to have very, very little impact on the average that Yvain calculates. I do not know why they'd see the extra work as worthwhile considering the expected amount of impact. Thinking that fudging only one of the IQs is going to be worthwhile is essentially falling for a Pascal's mugging. 3. Registration at LessWrong is free and it's not exclusive. At all. How likely is it, do you think, that this group of rationality-loving people has reasoned that claiming to have joined a group that anybody can join is a good way to brag about their awesomeness? I suppose you can argue that people who have karma on their accounts can point to that and say "I got karma in a gifted group" but lurkers don't have that incentive. All lurkers can say is "I read LessWrong." but that is harder to prove and even less meaningful than "I joined LessWrong". Putting the numbers where our mouths are: If the average IQ for lurkers / people with low karma on LessWrong is pretty close to the average IQ for posters and/or people with karma on LessWrong, would you say that the likelihood of post-making/karma-bearing LessWrongers lying on the survey in order to increase other's status perceptions of them is pretty low? Do you want to get these numbers? I'll probably get them later if you don't, but I have a pile of LW me

[-]Vaniver13y150

From the public dataset:

165 out of 549 responses without reported positive karma (30%) self-reported an IQ score; the average response was 138.44.

181 out of 518 responses with reported positive karma (34%) self-reported an IQ score; the average response was 138.25.

One of the curious features of the self-reports is how many of the IQs are divisible by 5. Among lurkers, we had 2 151s, 1 149, and 10 150s.

I think the average self-response is basically worthless, since it's only a third of responders and they're likely to be wildly optimistic.

So, what about the Raven's test? In total, 188 responders with positive karma (36%), and 164 responders without positive karma (30%) took the Raven's test, with averages of 126.9 and 124.4. Noteworthy is the new max and min- the highest scorer on the Raven's test claimed to get 150, and the three sub-100 scores were 3, 18, and 66 (of which I suspect only the last isn't a typo or error of some sort).

Only 121 users both self-reported IQ and took the Raven's test. The correlation between their mean-adjusted self-reported IQ and mean-adjusted Raven's test was an abysmal .2. Among posters with positive karma, the correlation was .45; among posters without positive karma, the correlation was -.11.

3Epiphany13y

Thank you for these numbers, Vaniver! I should have thanked you sooner. I had become quite busy (partly with preparing my new endless September post) so I did not show up to thank you promptly. Sorry about that.

2Vaniver13y

You're welcome!

0NonComposMentis13y

I have thought of that. But a person who wants to lie about his IQ would think this way: If I lie and other LWers do not, it is true that my impact on the average calculated IQ will be negligible, but at least it will not be negative; but if I lie and most other LWers also lie, then the collective upward bias will lead to a very positive result which would portray me in a good light when I associate myself with other LWers. So there is really no incentive to not lie. (I'm not saying that they definitely lied; I'm merely pointing out that this is something to think about.) Fair point; but very often the kind of clubs you join does indicate something about your personality and interests, regardless of whether you are actually an active/contributing member or not. Saying "I read LessWrong" or "I joined LessWrong" certainly signals to me that you are more intelligent than someone who joined, say, Justin Bieber's fan club, or the Twilight fan-fiction club. And if there are numbers showing that LW readers tend to have IQs in the gifted range, naturally I would think that X is probably quite intelligent just by virtue of the fact that X reads LW. One last point is that LWers might not be deliberately lying: Perhaps they were merely victim to the Dunning-Kruger effect when self-reporting IQs. I am not sure if there are any studies showing that intelligent people are generally less likely to fall prey to the Dunning-Kruger effect. Last but not least, I would again like to suggest that future surveys include questions asking people how much time they spent on average preparing for exams such as the SAT and the ACT -- as I pointed out previously, scores on such exams can be very significantly improved just by studying hard, whereas tests like iqtest.dk actually measure your native intelligence.

1Epiphany13y

Not true. It would probably take at least 20 minutes to fudge all the stuff that has to be fudged. When you're already fatigued from filling out survey questions, that's even less desirable at that time. At best, this would be falling for a Pascal's mugging. True that some people may. But would the majority of survey participants... at a site about rationality? They were not asked to assess their own IQ they were asked to report the results of a real assessment. To report something other than the results of a real assessment is a type of lie in this case. That's a suggestion for Yvain. I don't assist with the surveys.

8gwern13y

Make a copy and post it. Most browsers have the ability to print/save pages as PDFs or various forms of HTML.

[-]Epiphany13y320

Ok I managed to dig it up!

 E/I | S/N | T/F | J/P  (Category)

----------------------------------------------

75/25 75/25 55/45 50/50 (Overall population)

27/73 10/90 75/25 65/35 (Mensans)

15/85  03/97 88/12 54/46 (LessWrongers) *

From the December 1993 Mensa Bulletin.

* The LessWrongers were added by me, using the same calculation method as in the comment where I test my personality type predictions and are based on the 2012 survey results.

4DaFranker13y

Thanks for the analysis. I agree with your conclusion. On a less relevant note, it does feel good to see more evidence that the community we hang out with is smart and awesome.

[-]Epiphany13y230

This also explains a lot of things. People regard IQ as if it is meaningless, just a number, and they often get defensive when intellectual differences are acknowledged. I spent a lot of time doing research on adult giftedness (though I'm most interested in highly gifted+ adults) and, assuming the studies were done in a way that is useful (I've heard there are problems with this), and my personal experiences talking to gifted adults are halfway decent as representations of the gifted adult population, there are a plethora of differences that gifted adults have. For instance, in "You're Calling Who A Cult Leader?" Eliezer is annoyed with the fact that people assume that high praise is automatic evidence that a person has joined a cult. What he doesn't touch on is that there are very significant neurological differences between people in just about every way you could think of, including emotional excitability. People assume that others are like themselves, and this causes all manner of confusion. Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker Ne... (read more)

[-]Desrtopa13y110

Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker News are gifted, same goes for many of them. To those who feel so strongly, excited praise may seem fairly normal. To all those who do not, it probably looks crazy.

Would you predict then that people who're not gifted are in general markedly less inclined to praise things with a high level of intensity?

This seems to me to be falsified by everyday experience. See fan reactions to Twilight, for a ready-to-hand example.

[-]Kaj_Sotala13y260

My hypothesis would simply be that different people experience emotional intensity as a reaction to different things. Thus, some think we are crazy and cultish, while also totally weird for getting excited about boring and dry things like math and rationality... while some of us think that certain people who are really interested in the lives of celebrities are crazy and shallow, while also totally weird for getting excited about boring and bad things like Twilight.

This also leads each group to think that the other doesn't get similar levels of emotional intensity, because only the group's own type of "emotional intensity" is classified as valid intensity and the other group's intensity is classified as madness, if it's recognized at all. I've certainly made the mistake of assuming that other people must live boring and uninteresting lives, simply because I didn't realize that they genuinely felt very strongly about the things that I considered boring. (Obligatory link.)

(Of course, I'm not denying there being variation in the "emotional intensity" trait in general, but I haven't seen anything to suggest that the median of this trait would be considerably different in gifted and non-gifted populations.)

1Epiphany13y

Ok, where do I find them?

-1Desrtopa13y

If you have to go looking, you're lucky. If you want to find them in person, the latest Twilight movie is still in theaters, although you've missed the people who made a point of seeing it on the day of the premier.

3Epiphany13y

Haha, I guess so. I am very, very nerdy. I had fun getting worldly in my teens and early 20's, but I've learned that most people alienate me, so I've isolated myself into as much of an "ivory tower" as possible. (Which consists of me doing things like getting on my computer Saturday evenings and nerding so hard that I forget to eat.) Not really. What did they do when you saw them? How do we distinguish the difference between the kind of fanaticism that mentally unbalanced people display for, say, a show that is considered by many to have unhealthy themes and the kind of excitement that normal people display for the things they love? Maybe Twilight isn't the best example here.

6Desrtopa13y

I didn't. I don't particularly have to go out of my way to find Twilight fans, but if I did, I wouldn't. I think you're dramatically overestimating the degree to which fans of Twilight are psychologically abnormal. Harlequin romance was already an incredibly popular genre known for having unhealthy themes. Twilight, like Eragon, is a mostly typical work of its genre with a few distinguishing factors which sufficed to garner it extra attention, which expanded to the point of explosive popularity as it started drawing in people who weren't already regular consumers of the genre.

5Epiphany13y

I wouldn't be surprised if this is true. This still does not answer the question "What sample can we use that filters out fanaticism from mentally unbalanced people to compare the type of excitement that gifted people feel to the type of excitement that everyone else feels?" Not to assume that no gifted people are mentally unbalanced... I suppose we'd really have to filter those out of both groups.

4Eugine_Nier13y

Taboo "mentally unbalanced".

6Eugine_Nier13y

What distinction are you trying to make here?

9RobertLumley13y

we will all be brain-dead in 70 years.

0Epiphany13y

It's true that the downward trend can't go on forever, and to say that it's definitely going to continue would be (all by itself, without some other arguments) an appeal to history or slippery slope fallacy. However, when we see a trend as consistent and as potentially meaningful as the one below, it makes sense to start wondering why it is happening: IQ Trend Analysis

9RobertLumley13y

I was mostly just trying to point out that you are extrapolating from a sample size of three points. Three points which have a tremendous amount of common causes that could explain the variation. Furthermore you aren't extrapolating 10% further from the span of your data, which might be ok, but actually 100% further. You're extrapolating for as long as we have data, which is... absurd.

0Epiphany13y

One, I am used to seeing the term "sample size" applied to things like the people being studied, not a number of points done in a calculation. If there is some valid use of the term "sample size" that I am not aware of would you mind pointing me in the correct direction? Two, I am not sure where you're getting "three points" from. If you mean the amount of IQ points that LessWrong has lost on the studies, then it was 7.18 points, not three. Two points per year, which could be explained in other ways, sure. No matter what the trend, it could be explained in other ways. Even if it was ten points per year we could still say something like "The smartest people got bored taking the same survey over and over and stopped." There are always multiple ways to explain data. That possibility of other explanations does not rule out the potential that LessWrong is losing intelligent people. Not sure what these 10% and 100% figures correspond to. If I am to understand why you said that, you will have to be specific about what you mean. Including all of the data rather than just a piece of the data is bad why?

4RobertLumley13y

Three points referred to the number of surveys taken, which I didn't bother to look up, but I believe is three. 10% and 100% referred to the time span over which these data points referred to, ie. three years. Basically, I might be OK with you making a prediction for the next three months (still probably not) but extrapolating for three years based on three years of data seems a bit much to me.

-3Epiphany13y

Oh I see. The problem here is that "if the trend continues" is not a prediction. "I predict the trend will continue" would be a prediction. Please read more carefully the next time. You confused me quite a bit.

9RobertLumley13y

If you're not making a prediction, then it's about as helpful as saying "If the moon crashes into North America next year, LW communities will largely cease to exist."

3DaFranker13y

Looks like Aumann at work. My own readings, though more specifically on teenage giftedness in the 145+ range, along with stuff on ASD and asperger, heavily corroborate with this. When I was 17, my (direct) family and I had strong suspicions that I was in this range of giftedness - suspicions which were never reliably tested, and thus neither confirmed nor infirmed. It's still up in the air and I still don't know whether I fit into some category of gifted or special individuals, but at some point I realized that it wasn't all that important and that I just didn't care. I might have to explore the question a bit more in depth if I decide to return into the official educational system at some point (I mean, having a paper certifying that you're a genius would presumably kind of help when making a pitch at university to let you in without the prerequisite college credit because you already know the material). Just mentioning all of the above to explain a bit where my data comes from. Both of my parents and myself were all reading tons of books, references, papers and other information along with several interviews with various psychology professionals for around three months. Also, and this may be another relevant point, the only recognized, official IQ test I ever took was during that time, and I had a score of "above 130"² (verbal statement) and reportedly placed in the 98th and 99th percentiles on the two sections of a modified WAIS test. The actual normalized score was not included in the report (that psychologist(?¹) sucked, and also probably couldn't do the statistics involved correctly in the first place). However, I was warned that the test lost statistical significance / representativeness / whatever above 125, and so that even if I had an IQ of 170+ that test wouldn't have been able to tell - it had been calibrated for mentally deficient teenagers and very low IQ scores (and was only a one-hour test, and only ten of the questions were written, the rest dyn

3someonewrongonthenet13y

Was Mensa's test conducted on the internet? The internet has a systematic bias in personalities. For example, reddit subscriptions to each personality type reddit favor Introversion and Intuition 4,828 INTJ 4,457 INTP 1,817 INFP 1,531 INFJ

4A1987dM13y

IAWYC, but "the internet" is way too broad for what you actually mean -- ISTM that a supermajority of teenagers and young adults in developed countries uses it daily, though plenty of them mostly use it for Facebook, YouTube and similar and probably have never heard of Reddit. (Even I never use Reddit unless I'm following a link to a particular thread from somewhere else -- but the first letter of my MBTI is E so this kind of confirms your point.)

0someonewrongonthenet13y

Yeah...by "internet" what I meant was sites that most people do not know about - sites that you would only stumble upon in the course of extensive net usage. I once described it to a friend as "deep" vs "shallow" internet, with depth corresponding to the extent to which a typical visitor to the website uses the internet. Even within a website (say reddit) a smaller sub-reddit would be "deeper" than a main one. I'm myself am actually a counterexample to my own "extroverts don't use the internet as much" notion...but I'm only a moderate extrovert. (ENTP or ENFP depending on the test...ENTP description fits better. I listed ENTP in the survey.)

3Eugine_Nier13y

By that definition, there are many nearly disconnected "deep internets".

2someonewrongonthenet13y

Yes...i'm confused. Is this supposed to be a flaw in the definition? The idea here is to use relative obscurity to describe the degree to which a site is visited only by Internet users who do heavy exploring. There are only a few "shallow" regions... Facebook, Wikipedia, twitter...the shallowest being google. These are all high traffic and even people who never use computers have heard some of these words. There are many deep regions, on the other hand, and most are disconnected.

0Eugine_Nier13y

It is if you then proceed to claim to have statistics over users of the "deep internet".

0satt13y

Yeah, different websites have different personality skews, which complicates things. Fortunately there's evidence against Mensa having used an online sample: Epiphany said the results were published in December 1993. It's fairly easy to give a survey to an Internet forum nowadays, but where would Mensa have found an online sample back in '93? IRC? Usenet? (There is a rec.org.mensa where people posted about personality and the Myers-Briggs back in 1993, but the only relevant post that year was someone asking about Mensans' personalities to no avail.)

1Epiphany13y

I don't have any more data than that, sorry. To suggest that people on the internet may have certain personality types is a good suggestion, but it raises two questions: * Might your example of Reddit be similar to LW because LW gets lots of users from Reddit? (Or put another way, if the average LessWronger is gifted, maybe "the apple doesn't fall far from the tree" and Reddit has lots of gifted people, too.) * Might gifted people gather in large numbers on the internet because it's easier to find people with similar interests? (Just because people on the internet tend to have those personality types, it doesn't mean they're not gifted.) As for "the internet" having a systematic bias in personalities, I would like to see the evidence of this that's not based on a biased sample. It's likely that the places you go to find people like you will, well, have people like you, so even if you (or somebody else on one of those sites) observed a pattern in personality types across sites they hang out on, the sample is likely to be biased.

2Kindly13y

I'd say "LW has about as many gifted people as Reddit (proportionally)" should be a sort of null hypothesis: if this is true, then people on LessWrong are not actually surprisingly smart.

6gwern13y

I wouldn't say that's a reasonable null. Reddit has like 8 million users; 2% of the 310m American population is just 6.2m, so it would be difficult for Reddit to be 100% gifted while LW could easily be. The size disparity is so large that such a null seems more than a little weird.

0Kindly13y

I don't think I understand your objection. If LW were 100% gifted (while Reddit, presumably, is not?) wouldn't that be evidence that there's some sort of IQ selection at work? (or, conceivably, that just being on LW makes people smarter, although I think that's not supposed to be a thing).

4gwern13y

I'm saying that we could, just from knowing how big Reddit is, reject out of hand all sorts of proportions of gifted because it would be nigh impossible; a set of nulls (the proportions 0-100%), many of which (all >75%) we can reject before collecting any data is a pretty strange choice to make!

2Kindly13y

Well, really what I want to ask is: is LW any different, IQ-wise, from a random selection of Redditors of the same size? Possibly stating it in terms of a proportion of "gifted" people is misleading, but that's not as interesting anyway.

4gwern13y

I don't see the difference. A random selection of Redditors is going to depend on what Reddit overall looks like...

0Kindly13y

Well, I don't see the difference either, but I'm still not entirely sure what about this hypothesis seems unreasonable to you, so I was hoping this reformulation would help. The reasoning behind it is as follows: I figure a generic discussion board on the Internet has roughly the same IQ distribution as Reddit. If LW has a high average IQ, but so does Reddit, then presumably these are both due to the selection effect of "someone who posts on an online discussion board". So to see if LW is genuinely smarter, we should be comparing it to Reddit, not to the Normal(100,15) distribution.

2gwern13y

I would be shocked if that were true. Even after having grown stupendously, Reddit is still better than most discussion boards I happen to read.

0Kindly13y

Okay, fair enough. I don't actually have much experience with Reddit. I still think it's a reasonable reference class. For one thing, LW runs on Reddit-based code. In particular, I would say that being significantly smarter than Reddit is a good cutoff for the feeling of smugness to start kicking in.

2Nominull13y

Maybe it just means Reddit-folk are surprisingly smart? I mean, IQ 130 corresponds to 98th percentile. The usual standard for surprise is 95th percentile.

1someonewrongonthenet13y

That's a good point - I hadn't considered sample bias. Extending that point, though, Lesswrong and Mensa are a biased sample in more than the simple fact that the people are gifted. It is only a subset of gifted people that choose to participate in Mensa It should be mentioned, I'm using "internet" as shorthand for the "deep" internet ... not facebook. I'm talking websites that most people do not use, that you'd have to spend a lot of time on the internet to find. As such, the "internet" hypothesis would predict a greater bias towards smaller sub-reddits. Anyway, I was mostly posing an alternate hypothesis. When I first noticed the trend on the personality forums, this is what I thought was happening - Slacking off / internet addiction selects for Perceiving and low Conscientiousness. Non-social-networking internet use selects for Introversion. Any forum discussing an idea without immediate practical benefits selects for iNtuition. And then, factor in lesswrong/giftedness... If it's a math/science/logic topic, it selects for Thinking and iNtuition. High scores on Raven's matrices select for Thinking, iNtuition. High scores on Working memory components select for Judging. The ACT/SAT additionally select for Conscientiousness Strong mathematical affinity shifts those on the border of NTP and NTJ into *NTJ (people prefer dealing with intellectually ordered systems, even if they have messy rooms and chaotic lifestyles) A scientific/engineering ideology creates a shift towards the concrete (empirical evidence, practical gains in technology, etc) shifts those on the border of NTJ and STJ into ISTJ. In summary, I think LW and Mensa surveys are attracting a special subset of idea driven and logical people (iNtuitives and Thinkers) and likely to use the internet often/spot the survey. (Introverts)

2Epiphany13y

That's much nicer and much more detailed. Questions this raises: 1. Might the "deep" internet you refer to be selecting for gifted people? (I think this is likely!) 2. Do we have figures on personality types and IQs for internet forums in general, not from a biased sample set? These figures would test your theory.

0someonewrongonthenet13y

I agree with (1), but would claim that it also selectively attracts introverts (and I'm unsure whether or not it will bias J-P to the P side) (2) For each of these, I tried not to look at the data after finding the poll. I made predictions first. Just for fun / to correct for hindsight bias, anyone reading might want to do the same. To play, don't click on the link or read my prediction until you make yours. Also, here is some data which claims to represent the general population - http://mbtitruths.blogspot.com/2011/02/real-statistics.html for comparison. I've already seen similar data on another site, so I won't state my predictions on this one. A website posts stats for people who have taken the test. Unlike the above simple random sample, this selects for internet users. http://www.personalitypage.com/html/demographics.html Prediction: I'd consider this "shallow internet", so very weak biases to (I). The general population is (S), I'd expect a weak bias to (N) but not enough to overcome the general population's S centering. Result: apparently I suck at predictions. In hindsight all the top three would be predicted score high "Fi" on a Jungian cognitive function test, and Fi in theory would be more interested in taking personality tests. But that's hindsight, and I'm not sure if connection between MBTI and Jung hasn't been verified empirically. Here is a "deep internet" forum that I wouldn't ever visit... Christian singles chat forum! This should not suffer from the sample bias you mentioned earlier (He stated that websites I visit are likely to have users with similar personalities to me [ENTP]) http://christianchat.com/christian-singles-forum/34516-meyers-briggs-type-indicator-mbti-poll.html Prediction: I tried my best not to look at the data despite the high visual salience as soon as you open that link. Here is my prediction: I'd predict strong biases towards Introversion (because internet), slight biases towards iNtuition (because religion is idea-ba

0[anonymous]12y

I'm inclined to believe the survey results myself, but there is a third possibility. If a certain personality type (or distribution of types) reflects a desire to associate with gifted people, or to be seen as gifted, we'd likely expect that to be heavily overrepresented in MENSA; that's pretty much the reason the club exists, after all. We might also expect people with those desires to be less inclined to share average or poor IQ results, or even to falsify results. If the same personality type is overrepresented here, then we have a plausible cause for similar personality test results and for exaggerated IQ reporting, without necessarily implying that the actual IQ distributions are similar.

0Epiphany13y

---------------------------------------- Looking at Groups of IQs: I acknowledge that the sample set for the highest IQ groups are, of course, rather small, but that's all we've got. What's been happening with the numbers for the highest IQ groups, if indicative of what's really happening, is not encouraging. The highest two groups have decreased in numbers while the lowest two have increased. Also, it looks like the prominence of each group has shifted over time such that the highest group went from being 1/5 to 1/20 and the moderately gifted and normal groups have grown substantially. Exceptionally Gifted Respondents (Self-Reported IQ) (Defined as having an IQ of 160 or more) 2009: 11 (7%) 2011: 27 (3%) 2012: 22 (2%) (Decreased) Highly Gifted Respondents (Self-Reported IQ) (Defined as having an IQ between 145-159) 2009: 17 (11%) 2011: 88 (9%) 2012: 81 (7%) (Decreased) Moderately Gifted Respondents (Self-Reported IQ) (Defined as having an IQ between 132-144) 2009: 22 (14%) 2011: 125 (13%) 2012: 149 (11%) (Increased) Normal Respondents (Self-Reported IQ) (Defined as having an IQ between 100-131) 2009: 11 (7%) 2011: 91 (10%) 2012: 94 (9%) (Increased) ---------------------------------------- Each Group as a Percentage of Total IQ Respondents, by Year: 2009 Group IQ Distribution (As a percentage of 61 total IQ respondents) 18% Exceptionally Gifted 28% Highly Gifted 36% Moderately Gifted 18% Normal IQ 2011 Group IQ Distribution (As a percentage of 331 total IQ respondents) 8% Exceptionally Gifted 27% Highly Gifted 38% Moderately Gifted 28% Normal IQ 2012 Group IQ Distribution (As a percentage of 346 total IQ respondents) 6% Exceptionally Gifted 23% Highly Gifted 43% Moderately Gifted 27% Normal IQ

0hyporational13y

I don't find it that hard to see why Lesswrong and Mensa would both select for introverted personalities. Do you? I think most sensible people can deduce that IQ is positively correlated with SAT and ACT and all of them are positively correlated with "status". I agree that SAT and ACT are more difficult to fudge though. I haven't ever done either of them. Can they be easily redone several times? Do (smart) people liberally talk about their scores in the US? Many people do IQ tests of different calibers several times and could just remember or report the best result they've gotten. There are different levels of dishonesty. "Lying" is a bit crude.

0alfredmacdonald13y

I don't think anyone on Less Wrong has lied about their IQ. (addendum: not enough to seriously alter the results, anyway.) If you come up with a "valuing the truth" measure, LessWrong would score pretty highly on that considering the elaborate ways people who post here go about finding true statements in the first place. To lie about your IQ would mean you'd have to know to some degree what your real IQ is, and then exaggerate from there. However, I do think it's more likely than you mention that most people on LessWrong self-reporting IQ simply don't know what their IQ is in absolutely certain terms, since to know your adult IQ you'd have to see a psychometricist and receive an administered IQ test. iqtesk.dk is normed by Mensa Denmark, so it's far more reliable than self-reports. You don't know where the self-reported IQ figures are coming from -- they could be from a psychometricist measuring adult IQ, or they could be from somewhere far less reliable. It could be that they know their childhood IQ was measured at somewhere around 135 for example, and are going by memory. Or they could know by memory that their SAT is 99th percentile and spent a minute to look up what 99th percentile is for IQ, not knowing it's not a reliable proxy. Or they might have taken an online test somewhere that gave ~140 and are recalling that number. Who knows? Either way, I consider "don't attribute to malice what you can attribute to cognitive imperfection" a good mantra here. 126 is actually higher than a lot of people think. As an average for a community, that's really high -- probably higher than all groups I can think of except math professors, physics professors and psychometricists themselves. It's certainly higher than the averages for MIT and Harvard, anyway. About the similarity between self-reported IQ and SAT scores: SAT scores pre-1994 (which many of the scores on here are not likely to fall into) are not reliable as IQ test proxies; Mensa no longer accepts them. This is

-11Epiphany13y

[-]Bugmaster13y270

"Robot god apocalypse cult spinoff from Harry Potter."

That should be on a T-shirt.

5Nornagest13y

I think that's my favorite description on that list.

3Tenoke13y

I'd buy that shirt. This is instant classic.

1Tripitaka13y

http://www.spreadshirt.com/design-your-own-t-shirt-C59/product/103759664/view/1/sb/l I thinks it a nice robot, but maybe some of our art-inclined people would like to design a robot god thats got a Harry-Potterish feel about it?

7Bugmaster13y

I'm envisioning a robot in the classic Sistine Chapel God pose, only with menacingly glowing red eyes. Instead of pointing with its finger, it's holding a wand. There's a wizard hat on its head. The image could be done in silhouette, for that extra-stylized look. If I had any artistic skill, I'd draw it myself :-/

2thomblake13y

Spinoff is misspelled.

0Tripitaka13y

sigh Fixed: http://www.spreadshirt.com/design-your-own-t-shirt-C59/product/103760337/view/1/sb/l

1Bugmaster13y

This link takes me to a blank T-shirt design UI...

0[anonymous]13y

Myspace Fun Flash Generator

[-]John_Maxwell13y260

But I am skeptical of these numbers. I hang out with some people who are very closely associated with the greater Less Wrong community, and a lot of them didn't know about the survey until I mentioned it to them in person. I know some people who could plausibly be described as focusing their lives around the community who just never took the survey for one reason or another. One lesson of this survey may be that the community is no longer limited to people who check Less Wrong very often, if at all. One friend didn't see the survey because she hangs out on the #lesswrong channel more than the main site. Another mostly just goes to meetups. So I think this represents only a small sample of people who could justly be considered Less Wrongers.

Yeah, this also fits my observations--I suspect that reading LW and hanging out with LW types in real life are substitute goods.

[-]Tenoke13y220

Some of the 'descriptions of LessWrong' can make for a great quote on the back of Yudkowsky's book.

[-]Pablo13y210

Obnoxious self-serving, foolish trolling dehumanizing pseudointellectualism, aesthetically bankrupt.

;-)

[-]FiftyTwo13y150

Pratchett always includes a quote that calls him a "complete amateur," so there is some precedent for ostentatiously including negative reviews.

1alfredmacdonald13y

I have always despised the term "pseudointellectualism" since there isn't exactly a set of criteria for a pseudointellectual, nor is there a process of accreditation for becoming an intellectual; the closest thing I'm aware of is, perhaps, a doctorate, but the world isn't exactly short of Ph.D.s who put out crap. There are numerous graduate programs where the GRE/GPA combination to get in is barely above the undergrad averages, for example.

2Nisan13y

I'd like to have one of these quotes in cross-stitch to hang on my wall. (Hint: Christmas is around the corner!)

[-]Cthulhoo13y210

LESSWRONG
LW

LESSWRONG
LW

120

2012 Survey Results

120

120

On IQ Accuracy: