This post reads like it's taken out of context from a larger conversation. (Also, the title commits relational projection fallacy - who needs to know, and for what purpose?)
It actually was from a larger conversation. Nancy brought it up at the last Philadelphia meet up. I don't think any relevant details are missing and we didn't come to any thoughts that haven't already been brought up in the comments here. Was some part unclear?
Yes. I didn't understand where this snippet was coming from and where it was going. (This is related to the relational projection part.)
I suspect that any "conscientiousness, benevolence, and loyalty" test which is both as accurate and as easy to administer as an IQ test would be much easier to game than the IQ test. And, of course, less conscientious would be more tempted to game it, by lying to themselves and others.
Out of curiosity, I tried this one and easily ended up in the top 10% of conscientiousness by lying to a few questions, but not to too many, to avoid suspicion. There is no way I could game a random IQ test in a similar fashion, however.
Does anyone know of reliable tests of personality traits other than IQ?
Well, personality surveys can get away with using blatantly loaded questions because there's not much motivation to lie on an anonymous survey (though it's not exactly unheard of anyway, and there are issues of self-image vs. behavior to consider). This obviously wouldn't fly if something like college admissions or a job application was riding on the results, but I'm not going to condemn the existing tests for failing outside of the domain they were designed for.
I can't think off the top of my head of any good ways to measure conscientiousness in such an accurate and finely-grained way that the results could be used like IQ tests are used; you could use secret-test-of-character methods like e.g. leaving out a basket of donuts with a donation box and a hidden camera, but that'd give you only one data point. But I'm not going to say that designing a test like that is impossible, either.
you could use secret-test-of-character methods like e.g. leaving out a basket of donuts with a donation box and a hidden camera
That also has the problem that these things are easily gamed if their famous enough.
I think this is the most important reason for IQ tests to exist:
I've underestimated people who were bright enough but didn't present as intellectual
This is what many people do, all the time. (Even to themselves.) In extreme case, a school suspects that one of their students is mentally retarded and sends them to a psychologist... and the psychologist finds out that actually the child has a very high IQ. But there are also people with high IQ who use their social skills to hide this part of their personality and become invisible in the crowd; probably because the alternative is being alone. What a waste of brainpower!
Without the IQ tests, most people would judge intelligence by similarity with Hollywood stereotypes. They certaintly wouldn't stop trying to divide people to smarter and dumber. There just wouldn't be any other scale to prove them wrong.
It seems to me that you can find out a lot about people's intelligence by talking with them a little, though I've underestimated people who were bright enough but didn't present as intellectual.
If this is true, then unstructured interviews should be a good way to determine how effective a candidate will be in a position. The literature is clear that unstructured interviews are worthless, and IQ testing is the best measure we have, typically explaining about half of the variance.
Lots of people have tried to dethrone IQ as a measure for a very long time, trying lots of things. They've never succeeded; IQ really is that good a measure of cognitive ability, and cognitive ability really is that important for almost everything.
I think this misses the point of the OP, which wasn't that IQ or intelligence can accurately be guessed in a casual conversation, but rather that intelligence can be guessed more accurately than other important parameters such as "conscientiousness, benevolence, and loyalty", for which we don't have tools nearly as good as those we have for measuring IQ. The consequence of which being, since we can't assess these as methodically, people can fake them more easily, and this has negative social consequences.
I think this misses the point of the OP
On a second read, I agree with you- I don't think I paid much attention to the third sentence, because the first two both rubbed me the wrong way. I have known people who turned out to be all hat and no cattle, intelligence-wise, and see that as a general phenomenon, and think verbal ability can be very distinct from mathematical/technical ability. There's significant anecdotal and statistical evidence for that.
for which we don't have tools nearly as good as those we have for measuring IQ
We have good measures of conscientiousness, but are either benevolence or loyalty single factors? Benevolence or loyalty to a single entity we have moderately good tests for, and it's not clear to me it's possible to do better without mindreading.
That unstructured interviews are useless only contradicts the conjunction that people can learn IQ from conversation and that the interviewers do learn it and that they choose to evaluate on it. It is a standard parlor trick to guess people's IQ based on five minutes of conversation.
That unstructured interviews are useless only contradicts the conjunction that people can learn IQ from conversation and that the interviewers do learn it and that they choose to evaluate on it.
Agreed.
The literature is clear that unstructured interviews are worthless, and IQ testing is the best measure we have, typically explaining about half of the variance.
Side note: my understanding is that the correlation is about one half, so about a quarter of the variance is explained.
I think NancyLebovitz is not saying IQ isn't important, but that its a lot easier to read than other traits.
The first question is, "Who is 'we' and why do we need to know?" Context matters a lot here.
In fact, it's practically all that matters. In some situations yes, IQ is what you actually want to know. In other situations, no, you care more about other things.
I've never heard of anyone saying "I thought that person was really intelligent, but they turned out not to be"
No-one seems to be questioning this, but it conflicts with my experience as I often overestimate people's intelligence. This may be a failure mode confined to me, but I often meet and talk with people, who I'll "code" as being quite intelligent because they're well-spoken and broadly knowledgeable, (and tall and attractive and popular and hello-there-Halo-Effect). I'll then have to reassess this at a later point, generally by seeing them "in action" trying to reason about something.
If I understand you correctly, you're saying:
IQ might be important, but its easy for people to tell what someone's IQ is, so its not something you need to concentrate on. Things like conscientiousness, benevolence, and loyalty are also important, but much more difficult to figure out, and people spend lots of effort trying figure out those traits.
Is that right?
Not exactly. My best guess is that trying to figure out conscientiousness, benevolence, and loyalty are so hard that people mostly trust or mistrust without very good reasons.
And the reason loyalty is on the list is that companies don't want embezzlers, but they don't want whistleblowers, either.
You say not exactly, but you seem to be agreeing and clarifying?
Also there are definite strongish conscientiousness signals, such as education level and grooming/dress.
I think this post could use more context. Your point seems interesting and novel, but I'm not 100% certain what it is or what question you're trying to address.
It seems to me that you can find out a lot about people's intelligence by talking with them a little
This only works with people less intelligent than yourself. Someone with intelligence comparable to yours who's good at smooth talking can easily convince you that he's much smarter than you.
Ugh, can we taboo "smarter" please. Are we just playing with status markers? Can't we just go do awesome things instead? If people start faking that, well "mission fucking accomplished," as xkcd put it.
Much of concern with IQ seems to be about status, or more generally is purely about evaluating people without a stated purpose of the evaluation. Is your suggestion to just evaluate people based on awesome accomplishments just your way of playing along with this game, trying to divert status from IQ to accomplishments? If not, the usefulness of your proposal likely depends a lot on the purpose for which we're ranking people: if you want to predict future performance in domain X, then past performance in domain X might well be superior to IQ; but to predict future performance in a different domain Y, IQ is probably still the best bet.
I predict that, ceteris paribus, people who just go do things will outperform people who talk about IQ all day. :)
Based on a diversity of high functioning folks I have seen, I think single parameter models are a hopeless waste of time.
I predict that, ceteris paribus, people who just go do things will outperform people who talk about IQ all day. :)
All day, sure- the trope of Mensa underachievers is based in reality. But people who did some very awesome things reserved some time for IQ, and I think avoiding talk of IQ in order to avoid the low-status of Mensa is as silly as talking about IQ because of the high-status of intelligence.
Based on a diversity of high functioning folks I have seen, I think single parameter models are a hopeless waste of time.
This isn't the right comparison, though- the question isn't diversity among high-functioning folks, but the difference between high-functioning and low-functioning folks. (Now, social skills, energy, and so on are important parameters that a more complete model would have- but that doesn't mean single parameter models aren't worth the time they take to populate.)
It's not as hard as it seems, is making cancer go away with the power of prayer awesome enough for you?
I'm probably more gullible than average, but I'm pretty sure that people less intelligent than me have done this when talking to me too. A few times, I've made an estimate that a fellow programming student is the same or slightly higher skill level than me based on talking with them, but then when we work on the same problems in class, I have an easier time than they do. (Not the same thing as intelligence, but related.)
You believe that you’re more intelligent than they are because you are able to do one task better than them (coding), yet it sounds like they were able to do another task better than you (being able to successful convince you that they were more intelligent). I’m not sure why the latter should be ruled out as a sign of intelligence.
Actually, what I'd been getting at was that if they could convince me they were a better programmer than they really were, I could probably also be convinced that someone was more intelligent than they really were by similar means. If someone did convince me they were more intelligent than they really were, I'd have a harder time finding out I'd been fooled, for the same reason you mentioned.
I wouldn't take fooling me as a sign of all that much intelligence, though. I don't check the things people say about themselves for reasonableness very carefully. (Either not enough mental RAM, or force of habit from not having had enough RAM in the past - social interactions take more mental effort for me than for the typical person.)
Well, unless everyone is capable of fooling you, the ability to do so would seem to indicate at least some skill. I’m not sure of the intelligence conversion rate between “capability of deceiving you” and “capability of showing they’re better than you at programming in the particular class you share,” but your realization that the person is actually better at the former and not the latter seems to suggest the individual has a different set of skills, rather than merely being less skilled.
After reading this, I realized that I'd been conflating how easily I believe random information people tell me that I don't have much of anything invested in and don't expect them to have reason to lie about (which would probably not take much skill, since it often happens by accident when people are trying to make a joke), and things I'm actually trying to evaluate. I do have a sense of when someone is trying too hard to sound impressive, so then being able to pull it off would indicate skill. I see what you mean now.
I find that I don't agree with this comment, though perhaps if I thought about it more I would..
I often categorize people as 10-points-smarter-than-me, 20-points-smarter-than-me, etc, just naturally as I go about my day, and I'm (currently) fairly confident of my evaluations.
Sometimes I can get a pretty good estimate by speaking with someone for 5 minutes -- but I'm aware this is heavily weighted towards verbal acuity, which is just one dimension. A high verbal acuity for me is a marker of high IQ, though average verbal acuity is not strong evidence either way. (I also understand someone can have very high verbal skills while missing some others, so there is an upper bound to what I can predict.) I'm recently studying signs of high social acuity, and I think I'm getting more perceptive at noticing and distinguishing levels that are one or two levels above mine.
Someone that is relatively deficient in one of these parameters can recognize higher levels by evaluating in hind-site how effective and original a particular solution, choice of words or behavior was. It's much easier to judge a behavior than come up with the best behavior yourself. For example, during a meeting I'll realize that someone is manipulating me or others very well. It's much easier to recognize that manipulation that to recreate it. Though it does take some compensatory experience -- since recognizing these manipulations can be as subtle as realizing that a awkward situation has been avoided or people feel better about something that you would have predicted, and determining the intention of the manipulation, and whether it was deliberate or accidental, makes the computation more noisy.
The main way that I judge an IQ higher than mine is if they are faster or more clear on something I've already thought about (this translates to slight increases in IQ) or can succeed at things that I cannot do well or cannot do at all (this is where someone would be at least a level ahead). For example, I'm fairly good at solving problems, and working within a given frame, but I am not very good at choosing problems, because I'm not very strong in selecting frames -- this is a higher order cognitive skill I cannot do well, and no amount of time will increase my success very much. Thus I seek out mentors that are "10 or 20 points above me" to help me with that bit... I consider myself to be 'borrowing' their IQ points for a very short amount of time, so that I can then go and work on a problem within the way they've chosen to frame it. (Again, while I'm not as good at picking a frame, I feel like I am competent at evaluating whether my problem can be well solved within one.)
What I find curious is how people successfully model people of higher IQ in fiction. For example, the doctor in House seems very witty. Is he modeled by someone at least that witty? Likewise with Sherlock Holmes. I realize that the scenarios are contrived, so that the intelligence is mostly illusory, but how intelligent must one be to write a story that convinces someone, say, more intelligent than themselves watching a film that the film is about someone vastly more intelligent than them?
I often categorize people as 10-points-smarter-than-me, 20-points-smarter-than-me, etc, just naturally as I go about my day, and I'm (currently) fairly confident of my evaluations.
So you don't have any independent way to verify your evaluations. Let's apply the outside view here: Would you trust the assessment of someone of average or below average intelligence about the relative intelligence of people smarter than him. Note that there exist entire cottage industries of quacks, charlatans, and scammers based on convincing people that someone is smarter than they really are.
The outside view is very good to apply, especially in this case where there hasn't been much independent validation and lots of opportunity for confirmation bias. However, I would and do generally trust the assessment someone else makes about the intelligence of someone else. (With the exception of any assessments based on politics or tribe affiliation.) I guess I agree with the OP that intelligence is fairly straightforward to estimate with secondary signals.
I'm not familiar with any charlatans or scammers being successful by pretending to be smarter than they were. People pretending to be smarter than they are, are usually pretty transparent. I suspect this is just availability bias, though, do you have any examples in mind?
I'm not familiar with any charlatans or scammers being successful by pretending to be smarter than they were. People pretending to be smarter than they are, are usually pretty transparent.
They are if you're smarter then they really are.
I suspect this is just availability bias, though, do you have any examples in mind?
Well, there's Yvain's tale of how he was almost convinced by Velikovsky's pseudohistory.
It seems you are using 'seeming smart' as interchangeable with 'convincing' or 'persuasive'?
However, these are quite independent. Someone can easily convince me of something, without my thinking they are more intelligent than I am, and without convincing me that they are more intelligent than they are.
Consider a 'smooth talker'. I think people generally recognize that these smooth-talkers are more likable and persuasive on any topic, but there is no necessary correlation with having a higher IQ. In fiction, there are extreme examples like Forest Gump (low IQ, very smooth) and innumerable moderate examples like Peter Venkman in Ghostbusters. Whereas intelligent characters are often portrayed, though not always, as not very persuasive.
...Smooth-talkers and scammers will often break-down defenses by signaling equal intelligence when they actually have higher intelligence.
In the example you gave, how do we know Velikovsky wasn't very intelligent? (We do know he had the ability to write very well, to make a false history seem true.) My question isn't that he is or wasn't intelligent, but whether his deception of Yvain was due to Yvain over-estimating his intelligence.
..Can you think of an example (a fictional one might be easiest) where a deception (or even any conflict) was actually about someone overestimating someone's intelligence?
I though the cartoon was a good example. The tiger convinced the boy that he was smarter than he actually was, with smooth talking.
It seems to me that you can find out a lot about people's intelligence by talking with them a little, though I've underestimated people who were bright enough but didn't present as intellectual.
You're also liable to perceive people with low social skills as less intelligent than they are, because the social situation is too hard a burden on their processing capacities.
It's not really surprising that people's intelligence seems to be rarely overestimated, though, is it? Smartness is impossible to fake, but you can fake stupidity.
But yes, of course, for various purposes, IQ is not the one thing that we need to know. Who would have doubted that?
Smartness is impossible to fake
The right jargon, and sounding like you know stuff (otherwise called being assertive), goes a long way.
Fluidly using the right jargon, and signaling that you 'know stuff' without sounding like you're trying hard too show that you know stuff, requires a fair amount of intelligence. (Incidentally, an inability to maintain a natural flow of conversation when someone knows a lot of stuff is one way highly intelligent people reveal that their social acuity is not that high. Their IQ may be extremely high, but a five minute interview can often easily identify these things.)
A certain degree of being articulate and appropriately assertive can be trained – I think I see this happen in the military. However, I don't think it's a fake signal, I think this training really results in greater general intelligence, or greater ability to succeed in any case.
You're also liable to perceive people with low social skills as less intelligent than they are
Also, you are likely to underestimate the intelligence of people who are not native speakers of your language (because language skills influence all data your heuristics get), or have some speech problems.
Is this based on theoretical reasoning or do you have anecdotal evidence for it? I'm genuinely curious because I don't know what to think on this issue.
It just follows from my model of the world, that the intelligence of a person correlates with their ability to react quickly, to make and understand jokes, to use a nuanced vocabulary, etc., and all these abilities are impaired when the person must struggle with the language or speech itself.
Of course there are also other things correlated with intelligence which don't depend on language skills. I'm not saying that an intelligent person will seem like a complete idiot just because they speak another language. It's more like a person with IQ 150 will seem like a person with IQ 120; and a person with IQ 120 may seem like a person with IQ 90. The difference would depend on their language skils.
I don't know if anyone tested this experimentally, but it should be easy. Take a few foreigners, give them some standardized English tests and IQ tests... then let them do some verbals tasks (e.g. tell a story) in front of the audience... then let the audience rank them according to their intelligence.
lt;dr -- Speaking slowly, making mistakes in difficult phrases, misunderstanding jokes... is evidence of low IQ. But it's also what foreigners do when speaking your language.
It's more like a person with IQ 150 will seem like a person with IQ 120; and a person with IQ 120 may seem like a person with IQ 90. The difference would depend on their language skills.
I am pretty sure that this doesn't happen. The reason is that when you speak a language so badly that you make those kinds of mistakes that make you appear stupid, you also have an accent - and that is a sign to the listener that they have to account for your being someone with potentially poor command of a foreign language.
This won't make the effect vanish completely, but I think it weakens it quite a bit so that it may not be a big issue in practice.
Incidentally, some people argue that for this reason, it's better to not even try to have no accent. I'm not sure I agree. For one thing, an accent puts a sort of ceiling on how people will perceive your intelligence - you cannot, for example, make witty puns when you have a strong accent, because people will think it was unintentional and won't give you credit for it.
And there's brains like mine, which insist that the pronouncedness of one's accent is correlated with intelligence. It probably is, though I suspect not as strongly as my brain thinks it is.
It's not really surprising that people's intelligence seems to be rarely overestimated, though, is it? Smartness is impossible to fake, but you can fake stupidity.
In a hypothetical world where social skills (presentation) and IQ are inversely correlated, where you don't know they are inversely correlated, and where you spend most of your time interacting with people in the 140+ IQ range, a person with a 120 IQ could come along and impress you to such a great extent that you immediately hypothesize that their IQ is in the 160+ range, until you see something of substance.
If intelligence is underestimated in some, and talking to someone leads you to believe you can estimate their IQ, and people with better social skills initially appear more intelligent, then it is possible to overestimate intelligence. You just start with the benchmark presentation of someone you know to be intelligent, and when someone who is less intelligent but presents better than them comes along, you overestimate their IQ.
Happens to me all the time... (practicing overestimation, not being overestimated). One of the biases I've struggled with most.
Keith Stanovich argues that IQ tests not only miss personality traits such as agreeableness and conscientiousness but also purely cognitive abilities. These other cognitive abilities he terms "rationality". His ideas are very well summarized here:
http://lesswrong.com/lw/2g1/what_intelligence_tests_miss_the_psychology_of/
Thompson makes some interesting comments on the topic of Stanovich: http://drjamesthompson.blogspot.com/2014/02/the-many-headed-hydra-of-alternate.html
It seems to me that you can find out a lot about people's intelligence by talking with them a little,
Well you can if you are (1) reasonably intelligent yourself; and (2) reasonably unbiased. When large numbers of persons must be assessed, for example in college admissions, both of these requirements can pose problems.
I've never heard of anyone saying "I thought that person was really intelligent, but they turned out not to be"
This is how I feel about most (but not all) of the people I've met from Less Wrong.
I think you're describing a selection effect; it's easy to notice when someone you don't think capable does something well, but it's harder to notice failures of people you think are intelligent. There are usually too many ways to give people the benefit of the doubt, and once you've started thinking someone's intelligent, you'll probably just keep thinking that.
This is how I feel about most (but not all) of the people I've met from Less Wrong.
There's probably a more general effect in play here: people are smarter in writing than they are extemporaneously. IME this is true of almost everyone, but it's especially true of people who are famous for their writing: Paul Graham, Steve Yegge, Eliezer Yudkowsky, people like that.
I think it has less to do with that, and more to do with being impressed by an initial command of concepts I'm not as strong with, followed by total incompetence in other areas. The idea behind rationality is that the approach is general.
I've never heard of anyone saying "I thought that person was really intelligent, but they turned out not to be", and when there are scandals about people with fake credentials, they don't seem to come from people with fake credentials making mistakes-- instead, someone checks the history.
It seems to me that you can find out a lot about people's intelligence by talking with them a little, though I've underestimated people who were bright enough but didn't present as intellectual.
The real problems are with identifying conscientiousness, benevolence, and loyalty-- that's where the unpleasant surprises show up.