Assuming for the moment that it's true that a skilled PUA trainer would beat an untrained person at this test, how much of that effect do you think is attributable to simply being more confident vs actually having a more accurate model of human social behavior?
PUAs themselves will admit to confidence being important... in meeting people, and in its being a foundation for everything they do. But it's not a magic bullet.
I've seen an excerpt of a talk that one gave who explained that when he started, he actually attained some success at opening (i.e. initiating contact) through delusional self-confidence... however, this wasn't enough to improve his success at "closing" (i.e., getting numbers, kisses, dates, etc.), because he still made too many mistakes at understanding what he was supposed to do to "make a move", or how he was supposed to respond to certain challenges, etc.
Remember, if the signal is too easy to fake, it's not very useful as a signal.
Oh, and for bonus points, for the fabricated set of techniques, you could use stuff taught by Scientology, just to make sure there's consensus that it's bogus ;)
I think it would be a better test to reverse the PUA recommendations, i.e., teach them things that the PUAs predict would flop. If they succeed anyway, it's a slam dunk for the confidence hypothesis. But I doubt they would.
Actually, one thing I saw on Mystery's show suggests to me that it might be sufficient to train someone poorly -- one trainee on the show couldn't get it through his head as to tthe proper use of negging, and went around insulting women with what, as far as I could tell, was total confidence. And of course, It didn't work, at all, while the other guys who both understood the idea and applied it with careful calibration, achieved much greater success.
In other words, I think confidence alone is insufficient to replace social calibration -- the PUA term for having awareness (or reasonably accurate internal predictions) of what other people are thinking or feeling about you, each other, and the overall social situation. The principal value of PUA social dynamic theories to PUA practice is to train the socially ill-calibrated to notice the cues that more socially adept people notice instinctively (or at least intuitively).
In other words, having a theory of "status" or "value" helps you to to know what to pay attention to, to help tune in on the music of an encounter, rather than being misled by the words being sung.
(Of course , I'm sure we all know people who come along and wreck the music by confidently singing a new and entirely inharmonious tune. This sort of behavior should not be confused with being socially successful.)
I think it would be a better test to reverse the PUA recommendations, i.e., teach them things that the PUAs predict would flop
I don't think that would be a fair test. Techniques that PUAs think would flop, I would probably agree with them in predicting they'd flop - It's easier to know that something doesn't work, than that it does work. So they would actually end up at a disadvantage relative to a person with natural confidence and no PUA training.
I would want my control group to be given techniques that are entirely harmless and neutral, or as close to it as is reasonably possible.
Followup to: Do you have High-Functioning Asperger's Syndrome?
LW reader Madbadger uses the metaphor of a GPU and a CPU in a desktop system to think about people with Asperger's Syndrome: general intelligence is like a CPU, being universal but only mediocre at any particular task, whereas the "social coprocessor" brainware in a Neurotypical brain is like a GPU: highly specialized but great at what it does. Neurotypical people are like computers with measly Pentium IV processors, but expensive Radeon HD 4890 GPUs. A High-functioning AS person is an Intel Core i7 Extreme Edition - with on-board graphics!
This analogy also covers the spectrum view of social/empathic abilities, you can think about having a weaker social coprocessor than average if you have some of the tendencies of AS but not others. You can even think of your score on the AQ Test as being like the Tom's Hardware Rating of your Coprocessor. (Lower numbers are better!).
If you lack that powerful social coprocessor, what can you do? Well, you'll have to run your social interactions "in software", i.e. explicitly reason through the complex human social game that most people play without ever really understanding. There are several tricks that a High-functioning AS person can use in this situation: