Summary: Parapsychology (the study of unorthodox mental phenomena such as hypnosis or telepathy) is widely considered to be the study of something that doesn’t exist. Many parapsychology experiments are uncommonly good ways to practice science, since the experiments can be performed without expensive apparatus and require some understanding of statistics. Let’s do a few experiments and see if these things exist!
Tags: medium
Purpose: To learn how to do science on a confusing phenomenon.
Materials: Zenner cards are useful but a 52 deck can do in a pinch. Some variations may suggest special decks or other tools entirely.
Announcement Text: “The Control Group is Out of Control is a popular Scott Alexander article. In it, he examines parapsychology, the field of science that investigates such phenomenon as hypnosis, telepathy, and astral projection. If you think that psychic powers aren't real, then this is a fascinating field: it's a good example of how attempts to do science can give you the wrong answer, returning false positives. My science classes mostly focused on easily replicable experiments like testing acceleration due to gravity or the resistance different materials had to electricity. When studying something like telepathy or antidepressants where not every person responds the same way, how do you sort actual effects from statistical noise?
Designed by Karl Zener in the 1930s, Zener Cards are a set of twenty-five cards with one of five different visually distinct symbols on one side. Karl Z. used them as part of a series of parapsychology experiments attempting to prove psychic powers existed; the typical experiment involved a test subject attempting to guess which card was being held by the test administrator. The experiments initially showed that some people are naturally psychic, guessing correctly more often than chance. Later experiments concluded that this was entirely down to poor experimental design. So my question to you: are any of us psychic? What do you think you know, and how do you think you know it?
I'm bringing a couple packs of Zener cards and a couple packs of regular 52 decks. Let's do some science!”
Description: The basic zener card experiment works like this; one person (the Tester) takes a deck of 25 cards and shuffles them, then one by one draws a card from the top of the deck and asks another person (the Subject) to guess which symbol is on the card. If the Subject guesses correctly, this is theoretically an indication of telepathic ability.
Describe the experiment, ask for a volunteer, and do a couple of draws to show people how it works. Now, ask people to figure out how how they would determine if something weird was going on?
From here on, the meetup is largely in their hands. Helpful things to point out include:
- Before an experiment is run, ask people to state what they think will happen. If your audience is familiar with Bayes Theorem, ask for a couple of priors. If you want to do a more directed variation, this is a great time to explain how Bayes Theorem works. Explain that if people have complaints about the experimental procedure, now is the best time to voice their complaints and that pointing out flaws after the fact is poor form.
- Emphasize the distinction between “this proves psychic powers are real” and “this is evidence something weird is going on.” People can get very fixed on trying to explain why this doesn’t prove psychic powers are real and spend a lot of time trying to explain away statistics when the answer can be that nobody shuffled the deck. If you’re using the thumb on the scale variation, try not to smirk too much when you keep saying “evidence for something unusual” instead of “evidence for psychic powers.” If you're not using the thumb on the scale variation, smirking a bunch when you say "evidence for something unusual" provides a little bit of extra cover for people who are.
Variations: As written, once the basic experiment is explained then the meetup is largely in the attendees hands. However, you can take a stronger hand directing them or correcting them. This can ultimately look like a statistics lecture, and there isn’t anything particularly wrong with that if that’s what you’re going for. Still, try and at least have it be a hands-on lecture, and try to make it a good lecture. Practice the math yourself beforehand, have your notes handy if you don't work with it regularly, use an app like fatebook or a google sheet if you think that would help.
The thumb on the scale variation involves making sure something weird is happening. If you want to mess with your attendees, there are lots of ways to cheat the zener test. I suggest throwing in any of the following: stacking the deck such that it’s poorly shuffled and predictable, arranging a conspiracy between the Tester and the Subject to communicate via blinks or foot taps under the table, getting decks of cards with marked backs, arranging a reflective surface to be behind the Tester such that the Subject can see the cards, counting cards such that by the end of the deck it’s easier to guess, spotting the card in the reflection of the Tester's glasses, or snapping your fingers and developing actual psychic powers.
Some of these are actually in-scope for scientific problems! In particular, poor shuffling or counting cards can happen by accident and be picked up on by accident.
I have two important suggestions for the thumb on the scale variation. First, have some kind of distinguishing feature between you normally and you when cheating at cards. I use a nametag or index card stuck to my hat that says “The Hat Of Lies” but could probably stand to be more obvious about it. I think it's fine to wear The Hat Of Lies even if you're not using the thumb on the scale version. The important thing is that people don't feel like you're more likely to lie to them under normal circumstances.
Second, if you want your subterfuge to last a lot longer in the face of skeptics, cheat to lose. That is, instead of stacking the deck and then guessing correctly every time, stack the deck and guess wrong every time. That’s also statistically weird, someone paying attention to base rates should notice it’s weird, but in my experience people are way less likely to notice this is weird even if I say something like “gosh, it’s like I’m anti-psychic or something.” I suspect that making sure to have each trial come out exactly normal (that is, getting exactly the number of correct guesses that basic odds suggest you should, no more no less) would be even sneakier but you need to be sure people are going to repeat the experiment a lot (or have started by deciding there should be a lot of trials) for them to pick up on it. If you flipped a coin one thousand times and got exactly five-hundred heads and five-hundred tails, that is a little bit weird.
Other parapsychology experiments! Some of these are easier or harder to reproduce with materials you're likely to have in a normal meetup.
- Experimenter Effects and the Remote Detection of Staring!
- The Ganzfield Experiment!
- Hypnosis!
- Phrenology!
- Reflexology!
- Horoscopes!
A larger variation whose details are for the moment left as an exercise for the reader is doing any kind of science on any kind of effect. Ideally it’s something you can test in an average living room with equipment you have in your house, but plotting field trips is totally encouraged. Submitting a paper for publication is overkill, but if your group does produce a paper as a result of a meetup I declare that you win meetups for the day and you should totally brag about it.
Notes: The thing I like about this experiment is that the results are going to vary a bit. If someone gets the answer right six times out of twenty-five, is that a sign of psychic powers? (Or cheating?) How about seven times? How about ten? It’s possible someone just got lucky, but surely a proper Bayesian should update at some point towards something weird going on. This suggests a variation where you outright explain how Bayes Rule works before you begin.
A worthwhile note is that at some point when attempting to test if anyone is psychic, the attendees may suggest experiments totally unrelated to the Zener cards, like asking the subjects to guess what number they’re thinking of or to draw the scene they’re imagining. Some of these (like drawing the scene) are harder to quantify and do statistics to. That’s basically fine and can be fun. If you’re inclined to use the thumb on the scale variation then this might escalate into a game between you and the crowd. Lose gracefully when you get caught out. A good chunk of the fun here is in designing good experiments, so I suggest putting more energy into pointing out flaws in experimental design even if it’s obvious what you’re doing.
A further note on tone is that “haha, obviously ESP is fake, there’s no test that could change my mind” is not really the suggested angle to come into this with. “People who believe ESP might be real are idiots” is, I claim, completely wrong. Encourage people to take the idea seriously, at least long enough to test. What experimental result would change their mind? Invite them to perform the experiment, if it can be arranged! Ideally, people who believe in ESP and people who don’t believe in ESP should sit next to each other, talk about how science ought to be done, do some experiments, and if not change their mind then at least update in one direction or another. Plus, while this could be pure confirmation bias I’ve found that saying “only idiots believe in X” in a meetup is the surest way I know of to have someone standing right next to you who believes in X.
Credits: Zener cards (and the original experiment) were designed by Karl Zener.