Notes on Brainwashing & 'Cults'

by gwern15 min read13th Sep 2013104 comments


CultsSocial & Cultural DynamicsWorld Modeling

“Brainwashing”, as popularly understood, does not exist or is of almost zero effectiveness. The belief stems from American panic over Communism post-Korean War combined with fear of new religions and sensationalized incidents; in practice, “cults” have retention rates in the single percentage point range and ceased to be an issue decades ago. Typically, a conversion sticks because an organization provides value to its members.

Some old SIAI work of mine. Researching this was very difficult because the relevant religious studies area, while apparently completely repudiating most public beliefs about the subject (eg. the effectiveness of brainwashing, how damaging cults are, how large they are, whether that’s even a meaningful category which can be distinguished from mainstream religions rather than a hidden inference - a claim, I will note, which is much more plausible when you consider how abusive Scientology is to its members as compared to how abusive the Catholic Church has been etc), prefer to publish their research in book form, which makes it very hard to review any of it. Some of the key citation were papers - but the cult panic was so long ago that most of them are not online or have been digitized! I recently added some cites and realized I had not touched the draft in a year; so while this collection of notes is not really up to my preferred standards, I’m simply posting it for what it’s worth. (One lesson to take away from this is that controlling uploaded human brains will not be nearly as simple & easy as applying classic ‘brainwashing’ strategies - because those don’t actually work.)

Reading through the literature and especially the law review articles (courts flirted disconcertingly much with licensing kidnapping and abandoning free speech), I was reminded very heavily - and not in a good way - of the War on Terror.

Old American POW studies:

  • Clark et al 1981 Destructive Cult Conversion: Theory, Research and Practice
  • Lifton 1961 Thought Reform and the Psychology of Totalism
  • Ross & Langone 1988 Cults: What Parents Should Know
  • Schein, Schneier & Barker 1961 Coercive Persuasion
  • Singer 1978, 1979 “Therapy with Ex-cult Members” Journal of the National Association of Private Psychiatric Hospitals; “Coming Out of the Cults”, Psychology Today

Started the myth of effective brain-washing. But in practice, cult attrition rates are very high! (As makes sense: if cults did not have high attrition rates, they would long ago have dominated the world due to exponential growth.) This attrition claim is made all over the literature, with some example citations being:

  • Barker 1984, 1987 The Making of a Moonie: Choice of Brainwashing?; “Quo Vadis? The Unification Church”, pg141-152, The Future of New Religious Movements
  • Beckford 1981 “Conversion and Apostasy: Antithesis or Complementarity?”
  • Bird & Reimer 1982 “Participation rates in new religious movements and para-religious movements”
  • Robbins 1988 Cults, Converts and Charisma
  • Shupe & Bromley 1980 The New Vigilantes: Deprogrammers, Anticultists and the New Religions
  • Wright & Piper 1986 “Families and Cults: Familial Factors Related to Youth Leaving or Remaining in Deviant Religious Groups”
  • Wright 1983, 1987, 1988 “Defection from New Religious Movements: A Test of Some Theoretical Propositions” pg106-121 The Brainwashing/Deprogramming Controversy; Leaving Cults: The Dynamics of Defection; “Leaving New Religious Movements: Issues, Theory and Research”, pg143-165 Falling from the Faith: Causes and Consequences of Religious Apostasy
  • Wikipedia cites The Handbook of Cults and Sects in America, Hadden, J and Bromley, D eds. (1993)
  • a back of the envelope estimate for Scientology by Steve Plakos in 2000:

    In absolute numbers, that is from 8 million exposed to 150k active current, it means they’ve lost 7,850,000 bodies in the shop. That equates to a Retention Rate of 1.875%. Now, to be fair, over the course of 50 years “X” number of scientologists have dropped their bodies and gone off to Mars, etc,. who might still be members today if they weren’t dead We do not know what the mortality rate is for Scientologists. To significantly impact the RR, there would have to have been a 100% turn over in active membership due to generational shifting. There is no evidence that 150,000 active members of the CofS have died over the past 50 years. Beyond that, we would also need to apply the RR to deceased members to see what number would have continued beyond 15 years. Therefore, using the most favorable membership numbers and not discounting for lose of membership beyond the 15th year, we see a RR of 1.875%+“X”. If we assume that generational shifting accounts for a 10% turnover amongst current membership, that is, that the current membership would be 10% greater had members survived, X would equal 15,000 dead members, or, a total Retained Membership of 165,000. That would give the CofS a 50 year Retention Rate of 2.0625%.

Iannaccone 2003, “The Market for Martyrs” (quasi-review)

From the late-1960s through the mid-1980s, sociologists devoted immense energy to the study of New Religious Movements. [For overviews of the literature, see Bromley (1987), Robbins (1988), and Stark (1985).] They did so in part because NRM growth directly contradicted their traditional theories of secularization, not to mention the sensational mid-sixties claims God was “dead” (Cox 1966; Murchland 1967). NRM’s also were ideal subjects for case stud ies, on account of their small size, brief histories, distinctive practices, charismatic leaders, devoted members, and rapid evolution. But above all, the NRM’s attracted attention because they scared people.

…We have trouble recalling the fear provoked by groups like the Krishnas, Moonies, and Rajneeshees. Their years of explosive growth are long past, and many of their “strange” ideas have become staples of popular culture. [We see this influence not only in today’s New Age and Neo-Pagan movements, but also in novels, music, movies, TV shows, video games, university courses, environmentalism, respect for “cultural diversity,” and the intellectual elite’s broad critique of Christian culture.] But they looked far more threatening in the seventies and eighties, especially after November 18, 1978. On that day, the Reverend Jim Jones, founder of the People’s Temple, ordered the murder of a U.S. Congressman followed by the mass murder/suicide of 913 members of his cult, including nearly 300 children.

The “cults” aggressively proselytized and solicited on sidewalks, airports, and shopping centers all over America. They recruited young adults to the dismay of their parents. Their leaders promoted bizarre beliefs, dress, and diet. Their members often lived communally, devoted their time and money to the group, and adopted highly deviant lifestyles. Cults were accused of gaining converts via deception and coercion; funding themselves through illegal activities; preying upon people the young, alienated, or mentally unstable ; luring members into strange sexual liaisons; and using force, drugs, or threats to deter the exit of disillusioned members. The accusations were elaborated in books, magazine articles, newspaper accounts, and TV drama. By the late-1970s, public concern and media hype had given birth to anti-cult organizations, anti-cult legislation, and anti-cult judicial rulings. The public, the media, many psychologists, and the courts largely accepted the claim that cults could “brainwash” their members, thereby rendering them incapable of rational choice, including the choice to leave. [Parents hired private investigators to literally kidnap their adult children and subject them to days of highly-coercive “deprogramming.” Courts often agreed that these violations of normal constitutional rights were justified, given the victim’s presumed inability to think and act rationally (Anthony 1990; Anthony and Robbins 1992; Bromley 1983; Richardson 1991; Robbins 1985).]

We now know that nearly all the anti-cult claims were overblown, mistaken, or outright lies. Americans no longer obsess about Scientology, Transcendental Meditation, or the Children of God. But a large body of research remains. It witnesses to the ease with which the public, media, policy-makers, and even academics accept irrationality as an explanation for behavior that is new, strange, and (apparently or actually) dangerous.

…As the case stud ies piled up, it became apparent that both the media stereotypes (of sleep-deprived, sugar-hyped, brainwashed automatons) and academic theories (of alienated, authoritarian, neurotics) were far off mark. Most cult converts were children of privilege raised by educated parents in suburban homes. Young, healthy, intelligent, and college educated, they could look forward to solid careers and comfortable incomes. [Rodney Stark (2002) has recently shown that an analogous result holds for Medieval saints - arguably the most dedicated “cult converts” of their day.]

Psychologists searched in vain for a prevalence of “authoritarian personalities,” neurotic fears, repressed anger, high anxiety, religious obsession, personality disorders, deviant needs, and other mental pathologies. The y likewise failed to find alienation, strained relationships, and poor social skills. In nearly all respects - economically, socially, psychologically - the typical cult converts tested out normal. Moreover, nearly all those who left cults after weeks, months, or even years of membership showed no sign of physical, mental, or social harm. Normal background and circumstances, normal personalities and relationships, and a normal subsequent life - this was the “profile” of the typical cultist.

…Numerous studies of cult recruitment, conversion, and retention found no evidence of “brainwashing.” The Moonies and other new religious movements did indeed devote tremendous energy to outreach and persuasion, but they employed conventional methods and enjoyed very limited success. In the most comprehensive study to date, Eileen Barker (1984) could find no evidence that Moonie recruits were ever kidnapped, confined, or coerced (though it was true that some anti-cult “deprogrammers” kidnapped and restrained converts so as to “rescue” them from the movement). Seminar participants were not deprived of sleep; the food was “no worse than that in most college residences;” the lectures were “no more trance-inducing than those given everyday” at many colleges; and there was very little chanting, no drugs or alcohol, and little that could be termed “frenzy” or “ecstatic” experience (Barker 1984). People were free to leave, and leave they did - in droves.

Barker’s comprehensive enumeration showed that among the relatively modest number of recruits who went so far as to attend two-day retreats (claimed to be Moonies’ most effective means of “brainwashing”), fewer than 25% joined the group for more than a week, and only 5% remained full-time members 1 year later. Among the larger numbers who visited a Moonie centre, not 1 in 200 remained in the movement 2 years later. With failure rates exceeding 99.5%, it comes as no surprise that full-time Moonie membership in the U.S. never exceeded a few thousand. And this was one of the most successful cults of the era! Once researchers began checking, rather than simply repeating the numbers claimed by the groups, defectors, or journalists, they discovered dismal retention rates in nearly all groups. [For more on the prevalence and process of cult defection, see Wight (1987) and Bromley (1988).] By the mid-1980s, researchers had so thoroughly discredited “brainwashing” theories that both the Society for the Scientific Study of Religion and the American Sociological Association agreed to add their names to an amicus brief denouncing the theory in court (Richardson 1985).

Singer in particular has been heavily criticized; “Cult/Brainwashing Cases and Freedom of Religion”, Richardson 1991:

Dr. Singer is a clinical psychologist in private practice who earns a considerable portion of her income from cult cases. She has been an adjunct professor at the University of California at Berkeley, but has never held a paid or tenured-track position there. See H. Newton Malony, “Anticultism: The Ethics of Psychologists’ Reactions to New Religions,” presented at annual meeting of the American Psychological Association (New York, 1987) and Anthony, “Evaluating Key Testimony” for more details on Singer’s career.

…The [amicus curiae] brief further claimed that Singer misrepresents the tradition of research out of which terms like “thought reform” and “coercive persuasion” come. She ignores the fact that these earlier studies focus on physical coercion and fear as motivators, and that even when using such tactics the earlier efforts were not very successful. With great facility, Singer moves quickly from situations of physical force to those where none is applied, claiming that these “‘second generation’” thought reform techniques using affection are actually more effective than the use of force in brainwashing people to become members. Thus, Singer is criticized for claiming to stand squarely on the tradition of research developed by scholars such as Edgar Schein and Robert Lifton, while she shifts the entire focus to non-coercive situations quite unlike those encountered in Communist China or Korean prisoner of war camps. The brief points out, as well, that Singer ignores a vast amount of research supporting the conclusion that virtually all who participate in the new religions do so voluntarily, and for easily understandable reasons. No magical “black box” of brainwashing is needed to explain why significant numbers of young people chose, in the 1960s and 1970s, to abandon their place in society and experiment with alternative life styles and beliefs. Many youth were leaving lifestyles that they felt were hypocritical, and experimenting with other ways of life that they found to be more fulfilling, at least temporarily. Particularly noteworthy, but ignored by Singer, are the extremely high attrition rates of all the new religions. These groups are actually very small in numbers (the Hare Krishna and the Unification Church each have no more than two to three thousand members nationwide), which puts the lie to brainwashing claims. If “brainwashing” practiced by new religions is so powerful, why are the groups experiencing so much voluntary attrition, and why are they so small?

…Considerable research reported in refereed scholarly journals and other sources supports the idea that the new religions may be serving an important ameliorative function for American society. The groups may be functioning as “half-way houses” for many youth who have withdrawn from society, but still need a place to be until they decide to “return home.” Participation in some new religions has been shown to have demonstrable positive effects on the psychological functioning of individuals, a finding that Singer refuses to acknowledge.

“Overcoming The Bondage Of Victimization: A Critical Evaluation of Cult Mind Control Theories”, Bob and Gretchen Passantino Cornerstone Magazine 1994:

Neither brainwashing, mind control’s supposed precursor, nor mind control itself, have any appreciable demonstrated effectiveness. Singer and other mind control model proponents are not always candid about this fact: The early brainwashing attempts were largely unsuccessful. Even though the Koreans and Chinese used extreme forms of physical coercion as well as persuasive coercion, very few individuals subjected to their techniques changed their basic world views or commitments. The CIA also experimented with brainwashing. Though not using Korean or Chinese techniques of torture, beatings, and group dynamics, the CIA did experiment with drugs (including LSD) and medical therapies such as electroshock in their research on mind control. Their experiments failed to produce even one potential Manchurian Candidate, and the program was finally abandoned.

Although some mind control model advocates bring up studies that appear to provide objective data in support of their theories, such is not the case. These studies are generally flawed in several areas: (1) Frequently the respondents are not from a wide cross-section of ex-members but disproportionately are those who have been exit-counseled by mind control model advocates who tell them they were under mind control; (2) Frequently the sample group is so small its results cannot be fairly representative of cult membership in general; (3) It is almost impossible to gather data from the same individuals before cult affiliation, during cult affiliation, and after cult disaffection, so respondents are sometimes asked to answer as though they were not yet members, or as though they were still members, etc. Each of these flaws introduces unpredicatiblity and subjectivity that make such study results unreliable…The evidence against the effectiveness of mind control techniques is even more overwhelming. Studies show that the vast majority of young people approached by new religious movements (NRMs) never join despite heavy recruitment tactics. This low rate of recruitment provides ample evidence that whatever techniques of purported mind control are used as cult recruiting tools, they do not work on most people. Even of those interested enough to attend a recruitment seminar or weekend, the majority do not join the group. Eileen Barker documents [Barker, Eileen. New Religious Movements: A Practical Introduction. London: Her Majesty’s Stationery Office, 1989.] that out of 1000 people persuaded by the Moonies to attend one of their overnight programs in 1979, 90% had no further involvement. Only 8% joined for more than one week, and less than 4% remained members in 1981, two years later:

. . . and, with the passage of time, the number of continuing members who joined in 1979 has continued to fall. If the calculation were to start from those who, for one reason or another, had visited one of the movement’s centres in 1979, at least 999 out of every 1,000 of those people had, by the mid-1980s, succeeded in resisting the persuasive techniques of the Unification Church.

Of particular importance is that this extremely low rate of conversion is known even to Hassan, the best-known mind control model advocate whose book [Hassan, Steven. Combatting Cult Mind Control. Rochester, VT: Park Street Press, 1990?] is the standard text for introducing concerned parents to mind control/exit counseling. In his personal testimony of his own involvement with the Unification Church, he notes that he was the first convert to join at the center in Queens; that during the first three months of his membership he only recruited two more people; and that pressure to recruit new members was only to reach the goal of one new person per member per month, a surprisingly low figure if we are to accept the inevitable success of cult mind control techniques.

Objection: High Attrition Rates Additionally, natural attrition (people leaving the group without specific intervention) was much higher than the self-claimed 65% deprogramming success figure! It is far more likely a new convert would leave the cult within the first year of his membership than it is that he would become a long term member.

Gomes, Unmasking the Cults (Wikipedia quote):

While advocates of the deprogramming position have claimed high rates of success, studies show that natural attrition rates actually are higher than the success rate achieved through deprogramming

“Psychological Manipulation and Society”, book review of Spying in Guruland: Inside Britain’s Cults, Shaw 1994

Eventually Shaw quit the Emin group. Two months later he checked in with some Emin members at the Healing Arts Festival, a psychic fair. He avoided many Emin phone invitations for him to attend another meeting. He discovered that most, if not all, of the people who joined with him had dropped out. This is consistent with what Shaw has noted about most cults and recruits: the dropout rate is high.

Anthony & Robbins 1992, “Law, Social Science and the ‘Brainwashing’ Exception to the First Amendment”:

Lifton and Schein are also characterized in Molko (54) as attesting to the effectiveness of brainwashing, although Schein, an expert on Chinese coercive persuasion of Korean War POWs, actually thought, as do a number of scholars, that the Chinese program was relatively ineffective (Schein, 1959, p. 332; see also Anthony, 1990a; Schefiin & Opton, 1978)…Schein appears to actually have considered the communist Chinese program to be a relative “failure” at least, “considering the effort devoted to it” (Schein, 1959, p. 332; Anthony, 1990a, p. 302)…Various clinical and psychometric studies of devotees of well-known “cults” (Ross, 1983; Ungerleider & Wellisch, 1979) have found little or no personality disorder or cognitive impairment.

  • Ross 1983. “Clinical profile of Hare Krishna devotees”, American Journal of Psychiatry
  • Schein, E. (1959). “Brainwashing and totalitarianization in modern society”. World Politics, 2,430441.
  • Ungerleider, T., & Wellisch, D. K (1979). “Coercive persuasion (brainwashing), religious cults, and deprogramming”. American Journal of Psychiatry , 136,3,279-82.

“Brainwashed! Scholars of cults accuse each other of bad faith”, by Charlotte Allen, Lingua Franca Dec/Jan 1998:

Zablocki’s conversion to brainwashing theory may sound like common sense to a public brought up on TV images of zombielike cultists committing fiendish crimes or on the Chinese mind control experiments dramatized in the 1962 film The Manchurian Candidate. But among social scientists, brainwashing has been a bitterly contested theory for some time. No one doubts that a person can be made to behave in particular ways when he is threatened with physical force (what wouldn’t you do with a gun pressed to your head?), but in the absence of weapons or torture, can a person be manipulated against his will?

Most sociologists and psychologists who study cults think not. For starters, brainwashing isn’t, as Zablocki himself admits, “a process that is directly observable.” And even if brainwashing could be isolated and measured in a clinical trial, ethical objections make conducting such a test almost unthinkable. (What sort of waivers would you have to sign before allowing yourself to be brainwashed?) In the last decade, while brainwashing has enjoyed a high profile in the media-invoked to explain sensational cult disasters from the mass suicide of Heaven’s Gate members to the twelve sarin deaths on the Tokyo subway attributed to the Aum Shinrikyo cult-social scientists have shunned the the term as a symptom of Cold War paranoia and anticult hysteria. Instead, they favor more benign explanations of cult membership. Alternatives include “labeling” theory, which argues there is simply nothing sinister about alternative religions, that the problem is one of prejudicial labeling on the part of a mainstream culture that sees cult members as brainwashed dupes, and “preexisting condition” theory, which posits that cult members are people who are mentally ill or otherwise maladjusted before they join. (A couple of scholars have even proposed malnutrition as a preexisting condition, arguing that calcium deficiency may make people prone to charismatic susceptibility.)

Thus, when Zablocki published an indignant 2-part, 60-page defense of brainwashing theory in the October 1997 and April 1998 issues of Nova Religio, a scholarly journal devoted to alternative belief systems, he ignited a furor in the field. Pointing to the “high exit costs” that some cults exacted from those who tried to defect-shunning, forfeiture of parental rights and property, and veiled threats-Zablocki argued that these were indications of brainwashing, signs that some groups were using psychological coercion to maintain total control over their members. Although he admitted he could not prove brainwashing empirically, he argued that at the very least brainwashing should not be dismissed out of hand.

…Zablocki’s colleagues were unimpressed. In a response also published in Nova Religio, David Bromley, a sociologist at Virginia Commonwealth University who has studied the Reverend Sun Myung Moon’s Unification Church, complained that in Zablocki’s formulation brainwashing remained a vague, slippery, limiting, and ultimately untestable concept. Moreover, he pointed out, cults typically have low recruitment success, high turnover rates (recruits typically leave after a few months, and hardly anyone lasts longer than two years), and short life spans, all grounds for serious skepticism about the brainwashing hypothesis. Even if you overlook these facts, Bromley added, “the extraordinarily varied cultural origins, patterns of organizational development, and leadership styles of such groups pose a problem in explaining how they seem to have discovered the same ‘brainwashing’ psycho-technology at almost precisely the same historical moment.” A quick survey of the field reveals that Bromley is far from being the only doubter. Eileen Barker, a sociologist at the London School of Economics who has also studied the Unification Church, says, “People regularly leave the Moonies of their own free will. The cults are actually less efficient at retaining their members than other social groups. They put a lot of pressure on them to stay in-love-bombing, guilt trips-but it doesn’t work. They’d like to brainwash them, but they can’t.”

…To further complicate matters, researchers often bring very different, even conflicting approaches to their work. Psychologists, for example, tend to emphasize how a repeated environmental stimulus can elicit a conditioned response-depriving subjects of their autonomy. Sociologists, by contrast, typically endorse a voluntarist conversion model for religion, which posits that people join cults for generally rational reasons connected to the group’s ability to satisfy their needs: for a transcendent theology; for strong bonds of kinship and solidarity; for enough social support to enable them to quit drugs or otherwise turn their personal lives around. (For example, one study has shown that schizophrenics who joined cults functioned better than those who tried drugs or conventional psychotherapy.)

…In 1980 the New York state legislature, over objections from the American Civil Liberties Union, passed a bill that would have legalized deprogramming (it was vetoed by Governor Hugh Carey). “With deprogramming-with parents having their children abducted and held captive-the whole thing became intensely emotional,” says Thomas Robbins. “Who were the kidnappers: the parents, the cults, or the police? There were hard feelings on both sides.” Among the most outraged were social scientists who had never believed that people could be brainwashed into joining cults and who, as good civil libertarians, were appalled by deprogramming. Ofshe and Singer’s scholarly testimony (and fat fees) distressed a number of these scholars, whose credentials were equally respectable and whose own research had led them to conclude that coercive persuasion was impossible in the absence of some sort of physical coercion such as prison or torture.

…Zablocki made another, potentially more damning charge, however-one that Robbins did not take up. A significant amount of cult money, he wrote, has gone to scholars-in support of research, publication, conference participation, and other services. Zablocki did not name names. But a number of professors freely admit that nontraditional religions (in most cases, the Unificationists and Scientologists) have cut them checks. The list includes some of the most prominent scholars in the discipline: Bromley, Barker, Rodney Stark of the University of Washington, Jeffrey Hadden of the University of Virginia, and James Richardson, a sociologist of religion at the University of Nevada at Reno. All five have attended cult-subsidized conferences, and Bromley, Hadden, and Richardson have occasionally testified in court on behalf of cults or offered their services as expert witnesses against brainwashing theory. “This is an issue,” Zablocki wrote sternly, “of a whole different ethical magnitude from that of taking research funding from the Methodists to find out why the collection baskets are not coming back as heavy as they used to.”