For at least 2 years prior to January 2009, I procrastinated between 1-3 hours a day reading random internet news sites. After I created my first CT chart, I made the following prediction: "If I design a way to gain information about the world that does not involve reading internet news sites that also does not alter my way of achieving my other intrinsic goods, then I will stop spending time reading these internet news sites." The "does not alter my way of achieving my other intrinsic goods" was unpacked. It included: "does not alter my way of gaining social acceptance", "does not alter my relationships with my family members", etc. The specifics were unpacked there as well.

This was prediction was falsifiable - it would have failed if I had kept reading internet news sites. It was also bold - cogsci folk and good random human psychologists would have predicted no change in my internet news reading behavior. And it was also successful - after implementing the recommendation in January 2009, I stopped procrastinating as predicted. Now, of course there are multiple explanations for the success of the prediction, including "CT is true" and "you just used your willpower". Nevertheless, this is an example of a faisifiable, bold, successful prediction.

cogsci folk and good random human psychologists would have predicted no change in my internet news reading behavior.

Your model of human psychologists needs updating, then. Books on hypnotism that I read when I was 11 discuss needs substitution, secondary gain, etc. that would be relevant to making such a prediction. Any good human psychologist knows to look for what gains a behavior produces.

Of course, maybe you meant "good (random human) psychologists", not "good, random (human psychologists)" - i.e., psychologists who study th... (read more)

A Critique of Leverage Research's Connection Theory

by peter_hurford 6 min read20th Sep 201231 comments


Leverage Research is a recently formed New York-based research group "dedicate[d] to making the world a better place by the best and most effective means possible". I personally empathize and connect with this mission statement a lot, and I myself have made more or less an identical dedication personally, and through my involvement with the organizations GivingWhatWeCan and 80000 Hours.

One of the pieces of research developed by Leverage Research founder Geoff Anders (and presumably tested and developed by Leverage Research) is Connection Theory. Broadly, Connection Theory (CT) is a theory of mind that proports to explain and predict nearly all mental phenomena -- such as why people believe, desire, think, and act the way they do.

CT, if true, would be an amazing and profound success for psychology as a discipline. However, I don't think CT is true. Or rather, even if CT is true, I don't think the case made for CT is persuasive. In this essay, I intend to articulate my case against CT.



I don't write this essay with the intent to "hurt" Leverage Research or Geoff Anders, but rather with the intent to either:
(1) be persuaded for the truth of CT and understand how to use it to better make the world a better place, or
(2) successfully criticize CT and help an organization with a very laudable mission better accomplish it by correcting the errors they've made.

In many ways, I expect my criticism to "miss the mark" and misunderstand CT. In doing so, I hope to allow Leverage Research to clarify their theory in light of my objections to make it less misunderstood. In other ways, I expect my criticism to reveal flaws in CT that require revision and reassessment.

(Lastly, I make this essay with the disclaimer that I have no specialized knowledge or formal training in psychology, beyond pursuing an undergraduate degree in the field. I am currently in my Junior Year at Denison University.)


Connection Theory in More Detail

Connection Theory is a combination of the following four premises:

1. Everything a mind is aware of is (a) a sensation, (b) a spatial relation between sensations, (c) a representation, (d) an awareness of something, or (e) a combination of these.
2. Every mind is such that its representational content at a moment is determined entirely by it updating its representational content in the most elegant possible way on the basis of its representational content at the preceding moment, if it had any, and its current sensations, given the restriction that it believe that each of its intrinsic goods will be permanently achieved.
3. Every mind is such that at every moment it exists, it acts exactly in the way it believes will lead to each of its intrinsic goods being permanently achieved.
4. Every mind is such that something is one of its intrinsic goods if and only if it is one of that mind’s concepts and it is on the List of Intrinsic Goods.

Put more simply, CT asserts that every single person has certain fundamental desires about the way they want the world to be (usually just a few, like social acceptance and world peace), people always believe that their fundamental desires will eventually be satisfied, and that people will change their beliefs in completely rational ways except if such a change would cause them to stop believing their fundamental beliefs will be satisfied. CT also asserts that people will always act to best satisfy their fundamental desires, according to their beliefs.


The Evidence for Connection Theory

The Evidence for Connection Theory has been gathered in "Connection Theory: The Current Evidence" (link to an automatic download of a PDF file). The article mentions six pieces of evidence for CT: (1) Recommendation Tests, (2) Mind Mapping Tests, (3) Results from Everyday Use, (4) Results from Practical Application, (5) Explanation of Sociological Phenomena, and (6) Philosophical Arguments.

Recommendation Tests

The recommendation test starts with a CT chart -- that is, using Connection Theory to create a chart for a person that aims to explain their thoughts, feelings, and behavior by reference to their "intrinsic goods" (goals they fundamentally want to accomplish), "modes" (different attitudes toward life that can be triggered by various events), and "path diagrams" which connect various actions to instrumental goals that can eventually accomplish intrinsic goods.

The process for making a chart is given lengthy treatment in "Connection Theory: Theory and Practice" (link to an automatic download of the PDF) starting from page 32, though reading the whole document is probably necessary to understanding what is going on.

To make a long story short, the Recommendation Test involves creating the CT chart, then reviewing it to create a prediction in the form "If the participant takes {specified actions}, the participant will change in {specified way}". Using two different CT charts (one for Anders by Anders, the author of CT; and second for a friend of Anders by Anders), 60 predictions were made. Of those, 22 were tested, 16 came true, 5 were ruled too difficult to confirm, and 1 was ruled false but because of an error made in the first chart. This predictive success is taken as "strong evidence of CT".

Mind Mapping Tests

Mind Mapping Tests are a bit different -- for this test, one needs to generate a CT chart for a person again, but this time look through all the beliefs of the participant. For each belief, the CT theorist should be able to provide a "CT-compliant answer", ie an answer that shows the belief was derived either through "elegant updating" (being completely and ideally rational) or through a need to believe in order to believe one can still accomplish one's "intrinsic goods". Anders claims to have done CT charts for seven different people and has always been able to describe beliefs in CT-compliant ways. This is taken as "a moderate amount of evidence" for CT.


Unfortunately for the remaining four types of evidence for CT, very little elaboration is provided; instead, the existence of results is simply declared. For example:

Regarding the everyday use of CT, several people have used CT to achieved very noteworthy results in their everyday lives. People have improved relationships with their parents, kicked irrational fears of illness, maintained astonishing work ethics and more. Unfortunately, these results are not measured systematically enough to provide a lot of high-quality evidence in favor of CT. Still, it is better than nothing.

More distressingly, the existence of sociological explanations and philosophical arguments are not mentioned at all! Instead, there are promises that we'll see more, later...:

Just as CT can be used to explain psychological phenomena, it should be able to be used to explain sociological phenomena. I and some of my colleagues have begun attempting to give such explanations. So far, we have explained a few sociological phenomena. I will describe our results here in full in a later version of this document. Finally, there are a number of high-quality philosophical arguments that can be used to establish claims that relate to the central claims of CT. I will discuss these arguments fully in a later version of this document as well.


Objection #1: The Case For CT Is Very Incomplete and Inadequate

So what's wrong with CT? Well, first it's clear that a lot of the evidence provided for CT doesn't count. Obviously, we cannot accept the existence of "high-quality philosophical arguments" or the ability to "explain sociological phenomena" without that information actually being provided. Likewise, CT's "noteworthy results in [...] everyday lives" aren't actually documented -- we have to take Ander's word for it that these things occurred.

Mind Mapping and Recommendation Testing are more thorough, but still incomplete. For Mind Mapping, again Anders merely asserts that he's yet to find a belief that couldn't be explained in a CT-compliant way, though the beliefs surveyed and CT-compliant responses given are not listed.

Recommendation Testing is thus the only evidence for which usable data is given, however there is a lot of patchiness: Why were 38 predictions made left completely untested? What should we make of the 5 predictions ruled too difficult to confirm or deny? Why should 16 successes out of 60 predictions be taken as phenomenal results?


Objection #2: The Case For CT Uses Flawed Methodology

A second objection to CT is that the methodology used to produce CT is flawed. The gold standard for demonstrating a psychological theory is demonstrating explanatory power through a psychological experiment. For CT, no psychological experiments were provided and many confounding factors were left uncontrolled. For instance, in the Reccomendation Testing, one's improvement could be for a wide variety of reasons unrelated to the CT map.

The CT methodology also lends itself to significant issues of bias. The sample size in the Reccomendation Testing was only two participants -- the author of the study and someone closely related to the author. The participants could easily suffer from confirmation bias when constructing and using CT charts, guided by a conscious and/or subconscious wish for CT to be true.

A better standard for CT would be to evaluate over a dozen different charts of people who are unassociated with the author of the study and unaware of CT.


Objection #3: CT Conflicts With Established Science

Lastly, CT conflicts with established science. "Connection Theory: Current Evidence" concedes that documentation that beliefs can be changed from brain damage is one outstanding, yet to be investigated problem. Additional studies pile on in this direction to the point where postulating that people are ideally rational in all cases unrelated to their internal beliefs is quite implausible.

Between scope insensitivity, the affect heuristic, the anchoring effect, the priming effect, confirmation bias, clustering illusions, pareidolia, attentional biases, the availability heuristic, illusory correlations, the Dunning-Kuger effect, the Forer Effect, etc., enough cognitive biases have been documented to fill a book -- indeed, with Daniel Kahneman's Thinking Fast and Slow, they actually have! The myth of the rational person has been so thoroughly shattered in so many ways that don't ostensibly relate to intrinsic beliefs, that I don't think CT has much to stand on.



Thus CT should not be taken as a candidate for a psychological theory without significant improvement -- it is incomplete and inadequate, has flawed methodology, and conflicts well established science. For what it's worth, Leverage Research has had an appropriate lack of confidence in CT. None of their documentation asserts that CT is true, but rather makes it clear that the current evidence is circumstantial, and more data needs to be collected.

It's also worth noting that CT could be tested via replication, a process in which independent researchers seek to duplicate the results -- if the results can be duplicated, the theory is more likely to be true, and vice versa. Someone could follow the instructions and create a CT chart and see if it could be used to make accurate psychological predictions, especially under controlled circumstances. Personally, I don't want to invest the time needed to preform the replication until CT has stronger legs to stand on and has made an adequate case.

 (Note: This was cross-posted from my blog.)