You Be the Jury: Survey on a Current Event

As many of you probably know, in an Italian court early last weekend, two young students, Amanda Knox and Raffaele Sollecito, were convicted of killing another young student, Meredith Kercher, in a horrific way in November of 2007. (A third person, Rudy Guede, was convicted earlier.)

If you aren't familiar with the case, don't go reading about it just yet. Hang on for just a moment.

If you are familiar, that's fine too. This post is addressed to readers of all levels of acquaintance with the story.

What everyone should know right away is that the verdict has been extremely controversial. Strong feelings have emerged, even involving national tensions (Knox is American, Sollecito Italian, and Kercher British, and the crime and trial took place in Italy). The circumstances of the crime involve sex. In short, the potential for serious rationality failures in coming to an opinion on a case like this is enormous.  

Now, as it happens, I myself have an opinion. A rather strong one, in fact. Strong enough that I caught myself thinking that this case -- given all the controversy surrounding it -- might serve as a decent litmus test in judging the rationality skills of other people. Like religion, or evolution -- except less clichéd (and cached) and more down-and-dirty.

Of course, thoughts like that can be dangerous, as I quickly recognized. The danger of in-group affective spirals looms large. So before writing up that Less Wrong post adding my-opinion-on-the-guilt-or-innocence-of-Amanda-Knox-and-Raffaele-Sollecito to the List of Things Every Rational Person Must Believe, I decided it might be useful to find out what conclusion(s) other aspiring rationalists would (or have) come to (without knowing my opinion).

So that's what this post is: a survey/experiment, with fairly specific yet flexible instructions (which differ slightly depending on how much you know about the case already).

For those whose familiarity with the case is low:

I'm going to give you two websites advocating a position, one strongly in favor of the verdict, the other strongly opposed. Your job will be to browse around these sites to learn info about the case, as much as you need to in order to arrive at a judgment. The order, manner, and quantity of browsing will be left up to you -- though I would of course like to know how much you read in your response.

1. Site arguing defendants are guilty. 

2. Site arguing defendants are innocent.

I've chosen these particular sites because they seemed to contain the best combination of fierceness of advocacy and quantity of information on their respective sides that I could find. 

If you find better summaries, or think that these choices reflect a bias or betray my own opinion, by all means let me know. I'm specifically avoiding referring you to media reports, however, for a couple of reasons. First, I've noticed that reports often contain factual inaccuracies (necessarily, because they contradict each other). Secondly, journalists don't usually have much of a stake, and I'd like to see how folks respond to passionate advocacy by people who care about the outcome, as in an actual trial, rather than attempts at neutral summarizing. Of course, it's fine if you want to read media reports linked to by the above sites.

(One potential problem is that the first site is organized like a blog or forum, and thus it is hard to find a quick summary of the case there. [EDIT: Be sure to look at the category links on the right side of the page to find the arguments.] If you think it necessary, refer to the ever-changing Wikipedia article, which at the moment of writing seems a bit more favorable to the prosecution. [EDIT: I'm no longer sure that's true.] [EDIT: Now I think it's true again, the article having apparently changed some more. So there's really no telling. Be warned.])

After you do this reading, I'd like to know:

1. Your probability estimate that Amanda Knox is guilty.
2. Your probability estimate that Raffaele Sollecito is guilty.
3. Your probability estimate that Rudy Guede is guilty.
4. How much you think your opinion will turn out to coincide with mine.

Feel free to elaborate on your reasoning to whatever degree you like.

One request: don't look at others' comments until you've done the experiment yourself!

For those whose familiarity with the case is moderate or high:

I'd like to know, as of right now:

1. Your probability estimate that Amanda Knox is guilty.
2. Your probability estimate that Raffaele Sollecito is guilty.
3. Your probability estimate that Rudy Guede is guilty.
4. How much you think your opinion will turn out to coincide with mine.
5. From what sources you've gotten the info you've used to arrive at these estimates.

Then, if possible, do the experiment described above for those with little familiarity, and report any shifts in your estimates.


Again, everyone should avoid looking at others' responses before giving their own feedback. Also, don't forget to identify your prior level of familiarity!

If the level of participation warrants it, I'll post my own thoughts (and reaction to the feedback here) in a later post. (Edit: That post can be found here.)

260 comments, sorted by
magical algorithm
Highlighting new comments since Today at 9:01 AM
Select new highlight date
Moderation Guidelinesexpand_more

On a cursory reading of Wikipedia the obvious interpretation is that Knox and Sollecito are innocent and Guede is guilty. I didn't go through all the sites so I don't know if this would qualify as a litmus test, and assigning probabilities in this state of knowledge would be extra work.

EDIT: Read comments and am surprised at how many estimates of "Knox and Sollecito probably didn't do it" have probabilities in the range of 40% attached that they did. If it were a binary judgment or a confidence interval, then yes we should avoid extreme probabilities and widen intervals to compensate for known overconfidence biases. But in this case the hypothesis space of equally plausible possibilities is large, and some low-probability symbols were used to write the message (multi-person rape-murder vs. single-person rape-murder, female rape-murder vs. male rape-murder). It may not always be easy to unravel crime scenes (though this one sounds pretty straightforward) but to focus on Knox or Sollecito seems like privileging the hypothesis.

Unless there's major prosecutorial evidence not in Wikipedia, then this seems like a case of paying too much attention to other people's opinions (the jury, in a case where we have further information that the verdict gained media attention as possibly inaccurate), and I would suggest that anyone who gave a probability higher than .15 be more arrogant in the future.

If, of course, I just haven't done enough reading, then ignore the above.

I would suggest that anyone who gave a probability higher than .15 be more arrogant in the future.

This does not mean my assigned probability was 15%. It means, "Even after accounting for fudge factors and people using different numbers to express similar emotional degrees of certainty, if you gave a number higher than FIFTEEN PERCENT it means you've got a MAJOR problem with paying WAY too much attention to really lousy evidence, what other people think, and the authority of idiots."

The most important lesson here has nothing to do with innocence and guilt, but with people's confidently paraphrasing the opinions of others.

How much the more so when those opinions aren't saved as text, both to give the interpreters another chance to parse properly and to introduce the possibility that they could be embarrassed if later shown wrong.

The main reason my estimate is so high is because

  • I know my information came from two heavily biased sites, and

  • I found the "innocent" site a lot easier to follow and therefore paid more attention to it, so I know my information is particularly biased in that direction.

That said, I did consider a more-arrogant probability of 0.25 or so. My caution in this case isn't on general principle, but because I have something of an old history of embracing cause celebre cases like this only to decide on further reading that the person I'm defending is guilty as hell.

I agree with Eliezer but like Maxwell's point about assigning extra probability to Knox and Sollecito because the guilty argument was so poorly formated. "She was convicted but I don't get why, perhaps I don't understand this."

That said I think 15% or less more then accounts for this uncertainty. I gave Knox a 6% probability.

Side note, I am surprised that more people are not assigning probability to the chance that none of them did it.

What is completely sad (besides this horrible murder case), is the inability of either website linked to present a coherent, rational argument. In fact, I haven't been able to find one website that reveals all facts and then explains them with their point of view in a rational (or even semi-rational) way.

I find this situation almost as depressing as the murder. I couldn't come to any conclusion based on the poor quality of reasoning used on most websites. Wikipedia, as usual, presents a decent collection of facts.

From the Wikipedia article I could only ascertain that Rudy Guede is very likely guilty. My probabilities for the other two being guilty are low (but have a lot of uncertainty), certainly not enough for me to feel that the verdict is correct.

I began with zero familiarity with the case.

  1. Knox: 8%
  2. Sollecito: 10%
  3. Guede: 95%
  4. Agree with komponisto: 80%

Rationale for considering Sollecito more likely than Knox: They're linked quite closely here, but there's enough confusion surrounding the case that I can't take that completely for granted. That being the case, it's unlikely-but-possible that one of Knox or Sollecito was directly involved while the other wasn't, and my prior for a male committing a violent rape/murder is a lot higher than for a female.

Guede is clearly guilty. He fled town immediately after the murder. His DNA was found in the victim's body, by far the most difficult-to-contaminate piece of DNA evidence in the case, making it extremely likely he's the rapist. Very low prior on a rape/murder being committed by separate parties.

The inconsistencies in Knox and Sollecito's stories are definitely worth paying attention to. But there are several factors diluting their importance:

  • I already had a reasonably high prior on the prevalence of brutality and corruption in Italian police forces. This doesn't leave me with much faith in their competence, especially when it comes to interrogations.
  • It's known that Knox and Sollecito were intoxicated with alcohol and marijuana at the time. I don't know how many of you have ever been thoroughly trashed on this duo (I'm guessing the number is disproportionately low here), but memories formed under such circumstances are very fragile. I've personally experienced disagreements provoked by divergent recollections of events that had transpired an hour previously. Given that sort of influence, I know I would have a damn hard time reconstructing a coherent narrative of the night's events after the brutal murder of my roommate and several days of interrogation by Italian police.

The DNA evidence against Knox and Sollecito is also worth paying attention to, and is in fact the primary reason my estimate on their guilt is as high as it is. However, this is partially mitigated by a glaring problem with the case: Most of the evidence was collected before Guede was a suspect. If Guede had been identified from the start, the Knox-Sollecito hypothesis may not have been quite so "privileged". IIRC, the DNA of four unidentified individuals was also found on the knife. Plus the doubt wrt the knife matching the wounds. Plus the odds that Sollecito would take a murder weapon back to his apartment and put it away. My generally unfavorable prior wrt Italy's justice system also adds a fair amount of room for uncertainty here.

The fact that Guede wasn't initially identified also provides the police with a mild motive for contaminating the evidence. It's unlikely that such an action would be detected. That said, my prior for this sort of action is pretty low.

Finally, we come to the elephant in the case: The hypothesis that the murder was committed by three people working together, as the result of a sex game gone bad. This reads like a parody of a flailing prosecutor. Contrast prior with that for a "normal" rape/murder... ouch. And it just so happens that the one party implicated by actual, solid evidence is the one party who fled after the crime.

The rationale for my 80% chance of agreement with komponisto is mostly based on "metagaming" his position. He admits to having a "rather strong" opinion on the case, and it seems much, much harder to form a strong opinion in favor of guilt as opposed to innocence in this case. My estimate would be higher, but given how asymmetrical this case appears to me, I can't rule out "initial counterintuitiveness" and/or extra evidence as a possible motivation for posting this in the first place.

I'll read the other comments and post my update later today.

I've read the other comments, and for the most part, my estimates haven't moved. However, this remark by Psychohistorian jumped out at me:

The fact that the prosecutor is under investigation for previously using crazy psychotic hypersexual homicidal maniac theories without basis does a lot to explain how the prosecutors ended up with such a crazy theory.

Wow, I definitely missed this when I read about the case. The prosecution's theory was already setting off alarm bells. If this is true, that shaves another percentage point or two off of my estimate of guilt for K+S.

In general, I'm a bit surprised how much faith some commenters have in their causal understanding of human behavior and psychology, when that understanding seems to be derived from a process of imagining what they would do in those circumstances. I know that I would certainly try like hell to maintain a coherent account of the night's events, but taking this kind of interpersonal analogy for granted when assessing such a delicate situation strikes me as willingly throwing oneself into the arms of the typical mind fallacy.

Some commenters seem particularly focused on some of the more arcane details of the case, e.g., the toilet. How much can you really infer from this sort of thing? I get the distinct impression that people are falling back on intuitions gleaned from detective shows and mystery novels, which by construction tend to involve cases that hinge on the little things.

The estimates I came up with were a lot less confident than yours, but on reading yours they seem more like my intuition. I think I got burned on the calibration test and now I'm avoiding extremes.

The facts as presented are not accurate so that is throwing off your calculation.

For example, when discounting Knox's statement to the police most people consider that it was after a length interrogation but the truth is the questioning lasted one hour. They accept that Knox was mistreated but all the evidence points to them treating her toughly but as expected for a murder suspect. What is never mentioned in the interrogation story is that Sollecito told the police Knox was not with him the night of the murder and that he lied at Knox's request. This information was in fact what led to Knox placing herself at the crime and accusing an innocent man. It should also be noted that this is the second innocent black man Knox had tried to implicate.

Also missing from consideration is the fact that Luminol revealed footprints matching both Amanda Knox and Raffaele Sollecito’s feet in the corridor between Knox’s room and the victim’s room and nowhere in the house. The prosecution advanced the theory that these were made by the accused in the victim’s blood while the defense presented the argument that these could have been made at a different time. When one of the footprints contained both Knox and victim’s DNA the expert argued independent deposit.

These are not equivalent explanations. The defense position requires that there be a reason for Knox and Sollecito to have blood on their feet. No such reason was ever given but since they had dated for less than two weeks if such an event had occurred it certainly would have been fresh in everyone’s memory. So not only does it require that there be an occurrence unrelated to the murder where these footprints were made but further both Meredith and Amanda had to deposit DNA presumably by spitting of some kind of nasal discharge in the exact same spot. We should also add a reason for why these prints only appear between the victim’s room and Knox’s room and not in the rest of the corridor. That or we accept the obvious that they were made the night of the murder in the victim’s blood.

Then there is a series of other evidence missing from the Wikipedia article. No mention of Knox lying to the police to delay the discovery of the body. No mention of Knox going from a state of joking around to hitting herself repeatedly when asked to provide her fingerprints for exclusionary purposes. No mention of Knox’s story not matching the evidence of what happened November 2. So yes if you remove all the evidence and change what you even bother to mention then Knox and Sollecito are innocent but that is not reality. If you want the real evidence I would suggest you read here http://themurderofmeredithkercher.com/ everything is sourced to primary sources and there is next to no opinion. Just the facts. See if you still feel the calculation comes out as not guilty.

Perhaps this is after all a litmus test for rationality, in a different sense than (I suspect) komponisto intended.

I mean, here I am looking at a 100+ comments thread, discussing a highly charged issue, and everyone is thoughtful, respectful, and willing to take others' points into account.

That is... extremely unusual.

Few of us have had much investment or interest in the case before, I'm thinking. Presented as an abstract almost-philosophical problem, with a common framework of epistemology, it's much easier to discuss well.

5% for the couple, 99% for the first convict. 90% that my probability estimate is close to yours in the sense that you think the two are innocent and the one is guilty.

I'd read a bit about this in the news, and I checked out those sites and wikipedia.

Given the fact that there is no evidence of prior acquaintance of the couple and the man, combined with the fact that the man did not attempt to implicate the couple despite the overwhelming evidence against him, make it very unlikely that they were involved. That, and one person being crazy/desperate/disturbed enough to commit a brutal rape-homicide is much, much more likely than one person and a completely unrelated couple he's never met before being disturbed enough to commit a rape homicide. The defense's response to forensic evidence appeared pretty strong, and the pro-guilt group did not seem like they tried to seriously rebut this (they mentioned that one defendan't DNA was on the bra strap, but failed to mention three other unidentified people's DNA on it as well).

The fact that the prosecutor is under investigation for previously using crazy psychotic hypersexual homicidal maniac theories without basis does a lot to explain how the prosecutors ended up with such a crazy theory.

My estimate of your position was (as I perceived it) largely independent of my own analysis. You mention a number of confounding factors as to why people are likely to be wrong about these things, with no mention of why they might be right, and you posted this in response to their conviction. It would be very surprising to choose a decision you agree with as an example of problems with human rationality.

Thanks, your comment has changed my mind about Knox and Sollecito's guilt.

It would be very surprising to choose a decision you agree with as an example of problems with human rationality.

While it may indeed be legitimate to wonder if my having posted this implies anything about my opinion, I'll note that I didn't in this post cite the jury's decision itself as an example of rationality failure, but merely indicated -- in the context of an erupting international controversy and ongoing Internet flamewars --that the whole topic is by nature fraught with obstacles to rationality.

Having lived for 14 years in Italy, my impression is that several commenters severely overestimate the rationality and fairness of the italian police force.

priors

Amanda Knox is guilty: 75%, Raffaele Sollecito: 75%, Rudy Guede: 25%. I shall abbreviate future percentage triples as 75/75/25.

No knowledge of the case before reading this post. My prior is due to my assumption that trial people know what they are doing, and on the fact that I imagined that the trial was trying to show that the guilty were K+S instead of G.

acquiring information

Reading about G's DNA, which should be rather good evidence: switching to 50/50/75. I contemplated switching all the way to 25/25/75, but I figured there had to be some reason for the new trial.

Reading about the police's claim that the murder was linked to a group sex game; thinking that this would be a ridiculous motive. This made me think that maybe the trial people didn't knew what they were doing after all. Switching to 25/25/80.

Finally realized that the trial was in fact trying to show that the guilty were K+S+G instead of just G, not K+S instead of G. Stopped keeping track of percentages for some reason.

Reading about the police switching from K+S+L to K+S+G, which lowered my esteem of the police even more.

Reading about the DNA of K+S, figured it was natural for a woman and her boyfriend to have DNA all over the woman's own house.

Still trying to understand who was G relative to the others. I think he's a robber now. Definitely not part of the group sex thing. Even worse feelings toward the police.

Over all, the truejustice website seems more emotional than the friends of K website, which surprises me. I would have expected the family of the victim to have calmed down after the original G trial, yet truejustice still seemed angry; and doesn't even seem to be ran by the victim's family at all. They should be a lot less emotional about this than K's friend, which seem to be a lot more clearheaded than truejustice is.

I'm now quite convinced of the innocence of K+S, although I'm too shy to give an actual percentage. 5/5/95, if not more extreme.

I think he's a robber now. Definitely not part of the group sex thing. Even worse feelings toward the police.

I like the rapist theory. It's not like Amanda was the only promiscuous American collegian around - birds of a feather... And who would rob an exchange student? No, a consensual meeting gone bad sounds like a far more common scenario to me.

Why would you murder someone just because she didn't wanted to have group sex?

And even if you did, why the hell would you call the police yourself?

Furthermore, this case was decided by a jury. A group of average people with no real juristic knowledge at all. Just some random dumbasses who couldn't care less and just want to get home (no, I'm not narcist. Just consider: 50% of all people are by definition dumber than the average, and a jury is randomly selected. ie the average jury ranges from IQ 80 to 120, while a professional judge should start at about 125.) Do you really believe they would come to the correct decision?

And just think of the charges against the prosecutor (about inventing crazy conspiracy theories) . .

I guess for me its 5/5/90 for K/S/G.

Why would you murder someone just because she didn't wanted to have group sex?

Not sure what you are thinking, but to clarify: I meant an encounter between just Meredith and Guede, nothing to do with Knox or Sollecito. I could see a single guy (Guede) expecting sex and then resorting to rape and then murder to cover it.

(And I suspect the jury is on average better than the populace, since just about every country gives into the temptation to lard on extra conditions like 'if you're a felon you forfeit forever rights such as being on a jury or voting', which would disparately affect the lower percentiles.)

Ah, I thought you referred to an encounter between all of them. In this case, I agree with you - that's also what I think happened.

Regarding the juries: I've read to many reports about bad juries than to believe in them, and the fact that they are almost certainly less educated than a judge still remains.