So, I'll start things by picking up our conversation from the other thread. Rolf says he believes that the knife and bra clasp are among the strongest pieces of evidence against Knox and Sollecito; I responded by pointing to the recent independent review that strongly critiqued that evidence.
If [the independent experts'] main point is that the evidence doesn't meet the standard of scientific rigor, then I might not disagree with them on anything factual. Very little evidence, either way, does meet the standard of scientific rigor. Fingerprints never reach the standard of scientific rigor. DNA testing, as practiced, probably rarely if ever meets the standard of scientific rigor. Eyewitness testimony obviously can never come close to meeting the standard of scientific rigor. Heck, most science doesn't meet the standard of scientific rigor. We still need to evaluate evidence on its full merits.
Absolutely true -- but of course the reason scientific rigor exists is because errors happen when it isn't applied, and so we need to take this into account and scrupulously avoid overconfidence when evaluating such evidence. And, even more to the point here, Conti and Vecchiotti (the independent experts) don't merely say that it doesn't meet the standard of scientific rigor; they actually say specifically that it is "not reliable". This is important, because they were asked by the court to assess "the degree of reliability"; hence they could presumably have answered anywhere along a continuum -- and the answer they came back with was "pretty much zero".
So, to confirm that our initial analysis here diverges, around how much are you currently shifting based on the test results for the knife and for the bra clasp? For you, did either one shift P(guilt) by a factor of 100? 10? Not at all?
Roughly speaking, I would say that together, all the evidence against Knox and Sollecito, of which the bra clasp and knife are for me the overwhelming majority (everything else being near-negligible), shifted P(guilt) upward by about an order of magnitude -- a factor of 10, from a prior somewhere between 0.0001 and 0.001 to a posterior somewhere between 0.001 and 0.01. The Conti-Vecchiotti report cuts that down a bit, though not necessarily hugely, since their findings were pretty much what I already expected them to be.
You can guess my hypothesis for why the DNA tests came out the way they did. Do you have a specific alternative hypothesis or hypotheses you want me to consider? Is your main claim here that you believe the knife was accidentally contaminated in the laboratory, and the bra strap was accidentally contaminated in Kercher's room? Or is there a different alternative hypothesis I should consider first?
Yes, I think those would be my best explanations for those results, though there is significant uncertainty even about the attribution of the DNA in the first place. It's also possible that the knife was accidentally contaminated by investigators at Sollecito's apartment, that the clasp was contaminated in the lab, or that there was deliberate malfeasance applied to either of the items at some point; but these are less likely secondary possibilities, in my view.
I assume we're not going to quash any evidence, since we're a court of Bayes and not a court of law? That is, I'm proposing that whenever we want to exclude or diminish evidence, we should have a Bayesian reason for why the evidence doesn't really alter P(guilt). The proposal is partly because it's the correct Bayesian thing to do, and partly because trying to divine and mimic Italian criminal procedure and admissability rules would add (IMHO unnecessary) additional complexity.
Agree entirely. I would only add that a court of Bayes should always be more accurate than a court of law and never less.
Thanks k, today I'll give my thoughts on the knife. I'm sure there are some mistakes in my analysis below, but let's see if we can start to pinpoint areas of disagreement. Let "ddk.lc" be the specific hypothesis that the double-dna knife was accidentally contaminated in the laboratory, and "a.g" be the hypothesis that Amanda is guilty.
I want to estimate the base rate of lab cross-contamination in the late 2000's. Two observations:
Looks like Washington State only admitted to one case of laboratory cross-contamination in homicide cases
Recently, on the main section of the site, Raw_Power posted an article suggesting that we find "worthy opponents" to help us avoid mistakes.
As you may recall, Rolf Nelson disagrees with me about Amanda Knox -- rather sharply. Of course, the same can be said of lots of other people (if not so much here on Less Wrong). But Rolf isn't your average "guilter". Indeed, considering that he speaks fluent Bayesian, is one of the Singularity Institute's largest donors, and is also (as I understand it) signed up for cryonics, it's hard to imagine an "opponent" more "worthy". The Amanda Knox case may not be in the same category of importance as many other issues where Rolf and I probably agree; but my opinion on it is very confident, and it's the opposite of his. If we're both aspiring rationalists, at least one of us is doing something wrong.
As it turns out, Rolf is interested in having a debate with me on the subject, to see if one of us can help to change the other's mind. I'm setting this post up as an experiment, to see if LW can serve as a suitable venue for such an exercise. I hope it can: Less Wrong is almost unique in the extent to which the social norms governing discussion reflect and coincide with the requirements of personal epistemic rationality. (For example: "Do not believe you do others a favor if you accept their arguments; the favor is to you.") But I don't think we've yet tried an organized one-on-one debate -- so we'll see how it goes. If it proves too unwieldy or inappropriate for some other reason, we can always move to another venue.
Although the primary purpose of this post is a one-on-one debate between Rolf Nelson and myself, this is a LW Discussion post like any other, and it goes without saying that others are welcome and encouraged to comment. Just be aware that we, the main protagonists, will try to keep our discussion focused on each other's arguments. (Also, since our subject is an issue where there is already a strong LW consensus, one would prefer to avoid a sort of "gangup effect" where lots of people "pounce" on the person taking the contrarian position.)
With that, here we go...