There is no No Evidence
7Yoav Ravid
8Yoav Ravid
4MalcolmOcean
6TAG
1waveBidder
3TAG
4Jiro
2MalcolmOcean
4AprilSR
10[anonymous]
1AprilSR
4[anonymous]
4AnthonyC
2ChristianKl
1AprilSR
2ChristianKl
4MondSemmel
New Comment
If we treat the “is” in Absence of Evidence is Evidence of Absence as an “implies” (which it seems to me to be) and then apply modus tollens to it, we get “if you don’t have evidence of absence, you don’t have absence of evidence” and it is precisely this bullshit that Zvi is calling. If you have evidence of absence, say so.
Two comments:
First, as Jiro said, "implies" replaces "is evidence of", not just "is".
But second, since this is a probabilistic statement, using logical "implies" and modus tollens isn't appropriate.
So it would be "Absence of Evidence suggests Absence" (absence makes absence of evidence more likely), and the contrapositive would be "Presence Suggests Evidence" (presence makes evidence more likely).
But it is not just any absence of evidence, it is absence of evidence when we expect to see evidence in the case where the hypothesis is true. The full statement would be: Absence of evidence is evidence of absence proportional to expected evidence given presence. If we wouldn't expect any evidence for some hypothesis even if it was true, then an absence of evidence wouldn't be any evidence of absence.
The reason "no evidence" claims are often fallacious (even when they don't ignore evidence, as the original post shows they often do), is because they tend to treat absence of evidence as evidence of absence when we expect an absence of evidence (e.g, because we didn't have time to collect evidence yet) and therefore shouldn't actually update on it.
I do think you're getting it correct in the There is no No Evidence and Exactly what evidence isn't there? sections.
I'm surprised to see this downvoted. This comment follows all the discussion norms. What about this comment would you "like to see less of"? If you think there's a mistake here, explain it, I'd like to know. (ETA: the first vote on the parent was a downvote, and it remained the only vote for about a day)
A belated thanks for this—as I also just commented to Jiro, it seems basically right. And I like your elaboration on Jiro's point, and clarifying where I am/was confused/not.
1You talk about real evidence, but you don't unpack "real". For my money, the crucial point is whether you made an active effort to find evidence. It's rather vacuous to say there is no evidence, meaning that it has not landed on your doorstep. And it's a great way of supporting biases.
The idea that you need to actively seek confirmation, and, much more importantly, refutation is central to the Popperian approach. The Bayesian approach seems strangely passive, by contrast.
In a Bayesian context, seeking evidence is about narrowing the probability distribution from what should be a relatively flat prior. One could probably make a case for not making a decision until the cost of putting it off outweighs the gain by decreasing the uncertainty.
"Narrowing" can be passive or active. Passive narrowing is going to be a lot less efficient than active narrowing. If you are in a position to wait for as long as it takes , that would not be a problem...but you aren't't. You also don't have a pre existing mental database of every possible hypothesis, or the ability to assign infinitesimal probabilities to them.
If we treat the “is” in Absence of Evidence is Evidence of Absence as an “implies” (which it seems to me to be) and then apply modus tollens to it, we get “if you don’t have evidence of absence, you don’t have absence of evidence” and it is precisely this bullshit that Zvi is calling. If you have evidence of absence, say so.
No. The "is" in that doesn't mean "implies". The "is evidence of" means "implies".
Modus tollens then gives you "if you don't have absence, you don't have absence of evidence", which is not subject to the semantic tricks in the post.
Hmmm this is a good point and not one I have a refutation to, although it does still seem to me that most of what I'm trying to get at here is still right! Belated thanks.
It's important to distinguish between different kinds of evidence. There's never no rational evidence, but there might be no scientific evidence. The issue comes in when people conclude that there's no rational evidence because there's no scientific evidence.
The problem with the "no scientific evidence" line of thought is it devolves to:
"We know nothing at all unless a credentialed scientist conducted a blind RCT at the cost of millions of dollars and years of time. And the only information we learned from the RCT was a probability update for a binary question. And the results had to be reviewed by peer scientists, manually, and then published in a high impact journal or they are not credible".
Otherwise we are going to pretend we know absolutely nothing and will continue to do things per decades old tradition. (Even though those traditions were never checked with the same algorithm)
This is not a system that will develop effective treatments for aging and human disease.
Moreover the math says it is morally evil and kills millions.
My proposed fix is we develop automated systems to do all the above and we have the algorithms reviewed by all of the above but then reuse the same automated systems for thousands of experiments. And instead of explicit rcts we usually find the data by mining or proxy experiments.
I definitely share your concern that evidence which isn't "scientific" matters, but I still think whether or not there is scientific evidence isn't entirely irrelevant to decision-making when we care about creating organizations that consistently make good decisions.
Currently, we definitely care far too much about scientific evidence, but I disagree that the concept is entirely bullshit.
I am not saying it is bullshit. But failing to consider information also has a cost. And for some fields, "consistently good decisions" may not even be possible.
Yes, but I think making the distinction that way is going to be much harder for many people outside this community. I know very few people who don't read this website who even have any sort of probabilistic conception of "evidence" and "belief," and I have had (very conventionally well educated in STEM fields) people get angry at me for talking about things that way.
We are familiar with the ideas of rational evidence, scientific evidence, legal evidence, and so on, screening off some types of evidence for certain purposes, but most people aren't, at least not explicitly, and as far as I can tell have no idea that that's what our societal institutions are doing or why.
What's "scientific evidence"?
Things you would refer to when writing a paper about the subject? When it comes to a question like whether COVID is airbone I doubt that there's no prior art that you would cite that point into the direction.
Eliezer isn't given an exact definition but it seems to me like he's pointing to scientific knowledge being knowledge that comes from scientific papers.
There's weak evidence for most scientific statements that you could make that comes from scientific papers.
The fact that most published papers about homeopathy find that homeopathy is effective is a type of scientific evidence. On the other hand the fact that most high quality trials that are published show that it has no effect suggests that overall we should believe that it has no effect. Especially, if we combine the evidence from the high quality trials with evidence we have about published physical theories.
I can't think of good examples that are revelant for this discussion where there's no scientific evidence (nothing is published in a paper that's Bayesian evidence).
I've gotten pretty exasperated by the "there is no evidence" issue in public health communication in the last year or so, and hence appreciate this post.
Side note: There's a screenshot of a twitter conversation in this post, which doesn't load on lesswrong.com, but does load when I copy the screenshot's URL into a new window.
Curated and popular this week
