Unknown unknowns

by Paul Crowley 1 min read5th Aug 201126 comments

11


Sorry if this seems incomplete - thought I'd fire this off as a discussion post now and hope to return to it with a more well-rounded post later.

Less Wrongers are used to thinking of uncertainty as best represented as a probability - or perhaps as a log odds ratio, stretching from minus infinity to infinity. But when I argue with people about for example cryonics, it appears most people consider that some possibilities simply don't appear on this scale at all: that we should not sign up for cryonics because no belief about its chances of working can be justified.  Rejecting this category seems to me one of the key foundational ideas of this community, but as far as I know the only article specifically discussing it is "I don't know", which doesn't make a devastatingly strong case.  What other writing discusses this idea?

I think there are two key arguments against this.  First, you have to make a decision anyway, and the "no belief" uncertainty doesn't help with that.  Second, "no belief" is treated as disconnected from the probability line; so at some point evidence causes a discontinuous jump from "no belief" to some level of confidence.  This discontinuity seems very unnatural.  How can evidence add up to a discontinuous jump - what happened to all the evidence before the jump?