This is a special post for quick takes by Richard_Ngo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
What is the empirical track record of your suggested epistemological strategy, relative to Bayesian rationalism? Where does your confidence come from that it would work any better? Every time I see suggestions of epistemological humility, I think to myself stuff like this:
What predictions would this strategy have made about future technologies, like an 1890 or 1900 prediction of the airplane (vs. first controlled flight by the Wright Brothers in 1903), or a 1930 or 1937 prediction of nuclear bombs? Doesn't your strategy just say that all these weird-sounding technologies don't exist yet and are probably impossible?
Can this epistemological strategy correctly predict that present-day huge complex machines like airplanes can exist? They consist of millions of parts and require contributions of thousands or tens of thousand of people. Each part has a chance of being defective, and each person has a chance of making a mistake. Without the benefit of knowing that airplanes do indeed exist, doesn't it sound overconfident to predict that parts have an error rate of <1 in a million, or that people have an error rate of <1 in a thousand? But then the math says that airplanes can't exist, or should immediately crash.
Or to rephrase point 2 to reply to this part: "That will push P(doom) lower because most frames from most disciplines, and most styles of reasoning, don't predict doom." — Can your epistemological strategy even correctly make any predictions of near 100% certainty? I concur with habryka that most frames don't make any predictions on most things. And yet this doesn't mean that some events aren't ~100% certain.