If all beliefs in a Bayesian network are bounded away from 0 and 1, then an approximate update can be done to arbitrary accuracy in polynomial time.

The pathological behavior shows up here because there are two competing but mutually exclusive belief systems. And it is hard to determine when your world view should flip.

I hope that this makes it more interesting to you.

I like your second argument. But to be honest, there is a giant grey area between "non-replicable" and "fraudulent". It is hard to draw the line between, "Intellectually dishonest but didn't mean to deceive" and "fraudulent". And even if you could define the line, we lack the data to identify what falls on either side.

It is worth reading Time to assume that health research is fraudulent until proven otherwise? I believe that this case is an exception to Betteridge's law - I think that the answer is yes. Given the extraordinary efforts that the editor of Anaesthesia needed to catch some fraud, I doubt that most journals do it. And because I believe that, I'm inclined to a prior that says that non-replicability suggests at least even odds of fraud.

As a sanity check, high profile examples like the former President of Stanford demonstrate that fraudulent research is accepted in top journals, leading to prestigious positions. See also the case of Dr. Francesca Gino, formerly of Harvard.

And, finally, back to the line between intellectual dishonesty and fraud. I'm inclined to say that they amount to the same thing in practice, and we should treat them similarly. And the combined bucket is a pretty big problem.

Here is a good example. The Cargo Cult Science speech happened around 50 years ago. Psychologists have objected ever since to being called a pseudoscience by many physicists. But it took 40 years before they finally did what Feynman told them to, and tried replicating their results. They generally have not acknowledged Feynman's point, nor have the started fixing the other problems that Feynman talked about.

Given that, how much faith should we put in psychology?

The effect isn't large, but you'd lose that bet. See Nonreplicable publications are cited more than replicable ones.

It depends on subject matter.

For math, it is already here. Several options exist, Coq is the most popular.

For philosophy, the language requirements alone need AI at the level of reasonably current LLMs. Which brings their flaws as well. Plus you need knowledge of human experience. By the time you put it together, I don't see how a mechanistic interpreter can be anything less than a (hopefully somewhat limited) AI.

Which again raises the question of how we come to trust in it enough for it not to be a leap of faith.

Nobody ever read the 1995 proof.

Instead they wound up reading the program. This time it was written in C - which is easier to follow. And the fact that there were now two independent proofs in different languages that ran on different computers greatly reduced the worries that one of them might have a simple bug.

I do not know that any human has ever tried to properly read any proof of the 4 color theorem.

Now to the issue. The overall flow and method of argument were obviously correct. Spot checking individual points gave results that were also correct. The basic strategy was also obviously correct. It was a basic, "We prove that if it holds in every one of these special cases, then it is true. Then we check each special case." Therefore it "made sense". The problem was the question, "Might there be a mistake somewhere?" After all proofs do not simply have to make sense, they need to be verified. And that was what people couldn't accept.

The same thing with the Objectivist. You can in fact come up with flaws in proposed understandings of the philosophy fairly easily. It happens all the time. But Objectivists believe that, after enough thought and evidence, it will converge on the one objective version. The AI's proposed proof therefore can make sense in all of the same ways. It would even likely have a similar form. "Here is a categorization of all of the special cases which might be true. We just have to show that each one can't work." You might look at them and agree that those sound right. You can look at individual cases and accept that they don't work. But do you abandon the belief that somewhere, somehow, there is a way to make it work? As opposed to the AI saying that there is none?

As you said, it requires a leap of faith. And your answer is mechanistic interpretability. Which is exactly what happened in the end with the 4 color proof. A mechanistically interpretable proof was produced, and mechanistically interpreted by Coq. QED.

But for something as vague as a philosophy, I think it will take a long time to get to mechanistically interpretable demonstrations. And the thing which will do so is likely itself to be an AI...