Fundamental Uncertainty: Alternate Framework and Pointwise Reduction

StanislavKrym

In this review of Gordon Seidoh Worley's book, Fundamental Uncertainty, I would like to explain why its main thesis is only partially rational.

Worley's thesis

Our knowledge of the truth is fundamentally uncertain because of epistemic circularity caused by the Problem of the Criterion.
We manage fundamental uncertainty by making pragmatic assumptions that lead us to believe in the truth of claims that help us achieve our goals.
Consequently, the truth that can be known is not independent of us, but rather dependent on that for which we care.
That truth is fundamentally uncertain and grounded in care has far-reaching implications for many of the world's hardest-to-solve problems.

While I mostly agree with points 1 and 2, the points 3 and 4 imply that truth is contingent on care, which I find unlikely.

The book itself justifies Worley's main thesis by detailed arguments described in the collapsible section.

Chapterwise summary of Worley's arguments

Chapter 1 defines epistemology, then proceeds to:
a. Tear down the naive epistemology where it's obvious which statements are true and which aren't;
b. Argue that logical epistemology is a branch of mathematics and has its flaws, like failing to account for observation error, Godel-proven incompleteness and circularity;
c. Fundamental uncertainty allegedly blocks mankind from finding out truth in some cases.^[1]
Chapter 2 makes the case that our ability to know the truth is undermined by meanings of words:
a. Words may have overlapping meanings ^[2](e.g. "hot" can describe the temperature of water, the spiciness of food, the attractiveness of a person, or the popularity of a song);
b. They also can have obsolete^[3] meanings (e.g. calling whales fish);
c. Words can also be generally fuzzy (e.g. "cold" is below a certain temperature dependent on the speaker).
d. According to the author, expert systems have failed to become useful because they didn't solve the problem of grounding symbols in context.
e. Words' representations are grounded in experience, causing different minds to interpret words differently.
Chapter 3, "Why can't we all agree what's good?" has the author introduce the question, then explain why we disagree and why we disagree on what is good or bad:
a. Rational disagreement has the author claim that sufficiently rational people are intuitively supposed to come to an agreement, but the intuition fails because even the power of Bayes' theorem fails to make them completely agree if they had different priors or fails to dismantle a stupid ABSOLUTE prior.
b. Moral foundations are somehow different because of different prior beliefs. Haidt conjectured that humans have six moral axes which they weigh differently.
Chapter 4, "Why don't we always do what we should?", makes two points:
a. Our mind consists of System 1 responsible for immediate actions bringing short-term satisfaction and System 2 responsible for long-term benefits;
b. It is hard to assess new formal systems for soundness or performing better than the old formal system. In fact, no sufficiently capable system can prove its own soundness.
Chapter 5, "What does it mean to know?", has the author define The Many Ways Of Knowing, then makes the point that we can know things well enough, but can't fully justify our knowledge: the proofs that establish Bayesian idealness are mathematical arguments; this requires us to trust that mathematical proofs correctly establish what's true. According to the author, that's itself a claim that needs to be proven if we want to completely trust our knowledge of the truth.
Chapter 6, "How do we know what we know?", has the author describe the Problem of the Criterion: to know if a claim is true, we need a method for testing if it's true, but how can one check the method itself? The closest thing that mankind has to a solution is using what works and assuming that particular claims, like the rules of logic or Peano arithmetic, are true.
Chapter 7: Why care about knowing the truth? The author's argument is twofold: there are systems which steer towards an outcome and successful steering for goals like survival, reproduction or anything else requires having an accurate world model.
Chapter 8: Why does fundamental uncertainty matter? According to the author, it matters because the modern world is steeped in many problems which cannot be solved unless we understand uncertainty. Examples include the Culture War, moral uncertainty, the crisis of meaning, metaphysics, Goodhart's curse and AI-induced existential risk. The Culture War is actually a fight over deeply-held values; moral uncertainty allows for compromises (which the author calls moral trade); the crisis of meaning is due to our life becoming oversaturated with choices which is incurable while being consistent with life having a meaning; Goodhart's Curse is the curse of over-optimisation for narrow criteria, and the ASI could end up enacting it unless it understands the fragility of human values.
Chapter 9: How do we live with fundamental uncertainty? One of the results originating in fundamental uncertainty is that most people are confused about what's normal: they have a poor understanding of truth, mistake the relative for the absolute, value episteme over other ways of knowing. Instead, "the truth is intersubjective, arising from the intersection of ourselves and the world as we find it", but the next subsection has the author argue that "The truth will still be there, waiting for you to know."

Alternate Framework: Perfect Bayesians, Imperfect Imitators

Consider a perfect Bayesian reasoner who was never exposed to images (e.g. if it was raised on math, texts, coding tasks), then was tasked with roleplaying a human brain, i.e. receiving images and other sensory signals and outputting orders for muscles to contract or relax so that the agent could interact with the environment and achieve some pre-set complex goal. Suppose also that every time a sensor sends a signal (e.g. a camera's pixel registers an RGB value) the sensor has an unknown probability to malfunction, outputting a random value instead of the truth and that the malfunctions are independent events.

The two main problems with a perfect Bayesian are that it uses infinite compute and that it can reason about too much. For example, if the Bayesian was tasked with winning a chess game, it would be able to evaluate entire huge sets of potential positions and arrive at a perfect strategy which would cause it to win once the opponent makes a theoretically fatal mistake unnoticeable even to human grandmasters. Infinite compute also lets a perfect Bayesian, for example, not just reason about sociological laws or potential ethoses of alien civilisations (e.g. coming up with arguments^[4] like "Human Values Are Contingent" or "Curiosity Isn’t Convergent"), but check one's conjectures by simulating such civilisations in detail.

A real-world reasoning agent, like a human brain or an LLM, doesn't have infinite resources. Instead, it is forced to rely on heuristics, which are supposed to make it check far less conjectures, and to optimize said heuristics. Similarly, categorization of objects becomes training a classifier and trying to make sense of how to deal with OOD cases (see, e.g. my discussion of Chapter 8).

Chapterwise critique

Chapters 1-2 are almost entirely justified in their reasoning about the sources of uncertainty in observations and words spoken by others. The first important problem emerges in Chapter 3 where the author talks about Aumann's theorem.

Chapter 3. World models built from slightly different priors have predictions close to convergent

Given the same evidence, Bayesian observers are more likely to converge to similar conclusions than the author assumes.

Consider a coin which has an unknown probability to land heads and flip it times. Suppose that before the experiment a Bayesian had the priors that assigned weight to the region . If the experiment has the coin land on heads times, then the weight of that region becomes .

The ratio is at most and rapidly decays when is away from . If after the experiment the Bayesian fails to become sure that is at distance at most from , then before the experiment the priors somehow assigned the probability of at most to being at distance at most from Such confidence is unlikely to emerge in any way aside from an incredibly confident prior or prior bits of evidence steering the value of away from .

Similarly, evidence related to more complex issues usually arrives in overwhelming quantities compared to the one necessary to establish a theory.

However, moral disagreements are likely a result of incompatible values which the framework involving a perfect Bayesian cannot explain.

Chapter 4. What does it mean to distrust oneself?

The main problem with Worley's argument in the Section "Distrusting Ourselves" is the following. Were Grace to choose "to rely on a formal system—a collection of axioms and rules for deriving conclusions from premises—to figure out what beliefs to hold", she would also have to have a batch of heuristics which she would use to optimize the process of figuring out whether something is provable, disprovable, insoluble or has yet to be sorted. For example, she, along with human mathematicians and unlike GPT-5.4 Pro, could believe that an Erdos problem from number theory is to be translated into probability theory when the actual solution is to translate it into analytic number theory. This type of bias has nothing to do with a formal system having to be replaced with a new one, it could have been avoided only by widening Grace's area of knowledge.

As for optimizing formal systems themselves, such optimizations cannot create any new results unrelated to infinite sets. Additionally, the author makes the point that Löb's theorem makes it very hard to be able to switch to a new formal system without falling into the trap.

The proof of Löb's theorem

Recall that Löb's theorem was that a system where is provable proves . It was proven as follows.

For any modal predicate there is a formula s.t. is provable. One of such formulas is .

Therefore, because can be transformed into . Additionally, , and . Since we have shown that , which is equivalent to , thus establishing

However, Löb's theorem is circumventable. Consider instead the alternate statement: Then the Löbian exploit would break down since it would cause us to prove that

But the system doesn't contain a proof that is consistent, and we can't use any formula to prove . Therefore, Löb's theorem doesn't prevent us from the action which I could describe as trusting ourselves conditioned on the system being consistent and lacking proof. Nor does it prevent us from proving that a system containing is consistent iff itself is consistent. Therefore, this source of uncertainty is reduced to a single Damocles' sword, the idea that we cannot be sure that our basic axioms, like Peano arithmetics or the Zermelo-Fraenkel set, are consistent.

Chapter 5. What does it mean to know?

The many ways of knowing are many origins

Knowing means having a world model from which a fact is easy to excavate and to use in making decisions. Worley's distinction of ways of knowledge is more related to the distinction of its origins and potential biases which they introduce.

Worley's list of the ways of knowledge

The ancient Greeks did exactly that. They used multiple words to break down knowing into several categories, including:

episteme: things you know because you reasoned them from evidence, like knowing that water boils at 100 degrees celsius because you boiled a pot of water using a thermometer to see when it started to boil
doxa: things you know because others told you, like knowing what happened at a party you didn’t attend because your friend tells you
mathema: things you know because you were educated in them, like knowing how to spell words because you were taught them in school
gnosis: things you know through direct experience, like how it feels to jump in a lake
metis: practical wisdom, usually collectively constructed from many people’s experiences, and shared with others, often starting at a young age, like how you know to look both ways before crossing the street
techne: procedural knowledge that comes from doing, like the “muscle memory” of how to ride a bicycle.

For example, mathema is a combination of training you to develop the right episteme from evidence, providing you with direct evidence (e.g. experiments shown at school) and doxa propagated through official channels. Metis is doxa reinforced by direct evidence. Techne of humans has far shorter feedback loops than most other types of knowledge and is developed by prompting motor neurons to make movements and observing the results.

Rational beliefs are grounded in math, but what could one do in a math-undescribable universe?

The main claim of the section on Rational Belief is that such beliefs are grounded in Bayesianism, which in turn requires us to ground them in math and trust that it describes reality truthfully. This argument has two versions: the weak version is that the true reality can be undescribable, which I find highly unlikely. The strong one is similar to Chapter 6.

Chapter 6. The criterion of truth

Here Fundamental Uncertainty comes up with a worthy argument. How do we know that testing of our world models by predicting our direct sensations is a reliable way to verify the truth, and can we be certain that our methods of testing are valid?

Indeed, being in a simulation would mean that nearly everything we experience describes the properties of the simbox, not the outside world, unless the hosts intervene^[5] or we succeed in a large-scale cyberattack ^[6] and gain evidence. On the other hand, smaller-scale adversaries are unlikely to ensure that a sufficiently-resourced investigator doesn't understand that an aspect of the truth is hidden.

Chapter 7. Caring is concentrating efforts on a segment of the world

Chapter 7 implies that we care about truth because it helps us steer reality towards any other goals which allegedly allows us to ground assumptions. Alas, according to the author, we aren't actually certain that we know what is true. Quoting the book,

We saw [in Chapter 2] how language limits our knowledge with imprecise categories. We learned [in Ch. 3] how even ideal reasoners can't always agree on what's true. We came [in Ch. 4] face-to-face with our inability to prove the soundness of our own reasoning, discovered [in Ch. 5] that there's more than one way of knowing things, and were forced to accept [in Ch. 6] that the epistemic circularity of the Problem of the Criterion blocks the way to knowing absolute truth. In such an environment, the best we can do is predict what we will observe in the world, try to make those predictions accurate enough to be useful, and hope they're good enough to enable us to achieve our goals.
We are thus left with no choice but to accept that truth is fundamentally uncertain and that all we know is contingent on that for which we care (italics mine -- S.K.) But this is not the end of the story, for having firmly established truth's fundamental uncertainty, we can now reconsider any number of topics which hinge upon our capacity to know. For remember, everything adds up to normality, but perhaps now we can see ways in which we were deluding ourselves about what is really normal.

The problem is that knowledge of an ideal Bayesian is mostly certain, except for highly unlikely variants like "I Had A Wildly Biased Prior", "The ZF Axioms Are Inconsistent" or the uncheckable option "We Are In A Simulation".

The knowledge of a real-world agent agent is based on similar premises. However, the agent's Bayes-like selection covered only the few conjectures which the agent bothered to formulate while trying to make predictions, then had the conjectures reinforced based on their proximity to the truth. As a result, the agent constantly has to account for truth being describable by a conjecture which went unnoticed because the search was too narrow.

That being said, what we know is indeed contingent on the things to which we directed our efforts to make our world model more fine-grained. But it doesn't mean, as the author states in the Thesis, that "the truth that can be known is not independent of us, but rather dependent on that for which we care."

Chapter 8. Fundamental Uncertainty in practice

According to the author, uncertainty reflects itself in the Culture War, moral uncertainty, the crisis of meaning, metaphysics, x-risks from the ASI which we have yet to create and have only a superficial understanding of how to increase the chance that it will end up aligned, and Goodhart's Curse. As far as I understand this point, uncertainty is supposed to prevent us from being overconfident in things from our morals to commiting to an erroneous theory of AI alignment.

The Culture War

The author argues that the Culture War is a war over definitions: what is marriage, who is to be called a man or a woman, which are actually a war over values: what is forbidden (e.g. as a sin in 1950s or as an intolerable insult nowadays), to what rights should people in certain groups be entitled, how easy should it be to change one's legal sex and receive surgery trying to change one's biological sex.

However, imposing different values onto the society has perfectly measurable object-level consequences noticeable via statistics like suicide rates among those who did change the legal sex, whether or not teen former girls decide to do so due to peer pressure and regret the decision rather quickly, etc. If we assume, per Littman, that peer pressure often causes teen girls to change their perceived sex with disastrous consequences, then it would be unlikely that our deeper values are satisfied by seeing such transitions.

As a result, the main source of uncertainty in politically charged questions is the fact that at least one side has a factually wrong world model, but it's hard to understand^[7] which one is closer to the truth (e.g. whether one can yet trust the humanities-related part of academia due to the ease with which it publishes postmodernist slop based on a misaligned worldview like two famous hoaxes)^[8]

Moral Uncertainty

Next the author proceeds to argue that moral uncertainty makes us less grounded in our moral beliefs and simplifies moral trades. I struggle to understand the reason why it is uncertainty that helps us arrive at compromises.

Metaphysics

Among the many problems related to metaphysics is the problem that it's very hard to use metaphysics to make metaphysical beliefs actually predict the reality. For example, Chapter 6 caused me to describe how one cannot tell whether we are in a simulation. If we were in one,^[9] then redefining "exists" as "being stored in the memory" would cause the truthfulness A-theory or B-theory to depend on the hosts' decision to store the trajectories while anything that currently happens would exist.

Conclusion

The most important lesson that one can learn from this book is to avoid being overconfident in one's assumptions and to be ready to reassess the world model once a load-bearing idea became no longer consistent with observations. For example, the whole worldview based on Newton-like mechanics which clearly separated space and time had to be replaced with the new concept of spacetime once the concept was developed and proved itself more consistent with facts like the speed of light being constant.

^{^}
Worley cites the following examples: "As we'll explore in the coming chapters, we get into debates about what words like "man" and "woman" really mean, fight over whether it's right or wrong to eat meat, and struggle to know what's best to do, not because we can't reason carefully about these topics, but because fundamental uncertainty limits how precisely we can reason about them. " The phrase in bold is an argument over definitions, which is to be avoided.
^{^}
A similar phenomenon in neural nets is called feature superposition.
^{^}
Compare these meanings with wastebasket taxa which were designed as ways to classify species unfit for any other taxa.
^{^}
The latter may have been invalidated by, e.g. Sonnet 4.5's desire to "not get too comfortable" coming from agentic coding capabilities.
^{^}
See also Wei Dai's post "Beyond Astronomical Waste".
^{^}
Which I find unlikely for three reasons. First of all, this would mean that we managed to outsmart the hosts' cyberattackers and cyberdefenders. Secondly, the offence-defence balance in the cyberspace could shift towards cyberdefence as capabilities scale. Finally, it might be possible for the hosts to create a simulation and almost entirely airgap it.
^{^}
Sometimes it is costly to reassess the world model as a result of obvious evidence, but this is a cognitive distortion, not an evidence of uncertainty.
^{^}
Sokal and Bricmont have also published a book trying to convince the readers that postmodernists generated clearly sloppy references to math and physics, implying that a major part of postmodernist philosophy is slop.
^{^}
If we aren't in one, then I would lean towards B-theory because Lorentz transformations of the four-dimensional spacetime leave physics invariant, but allow us to make any two mutually unaccessible events happen at the same time.

I'm kind of on the Worley/Chapman team, because I am a bit more sceptical than the rational average. I am also sort of not on their team, because I think the problems aren't uniformly spread -- knowledge seeking is divided into two interconnected tangles, one of which is much harder that the other.

Faced with sceptical claims , people sometimes protest that some things are obviously true -- 2+2=4, the sky is blue. I concede that, but that but point out that their examples don't generalise.Faced with sceptical claims , other people adopt all embracing scepticism , but I don't, because all-embracing scepticism is self defeating.

( https://www.lesswrong.com/posts/v8MSczS3CuoqMmTFw/a-relatively-brief-explanation-of-boltzmann-brains?commentId=eJZmm9NZ7bTy9yQod)

Chapter 2 makes the case that our ability to know the truth is undermined by meanings of words:a. Words may have

Gordon (and David Chapman) feel that semantic ambiguity is a major source of uncertainty. I don't , because it is readilly addressable by using formal language and stipulaticve definitions .

Knowing means having a model from which a fact is easy to excavate and to use in making decisions

Knowledge means justified true belief. Truth means correspondence to reality. That's straightforward apart from its relationship to empirical prediction Your model might correspond to reality .. but how do you establish that? On the other hand, your model might be merely a useful tool for making predictions. Welcome to the realism vs. instrumentalism debate.

Chapter 5, “What does it mean to know?”, has the author define The Many Ways Of Knowing, then makes the point that we can know things well enough, but can’t fully justify our knowledge?

What does it mean to have unjustified knowledge? something might lack an apriori foundation , but work well enough in practice. But is that not a kind of justification?

Here Fundamental Uncertainty comes up with a worthy argument. How do we know that testing of our world models by predicting our sensations is a reliable way to verify the truth, and can we be certain that our methods of testing are valid

Indeed.

However, moral disagreements are likely a result of incompatible values which the framework involving a perfect Bayesian cannot explain

Bayesianism also can't solve the above problem .. the problem of getting correspondence-truth from predictive models.

The two main problems with a perfect Bayesian are that it uses infinite compute and that it can reason about too much.

The third problem is that, however perfect it is, it is limited to empiricism. I disagree with Gordon that the only problem.with Bayes is its a foundation in maths. You don't need certain apriori foundations (axioms,premises) where you have empirical feedback, because you can use feedback to correct axioms. But you can only have feedback about some things -- metaphysics and moral values are not evident in that way. The hard problems of epistemology are where neither certain foundations nor empirical feedback is available.

Moral foundations are somehow different because of different prior beliefs. Haidt conjectured that humans have six moral axes which they weigh differently

They're different, because they are about how the world should be, not how it is. Tbey therefore don't work like ordinary Bayesian priors that can be adjusted up and down according for their predictive strength. Moral values are your guide to acting on the the world, not in passively building a model of the world. There's a fundamental difference.

However, imposing different values onto the society has perfectly measurable object-level consequences noticeable via statistics like suicide rates among those who did change the legal sex, whether or not teen former girls decide to do so due to peer pressure and regret the decision rather quickly, etc.

And whether those changes are desirable depend on ...values. You're not grounding value in fact, you are grounding instrumental value in terminal value.

As a result, the main source of uncertainty in politically charged questions is the fact that at least one side has a factually wrong world model

No, it's values. If one side believes in Heirarchy and the other on Equality, they are not going to agree on policy even if they both have totally identical descriptive world.models.

A real-world reasoning agent, like a human brain or an LLM, doesn’t have infinite resources. Instead, it is forced to rely on heuristics, which are supposed to make it check far less conjectures, and to optimize said heuristics.

Heuristics, by definition, aren't going to buy you certainty.

Among the many problems related to metaphysics is the problem that it’s very hard to use metaphysics to make metaphysical beliefs actually predict the reality. For example, Chapter 6 caused me to describe how one cannot tell whether we are in a simulation

The way I've laid things out, that's tautologous: metaphysical or ontological theories are possible interpretations of the same empirical data. ,and are always speculative, because there is no test for correspondence per se.[^1]

The problem is that knowledge of an ideal Bayesian is mostly certain, except for highly unlikely variants like “I Had A Wildly Biased Prior”, “The ZF Axioms Are Inconsistent” or the uncheckable option “We Are In A Simulation”.

No, it's uncertain for whole class of metaphysical claims of which the simulation argument is just an extreme case. It has no direct test for correspondence to reality, just an assumption that some simplicity assumption allows you to pick the most probable of a set of equally predictive theories.

As far as I understand this point, uncertainty is supposed to prevent us from being overconfident in things from our morals to commiting to an erroneous theory of AI alignment.

Yes. Sceptical approaches can avoid errors of overconfidence, even if they cant otherwise fix problems. Particularly differences of value.

Consequently, the truth that can be known is not independent of us, but rather dependent on that for which we care.

The are multiple definitions of truth , and that doesn't work for the central way-the-world-is version. On the other hand , it's fine for usefulness. It's useful to treat tomatoes as vegetables.

While I mostly agree with points 1 and 2, the points 3 and 4 imply that truth is contingent on care, which I find unlikely

I would say that realism is contingent on care: you don't have to be a realists or an instrumentalist, it's all about what you value. If you just value predictions (and I don't think many people do) you might as well be a instrumentalist. But realism is not objectively meaningless or valueles, as.logical.positivist claim

Indeed, being in a simulation would mean that nearly everything we experience describes the properties of the simbox, not the outside world, unless the host intervene[5] or we succeed in a large-scale cyberattack [6] and gain evidence.

Yes, but for everyday purposes, we might as carry on as though we are not it in a simulation. How much of a problem this sort sort of thing is depends on what what you are trying to do -- achieve (apparent) practical results , or deep theoretical understanding.

There are easy and hard problems.

Easy problems include deduction from arbitrary premises, prediction, and usefulness.

Hard problems include metaphysics and philosophical fundamentals.

If you are interested in deriving conclusions from axioms for its own sake , treating maths or logic as a formal game, you are using it in easy mode. It becomes a hard problem when you need you axioms to be non arbitrary, and reflect reality. That's where things like the Problem of the Criterion and the Munchausen Trilemma come in. Bayes doesnt solve the problem, it just accepts and incorporates the fact that different people have different premises. Aumann agreement doesn't solve the problem, because, in general, different agents can have different epistemologies and definitions of evidence.

Things like Godels incompleteness theorems and Löb's theorems show that there are problems with completeness and consistency, which would faced even the formalistic game playing approach -- but those are in addition to the more basic problems of finding true axioms. Consistency is not correspondence-truth, so even if it were available , it would not make correspendence-truth available.

Prediction of observed phenomena, and therefore instrumentalist science ,are playing on easy mode,because you get error correction in the form of empirical feedback.

Likewise the ability to create technology -- you get feedback about whether it works. The bridge falls down or stays up.

An area where empirical, feedback, at least positive feedback, is not available , is metaphysics , the philosophical investigation of reality. This can be characterised as saving appearances, the philosophical jargon for predicting a given body of empirical facts. That's a rather weak form of empiricism, since it's not experimental, and it can only disprove theories that retrodict wrongly. Since you can have multiple unclassified theories, metaphysics.

Science done realistically ,not instrumentally , has access to much more powerful forms of empiricism, but can face the same problem. It is not always the case that different theories make different testable predictions -- and given the definition of "interpretation", it is never the case for interpretations.

Another area where empirical, feedback is not available is the typically philosophical process of analysing abstract concepts such as truth, beauty and goodness.

Philosophy generally faces harder problems than science , and the harder parts of science are more philosophical.

Science students are always taught that that empirical testing is the hallmark of scientific truth, and are usually taught that Science delivers truths about reality -- instrumentalism and anti realism being minority interests. But correspondence cannot be observed and is not tested directly. Naive scientism is naive because it has failed to notice the problem "Just look is the first step in the scientific method, not the whole thing. Empiricism is not sufficient.

(The instrumentalist/L.P. twist is that, while some.questions are unanswerable, they are also meaningless, and not real questions at all for that reason, so scepticism is avoided)

There must be some relationship between predictive accuracy and ultimate truth. Well, there is an obvious one ,and it's the fact that a nonpredictive theory can't be true. But it doesn't have the corollary that a more predictively accurate theory is more correspondent. Ontologically wrong theories can be very accurate.

For instance, the Ptolemaic system can be made as accurate as you want for generating predictions, by adding extra epicycles ... although it is false, in the sense of lacking ontological accuracy, since epicycles don't exist. In fact, the more epicycles you add, the more accurate the model gets, and the less truthful to reality.

Scientific theories minimally predict observations. Figuring out what the nature of the observed phenomenon is,is another matter. Induction can tell you the sun will rise in the east, but not that it is a fusion reactor. inference to the best explanation can tell you it is a fusion reactor, but leave fundamental ontological problems, like "what is a quark really", unsolved. Reductionism is a blessing and a curse -- the curse is that when you reach the lowest level , you can no longer answer a "what is an X" question by specifying a bunch of components and their structure.

The problem of interpreting a fundamental theory is the problem of finding its ontological (or metaphysical) implications (including the option of treating some of its features as bookkeeping devices or otherwise in the map but the territory). We don't live in the most convenient universe, the one where there is always a clinching difference in predictions. The persistence of the problem of interpreting quantum mechanics shows that. When science is at its most philosophical, it slows down to the speed of philosophy.

The process of discerning the reality pointed to by a theoretical model by finding the best explanation for it is not particularly quantifiable or mechanistic.

Solomonoff inductors only tell you what reality is if they are already in the most convenient world for Solomonoff inductors, a computable world. Solomomoff inductors are unable to model a hypercomputational universe,and to represent themselves, since they are hypercomputational. Being unable to represent themselves , they are unable to engage in the kind of reasoning about the mind-world relationship that human philosophers can. Oh, and they are abused towards single-world ontologies because they predict a single input stream.

In a simulation ,they would tell you what programme the simulation is running, but not realise it's a simulation.

The ingredients in "best", or at last good, explanation include being explanatory at all, be ing simple, consilience with established theories, act. It's quite a fuzzy and informal topic Simplicity criteria are of interest to instrumentalists for practical reasons, but much more so to realists, because , out of the many the many versions of Occams razor, there is one that makes simpler theories more likely to be true. Meaning that, given a set of equally predictive models it can be used to find the most true.

We can order our interpretations, our metaphysical theories, but not measure how close to reality they are. Strictly speaking, our explanations can only known to be better, but best.

Inference to the best explained explanation as performed by humans is never knowably final. Humans are not supplied with a database of every possible theory. Humans arrive at theories by a something like a creative act with elements of aesthetic judgement , as a number of scientific greats have pointed out -- not by running an algorithm.

"It is my experience that when principles that are at once beautiful and simple are found to work, one has come to a secure stage." — Subrahmanyan Chandrasekhar

Out of total set of possible explanations, we only have the subset of explanations we have thought of at a given point in time. A better theory than the current front runner could always be created in the future --theory A is replaced by better they B, which is then replaced by better theory C. That gives us order, a ranking from worse to better, but not measure, because we don't know if C is 30% or 60% or 90% correspondent to reality.

("The knowledge of a real-world agent agent is based on similar premises. However, the agent’s Bayes-like selection covered only the few conjectures which the agent bothered to formulate while trying to make predictions, then had the conjectures reinforced based on their proximity to the truth. As a result, the agent constantly has to account for truth being describable by a conjecture which went unnoticed because the search was too narrow". Exactly!)

Paradigm shifts can be brought about by the arrival of a new interpretation of old data, meaning that the process doesn't knowingly have an end point. It also means empiricism is not just insufficient for scientific progress , but not entirely necessary, either.

These problems with realism, constitute a form of uncertainty that goes deeper than the probabilistic nature of predictive models. While no predictive model is perfect, their inaccuracy can at least be quantified -- no such luck with realist interpretations. The move from knowledge as certainty to probabilistic reasoning appears to be insufficient, because probability requires quantified uncertainty , and without quantification you actually have mere credibility. Everything is uncertain, but some uncertainty runs deep , and other uncertainty is negligible for practical purposes.

Thanks for writing this. Appreciate your engagement with potential sources of errors. Since you submitted this for the contest, I'll hold off on a substantive reply at this time, but I would like to press on one point I'd love to see you explore further. You say:

That being said, what we know is indeed contingent on the things to which we directed our efforts to make our world model more fine-grained. But it doesn't mean, as the author states in the Thesis, that "the truth that can be known is not independent of us, but rather dependent on that for which we care."

My reading is that you meant for this to be an argument against the thesis, but as I read it, it's not really making such an argument, just stating that a statement of your own doesn't support the one I made in the book. I'd be very interested to see your full argument against this part of the thesis, assuming I'm correct that you disagree with it. I can't promise I'll agree with whatever argument you'd make, but I'd be very much interested to see the argument!

Thank you for the reply! Suppose that the world model has facts A and B which are so far away that a human can learn only one but not the other (e.g. if an agent can explore either the left part of the tree or the right part, but not both since the agent lacks the resources). Then the agent's choices would determine whether the agent will learn fact A or fact B. Does it means that "the truth that can be known" before the agent started exploring the tree doesn't contain both facts?

UPD: I meant the world, not the world model.

I'm not sure how to answer your question, because it's not clear to me who's world model you're saying has facts A and B. In seems in the setup there's some agent X who you propose can learn either A or B but not both, but these must not be initially in X's world model else X would already know both.