# -2

I wrote a comment that captures a core part of what I'm trying to explain, so I will copy it here in its own post.

If we take as assumption that everything humans have observed has been made up of smaller physical parts (except possibly for the current elementary particles du jour, but that doesn’t matter for the sake of this argument) and that the macro state is entirely determined by the micro state (regardless of if it’s easy to compute for humans), there is a simple conclusion that follows logically from that.

This conclusion is that nothing extraphysical can have any predictive power above what we can predict from knowledge about physics. This follows because for something to have predictive power, it needs to have some influence on what happens. If it doesn’t have any influence on what happens, its existence and non-existence cannot allow us to make any conclusions about the world.

This argument applies to mathematics: if the existence of mathematics separately from physics allowed us to make any conclusions about the world, it would have to have a causal effect on what happens, which would contradict the fact that all macro state we’ve ever observed has been determined by just the micro state.

Since the original assumption is one with very strong evidence backing it, it’s safe to conclude that, in general, whenever we think something extraphysical is required to explain the known facts, we have to be making a mistake somewhere.

It may seem like just a simple tautology, but tautologies and their consequences are not always obvious, and this particular tautology has many, many consequences. It can help to avoid many confusions about things like mathematics, qualia, free will, subjective probability, and truth. I consider that noticing this tautology and deciding to ground my philosophy on it to be in the top 3 best decisions I've ever made.

One particularly important consequence of this observation is that we can apply the basic tenet of rationality that says that we should not believe things without evidence to rule out any philosophy which assumes the existence of extraphysical things.

Note that you do not need to reject any mathematics to accept this philosophy, as I attempted to explain in my previous post. The most important thing I was attempting with that post was showing one way to break down mathematics to physics without falling into the trap of rejecting large swaths of mathematics. Naturally, I am less confident that my specific way of breaking it down is correct than the fact that there has to exist such a breakdown.

One very important thing to do when applying this approach to philosphy is to remember that everything should add up to normality. When we can't figure out how to break things down to physics in a way that adds up to normality, we are very likely to be making a mistake. I have in general been able to integrate this philosophy into my beliefs and have it all add up to normality.

Edit: While this argument holds for any explanation of macro phenomena that relies on the existence of non-physical things, it does not apply to explanations of why the laws of physics are what they are, or explanations that allow us to predict the initial configuration of the universe. I will note that all human experience with mathematics falls into the bucket of macro phenomena, and that positing a mathematical universe does not actually give any predictive power for why the laws of physics are what they are, or why the physical world is ordered this way.

# -2

New Comment

Consider a computer that takes a number , adds to it to get , then multiplies these numbers to get and prints the parity of the result. The fact that the resulting parity is always even can be determined by another computation that is external to this computer, computing the fact by searching for and verifying its proof. Thus we have two different computations, one reasoning about the other, neither embedded in the other in a straightforward sense.

The computation that is being reasoned about doesn't have to be real to be reasoned about. The computer in the above example doesn't need to be built for the proof of its result being even to hold. The fact about parity would hold if the computer is built, but whether it's actually built is irrelevant for the purposes of computing the fact. There is no causal interaction. Thus people and machines can think about mathematics that doesn't concretely exist in the physical world.

The frame that gestures at a mathematician and says "it's just a bunch of atoms" or "it's just an algorithm" is coherent. The issue is that it doesn't have anything relevant to say about this point, which is not the same as being some sort of refutation.

I disagree that what you're saying contradicts what I'm saying. The physical world is ordered in such a way that the reasoning you described works: this is a fact about physics. You are correct that it is a mysterious fact about physics, but positing the existence of math does not help explain it, merely changes the question from "why is physics ordered in this way" to "why is mathematics ordered in this way".

[-]TAG20

Physics doesn't guarantee that mathematical reasoning works.

“why is mathematics ordered in this way”.

There are answers to that question.

Physics doesn’t guarantee that mathematical reasoning works.

All of math can be built on top of first-order logic. In the sub-case of propositional logic, it's easy to see entirely within physics that if I observe that "A AND B" corresponds to reality, then when I check if "A" corresponds to reality, I will also find that it does. Every such deduction in propositional logic corresponds to something you can check in the real physical world.

The only infinity in first-order logic are quantifiers, of which only one is needed: FORALL, which is basically just an infinite AND. I don't think it's too surprising that a logical deduction from an infinite AND will hold in every finite case that we can check, for similar reasons to why logical deductions hold for the finite case.

It is mysterious that physics is ordered in a way that this works out, but pending the answer you say exists, it's not any more mysterious than asking why math is ordered that way.

[-]TAG20

Again: every mathematical error is a real physical even in someone's brain, so , again, physics guarantees nothing.

The only infinity in first-order logic are quantifiers,

There are of course, lots of infinities in maths. Our ability to reason about them means mathematical reasoning includes symbolic reasoning, not just direct calculation. That's a computation or pyschology-level observation -- it doesn't add anything to point out that brains are made of quarks.

Again: every mathematical error is a real physical even in someone’s brain, so , again, physics guarantees nothing.

I don't get what you're trying to show with this. If I mistakenly derive in Peano Arithmetic that 2 + 2 = 3, I will find myself shocked when I put 2 apples inside a bag that already contains 2 apples and find that there are now 4 apples in that bag. Incorrect mathematical reasoning is physically distinguishible from correct mathematical reasoning.

There are of course, lots of infinities in maths.

Everything we know about all other infinities can be built on top of just FORALL in first-order logic.

[-]TAG20

Incorrect mathematical reasoning is physically distinguishible from correct mathematical reasoning.

Only for the subset of maths that's also physical. You can't resolve the Axiom of Choice problem that way.

Everything we know about all other infinities can be built on top of just FORALL in first-order logic.

Whatever "built on top of" means. Clearly, we can intend transfinite models.

Whatever "built on top of" means.

In ZFC, the Axiom of Infinity can be written entirely in terms of ∈, ∧, ¬, and ∀. Since all of math can be encoded in ZFC (plus large cardinal axioms as necessary), all our knowledge about infinity can be described with ∀ as our only source of infinity.

Only for the subset of maths that’s also physical. You can’t resolve the Axiom of Choice problem that way.

You can't resolve the Axiom of Choice problem in any way. Both it and its negation are consistent.

There are answers to that question.

If you don't mind, I would be interested in a link to a place that gives those answers, or at least a keyword to look up to find such answers.

[-]TAG30

It's called philosophy of maths.

Reasoning practiced in the physical world is a genre of computation, so what matters about physics in enabling reasoning is that it can be used to implement computation, to build computers. The reasoning can then be about all sorts of things, physics being among them but not different from others in a way relevant to what makes computation reasoning.

Sure, I think I agree. My point is that because all known reasoning takes place in physics, we don't need to assume that any of the other things we talk about exist in the same way that physics does.

I even go a little further than that and assert that assuming that any non-physical thing exists is a mistake. It's a mistake because it's impossible for us to have evidence in favor of its existence, but we do have evidence against it: that evidence is known as Occam's Razor.

There is also little point in saying that physics exists. You can reason about physics the same way as about other things, namely without having any use for the assumption that it exists.

Existence of physics makes sense as a statement about values, a claim of caring about physics more than about some other things (which don't exist in this sense). This is an ingredient of decision theory, not of reasoning.

The fact that I live in a physical world is just a fact that I've observed, it's not a part of my values. If I lived in a different world where the evidence pointed in a different direction, I would reason about the different direction instead. And regardless of my values, if I stopped reasoning about the physical world, I would die, and this seems to me to be an important difference between the physical world and other worlds I could be thinking about.

Of course this is predicated on the concept of "I" being meaningful. But I think that this is better supported by my observations than the idea that every possible world exists and the idea that probability just represents a statement about my values.

To formulate useful abstractions, it's important to figure out what data is relevant in what arguments. For existence in the sense that physics exists, I don't see how it's relevant for reasoning (including about physics), but I do see how it's relevant to decision making, and that is the distinction I pointed out. Where the fact itself comes from is distinct from where it's useful, and I'm pointing specifically at the latter here.

You then say "this is predicated on the concept of "I" being meaningful". Presumably it's the argument for reality of physics you gave that's predicated on this, not something else. Parity of is not predicated on this, it shouldn't be part of the proof of evenness. So it's another example for the principle I'm describing in this comment.

(To be clear, I'm not ruling out being wrong about the claim of something not being relevant. But the relevant wrongness would need to be in the form of it turning out to be relevant to the particular arguments in question after all, rather than merely true or known, or relevant to something else.)

Okay, let's forget the stuff about the "I", you're right that it's not relevant here.

For existence in the sense that physics exists, I don’t see how it’s relevant for reasoning, but I do see how it’s relevant to decision making

Okay, I think my view actually has some interesting things to say about this. Since reasoning takes place in a physical brain, reasoning about things that don't exist can be seen as a form of physical experiment, where your brain builds a description which has properties which we assume the thing that doesn't exist would have if it existed. I will reuse my example from my previous post to explain what I mean by this:

To be more clear about what I mean by mathematical descriptions “sharing properties” with the thing it describes, we can take as example the real numbers again. The real numbers have a property called the least upper bound property, which says that every nonempty collection of real numbers which is bounded above has a least upper bound. In mathematics, if I assume that a variable x is assigned to a nonempty set of real numbers which is bounded above, I can assume a variable y which points to its least upper bound. That I can do this is a very useful property that my description of the reals shares with the real numbers, but not with the rational numbers or the computable real numbers.

So my view would say that reasoning is not fundamentally different from running experiments. Experiments seem to me to be in a gray area with respect to this reasoning/decision-making dichotomy, since you have to make decisions to perform experiments.

Reasoning being real and the thing it reasons about being real are different things. Truth of an experiment being real is not the same as relevance of the experiment being real. Would the experiment behave differently if it's not real? I'd say it would need to be a different experiment to do that, not being real wouldn't suffice, so the distinction of being real vs. not is not useful for this purpose.

Reasoning being real and the thing it reasons about being real are different things.

I do agree with this, but I am very confused about what your position is. In your sibling comment you said this:

Possibly the fact that I perceive the argument about reality of physics as both irrelevant and incorrect (the latter being a point I didn’t bring up) caused this mistake in misperceiving something relevant to it as not relevant to anything.

The existence of physics is a premise in my reasoning, which I justify (but cannot prove) by using the observation that humanity has used this knowledge to accomplish incredible things. But you seem to base your reasoning on very different starting premises, and I don't understand what they are, so it's hard to get at the heart of the disagreement.

Edit: I understand that using observation of the physical world to justify that it exists is a bit circular. However, I think that premises based on things that everyone has to at least act like they believe is the weakest possible sort of premise one can have. I assume you also must at least act like the physical world is real, otherwise you would not be alive to talk to me.

I don't see how reality of physics is used in your reasoning. I did see that you claim that you do use it, and that you mentioned it in the posts, but I don't see how it's doing some work in some argument. I don't see how humanity accomplishing incredible things has anything to do with the world being real, we could similarly accomplish incredible things in a world that isn't real.

My frame is grounded in thinking about decision theory, where one thing that keeps coming up is counterfactuals, reasoning about what would happen under conditions that at some point are revealed to in fact fail to hold. This is reasoning about situations and worlds that are not real, and for this form of decision making to make sense, it's necessary for the reasoning about worlds that are not real to reach meaningful conclusions. This makes discussion of detailed claims about worlds that are not real a normal thing, not something strange. And when that crystallizes into an intuitive way of looking at things in general, it becomes apparent that if our physical world wasn't real in a similar sense, literally nothing about anything would change as a result. Relevance of the world being real is largely an illusion.

What matters about the world being real is that it seems to be the case that we care about what happens in the physical world, possibly more than about what happens in some other hypothetical worlds. That's a formulation of the meaning of the physical world being real that's more clear to me than how these words are normally used (i.e. without a satisfactory clarification or an argument for truth of the claim that might also serve that purpose).

I completely agree that reasoning about worlds that do not exist reaches meaningful conclusions, though my view classifies that as a physical fact (since we produce a description of that nonexistent world inside our brains, and this description is itself physical).

it becomes apparent that if our physical world wasn’t real in a similar sense, literally nothing about anything would change as a result.

It seems to me like if every possible world is equally not real, then expecting a pink elephant to appear next to me after I submit this post seems just as justified as any other expectation, because there are possible worlds where it happens, and ones where it doesn't. But I have high confidence that no pink elephant will appear, and this is not because I care more about worlds where pink elephants don't appear, but because nothing like that has ever happened before, so my priors that it will happen are low.

For this reason I don't think I agree that nothing would change if the physical world wasn't real in a similar sense as hypothetical ones.

What I mean by reaching meaningful conclusions about counterfactuals is that you start with a problem statement, a description of a possibly counterfactual situation, and then you see what follows from that. You don't get to decide that pink elephants follow just because the situation is counterfactual, any pink elephants would need to follow from the particular problem statement that you start with. Existence of other counterfactuals (other possible worlds) with pink elephants is completely irrelevant, because we are not reasoning about them at the moment.

Similarly, if you reason about the physical world that isn't real, it doesn't matter that there are other alternative physical worlds that are also not real with different properties, because we are reasoning about this particular not-real world, not those other ones. The problem statement constrains the expectations, not reality of the thing referenced by the problem statement.

let's forget the stuff about the "I", you're right that it's not relevant here

No, I believe I'm wrong. I reversed the point about "I" in the grandparent about 10 minutes after posting, which turns out to be too late. I originally said that I don't see how that's relevant at all, but then noticed that it's not true, since it's relevant to your argument about reality of physics. Possibly the fact that I perceive the argument about reality of physics as both irrelevant and incorrect (the latter being a point I didn't bring up) caused this mistake in misperceiving something relevant to it as not relevant to anything.

[-]TAG20

My point is that because all known reasoning takes place in physics, we don’t need to assume that any of the other things we talk about exist in the same way that physics does.

It’s a mistake because it’s impossible for us to have evidence in favor of its existence, but we do have evidence against it: that evidence is known as Occam’s Razor.

But it's not conclusive in every case, because the simplest adequate explanation need not be a physical explanation.

clearly physical brains can think about non physical things.

Yes, but this is not evidence for the existence of those things.

But it’s not conclusive in every case, because the simplest adequate explanation need not be a physical explanation.

There is one notion of simplicity where it is conclusive in every case: every explanation has to include physics, and then we can just cut out the extra stuff from the explanation to get one that postulates strictly less things and has equally good predictions.

But you're right, there are other notions of simple for which this might not hold. For example if we define simple as "shortest description of the world which contains all our observations". Though I think this definition has its own issues, since it probably depends on the choice of language.

Still, this is the most interesting point that has been brought up so far, thank you.

Edit: I was too quick with this reply and am actually wrong that my notion of simplicity is conclusive in every case. I still think this applies in every case that we know of, however.

Edit 2: I think the only case where it is not conclusive is the case where we have some explanation of the initial conditions of the universe which we find has predictive power but which requires postulating more things.

[-]TAG20

Yes, but this is not evidence for the existence of those things.

I didn't say it was.

But it’s not conclusive in every case, because the simplest adequate explanation need not be a physical explanation.

There is one notion of simplicity where it is conclusive in every case: every explanation has to include physics, and then we can just cut out the extra stuff from the explanation to get one that postulates strictly less things and has equally good predictions.

Why posit that an explanation has to include physics even in cases, like this, where it adds nothing? In those cases it's simpler not to include physics.

I'm not claiming that there is a mysterious fact about physics here, or that what I'm saying contradicts what you're saying. I sketched a point that makes sense to me and stands on its own, vaguely hoping but not claiming that it's relevant or helpful. It can be very difficult to communicate or discuss an issue that's not clearly formulated, so that exchanging smaller and more clearly formulated arguments that don't depend on comprehending the specific issue is more practical.

Well if you're not saying it, then I'm saying it: this is a mysterious fact about physics ;P

I interpreted "which is not the same as being some sort of refutation" as being disagreement, and I knew my use of the word "contradicts" was not entirely correct according to its definition, but I couldn't think of a more accurate word so I figured it was "close enough" and used it anyway (which is a bad communication habit I should probably try to overcome, now that I'm explicitly noticing it). I'm sorry if I came across harshly in my comment.

I do agree, but I'm far less confident than you about the completeness of my mapping of macro experiences to sub-micro state.  Remember, the vast majority of physics knowledge is not calculated bottom-up from quantum measurements and statistics, it's actually measured at the macro level and assumed to (and occasionally confirmed in very VERY specific instances) have come from these lower levels.

I still find it easy to reject invisible pink dragons, but I lean much more on macro-level irrelevance than on lack of low-level physics modeling.

This is fair, though the lack of experiments showing the existence of anything macro that doesn't map to sub-micro state also adds a lot of confidence, in my opinion, since the amount of hours humans have put into performing scientific experiments is quite high at this point.

Generally I'd say that the macro-level irrelevance of an assumption means that you can reject it out of hand, and lack of micro-level modelling means that there is work to be done until we understand how to model it that way.

[-]TAG20

This is fair, though the lack of experiments showing the existence of anything macro that doesn’t map to sub-micro state also adds a lot of confidence,

There are things that don't have reductive explanations. We didn't get a reductive explanation of consciousness the day we found out brains a re made of neurons. Whether "map to" means "explained by" is another question.

I haven't used the word "reduce" since you gave a definition of it in the other thread which didn't match the precise meaning I was aiming for. The meaning I am aiming for is given in this paragraph from this post:

If we take as assumption that everything humans have observed has been made up of smaller physical parts (except possibly for the current elementary particles du jour, but that doesn’t matter for the sake of this argument) and that the macro state is entirely determined by the micro state (regardless of if it’s easy to compute for humans), there is a simple conclusion that follows logically from that.

It doesn't matter if we have found an explanation for consciousness yet. We still know with high confidence that it has to be entirely determined by the small physical components of the brain, so we can have high confidence that any attempted explanation will be wrong if it relies on the existence of other things than the physical components.

I consider that noticing this tautology and deciding to ground my philosophy on it to be in the top 3 best decisions I've ever made.

Interesting that in a post claiming that everything can be deduced from "bottom up" reasoning you justify your conclusions using tautological axioms rather than bottom-up reasoning.  It's almost as though doing rationality requires one to make assumptions about the thing being reasoned about rather than deducing everything from the bottom up.

I don't say in this post that everything can be deduced from bottom up reasoning.

In general I have not found anything which a mathematical universe explained which did not either contradict reductionism, or which could not be explained at least as well using the physical view of the world. I conclude from this that it's unlikely that there exists a mathematical universe.

I will refer to this other comment of mine to explain this miscommunication.

I'm afraid that doesn't make your position any more clear to me.

The tautological belief that everything is made of small physical parts itself not a "small physical part", it is one of the most broad claims about the universe possible.

It seems to me that you belief in at least 3 truths axiomatically:

1. The universe has the convenient property that the outcomes of large physical systems can be predicted by small systems such as human minds.
2. All systems are made up of small physical parts
3. There are no other axioms besides 1. 2. and this one

Even if I accept 1. then 2. and 3. neither follow from 1. nor do they have additional predictive power that would cause me to accept them.

The assumption is that everything is made up of small physical parts. I do not assume or believe that it's easy to predict the large physical systems from those small physical parts. But I do assume that the behavior of the large physical systems is determined solely from their smaller parts.

The tautology is that any explanation about large-scale behavior that invokes the existence of things other than the small physical parts must be wrong, because those other things cannot have any effect on what happens. Note that this does not mean that we need to describe everything in terms of quantum physics. But it does mean that a proper explanation must only invoke abstractions that we in principle would be able to break down into statements about physics, if we had arbitrary time and memory to work out the reduction. (Now I've used the word reduction again, because I can't think of a better word, but hopefully what I mean is clear.)

This rules out many common beliefs, including the platonic existence of math separately from physics, since the platonic existence of math cannot have any effect on why math works in the physical world. It does not rule out using math, since every known instance of math, being encoded in human brains / computers, must in principle be convertible into a statement about the physical world.

Maybe I'm just confused because I recently had an argument with someone who didn't believe in infinity.

When I pointed out all of physics is based on the assumption that spacetime is continuous (an example of infinity) his response was essentially "we'll fix that someday".

So, given that you deny "the mathematical universe", does that mean you think spacetime isn't continuous?  Or are "small physical parts" allowed to be infinitely subdivided?

Is it really based on an assumption that there's no quantized layer, even down very deep?  I'm not sure I have a belief either way, but I don't know of any macro-scale implication that requires true continuously-divisible units.

You can't have e.g. Lorenz Invariance on a quantized spacetime.  Of course, you can always argue that "the quantization is really small giving us approximate Lorenz Invariance", but this would be a claim not only without evidence, but actually against all of the evidence we have.

Similarly, the Schrodinger Wave Equation is defined mathematically as an operator evolving in continuous spacetime.  Obviously you can approximate it using a digital world (as we do on our computers), but the real world doesn't show any evidence of using such an approximation.

My view is compatible with the existence of actual infinities within the physical universe. One potential source of infinity is, as you say, the possibility of infinite subdivision of spacetime. Another is the possibility that spacetime is unboundedly large. I don't have strong opinions one way or another on if these possibilities are true or not.

Okay, but if actual infinities are allowed, then what defines small in the "made up of small parts"? Like, would tiny ghosts be okay because they're "small"?

The Unreasonable Effectiveness of Math makes a predictable claim: models which can be represented using concise mathematical notation are more likely to be true, but this includes the whole mathematical universe.

What part of the mathematical universe do you reject exactly?

I'm still trying to understand this quote:

In general I have not found anything which a mathematical universe explained which did not either contradict reductionism, or which could not be explained at least as well using the physical view of the world. I conclude from this that it's unlikely that there exists a mathematical universe.

And so far it sounds like you're fine with literal infinity.  So what part of a mathematical universe do you find distasteful?  Is it all infinities larger than  or the idea that "2" exists as an abstract idea apart from any physical model, or something else?

So what part of a mathematical universe do you find distasteful?

the idea that “2” exists as an abstract idea apart from any physical model

It's this one.

Okay, but if actual infinities are allowed, then what defines small in the “made up of small parts”? Like, would tiny ghosts be okay because they’re “small”?

Given that you're asking this question, I still haven't been clear enough. I'll try to explain it one last time. This time I'll talk about Conway's Game of Life and AI. The argument will carry over straightforwardly to physics and humans. (I know that Conway's Game of life is made up of discrete cells, but I won't be using that fact in the following argument.)

Suppose there is a Game of Life board which has an initial state which will simulate an AI. Hopefully it is inarguable that the AI's behavior is entirely determined by the cell states and GoL rules.

Now suppose that as the game board evolves, the AI discovers Peano Arithmetic, derives "2 + 2 = 4", and observes that this corresponds to what happens when it puts 2 apples in a bag that already contains 2 apples (there are apple-like things in the AI's simulation). The fact that the AI derives "2 + 2 = 4", and the fact that it observes a correspondence between this and the apples, has to be entirely determined by the rules of the Game of Life and the initial state.

In case this seems too simple and obvious so far and you're wondering if you're missing something, you're probably not missing anything, this is meant to be simple and obvious.

If the AI notices how deep and intricate math is, how its many branches seem to be greatly interconnected with each other, and postulates that math is unreasonably effective. This also has to be caused entirely by the initial state and rules of the Game of Life. And if the Game of Life board is made up of sets embedded inside some model of set theory, or if it's not embedded in anything and is just the only thing in all of existence, in either case nothing changes about the AI's observations or actions and nothing ought to change about its predictions!

And if the existence or non-existence of something changes nothing about what it will observe, then using its existence to "explain" any of its observations is a contradiction in terms. This means that even its observation of the unreasonable effectiveness of math cannot be explained by the existence of a mathematical universe outside of the Game of Life board.

Connecting this back to what I was saying before, the "small parts" here are the cells of the Game of Life. You'll note that it doesn't matter if we replace the Game of Life by some other similar game where the board is a continuum. It also doesn't even matter if the act of translating statements about the AI into statements about the board is uncomputable. All that matters is that the AI's behavior is entirely determined by the "small parts".

You might have noticed a loophole in this argument, in that even though the existence of math cannot change anything past the initial board state, if the board was embedded inside a model of set theory, then it would be that model which determined the initial state and rules. However, since the existence of math is compatible with every consistent set of rules and literally every initial board state, knowing this would also give no predictive power to the AI.

At best the AI could try to argue that being embedded inside a mathematical universe explains why the Game of Life rules are consistent. But then it would still be a mystery why the mathematical universe itself follows consistent rules, so in the end the AI would be left with just as many questions as it started with.

[-]TAG20

the idea that “2” exists as an abstract idea apart from any physical model

It’s this one.

Note that "Platonism false" does not imply "physicalism true". Numbers just might not be real entities at all, as in Formalism.

Now suppose that as the game board evolves, the AI discovers Peano Arithmetic, derives “2 + 2 = 4”, and observes that this corresponds to what happens when it puts 2 apples in a bag that already contains 2 apples (there are apple-like things in the AI’s simulation). The fact that the AI derives “2 + 2 = 4″, and the fact that it observes a correspondence between this and the apples, has to be entirely determined by the rules of the Game of Life and the initial state.

If the AI discovers transfinite maths or continuum mechanics, that fact is also entirely determined by rules of the Game of Life and the initial state. And neither of them can apply to a GoL universe-- they are not "physics".

Now, at this point, you need to choose between stipulating that the non-physical maths is false because it is non physical (finitism); or accepting that Platonism and physicalism are both false.

If the AI notices how deep and intricate math is, how its many branches seem to be greatly interconnected with each other, and postulates that math is unreasonably effective

But it's not maximally effective: maximal effectiveness would mean that any mathematical truth is a physical truth.

You’ll note that it doesn’t matter if we replace the Game of Life by some other similar game where the board is a continuum

If the physical universe is any way a subset of the mathematical "universe" , you have the same problem.

I mean, but our universe is not Conway's Game of Life.

Setting aside for now the problems with our universe being continuous/quantum weirdness/etc, the bigger issue has to do with the nature of the initial state of the board.

Whether or not math would be unreasonably effective in a universe made out of Conway's Game of Life depends supremely on the initial state of the board.

If the board was initialized randomly, then it would already be in a maximum-entropy distribution, hence "minds" would have no predictive power and math would not be unreasonably effective.  Any minds that did come into existence would be similar to Boltzmann Brains in the sense that they would come into existence for one brief moment and then be destroyed the next.

The initial board would have to be special for minds like ours  to exist in Conway's Game of Life.  The initial setup of the board would have to be in a specific configuration that allowed minds to exist for long durations of time and predict things.  And in order for that to be the case, there would have to be some universe wide set of rules governing how the board was set up.  This is analogous to how the number "2" is a thing mathematicians think is useful no matter where you go in our universe.

Math isn't about some local deterministic property that depends on the interaction of simple parts but about the global patterns.