TL;DR Summary: Mathematical truths can be cashed out as combined claims about 1) the common conception of the rules of how numbers work, and 2) whether the rules imply a particular truth. This cashing-out keeps them purely about the physical world and eliminates the need to appeal to an immaterial realm, as some mathematicians feel a need to.
Background: "I am quite confident that the statement 2 + 3 = 5 is true; I am far less confident of what it means for a mathematical statement to be true." -- Eliezer Yudkowsky
This is the problem I will address here: how should a rationalist regard the status of mathematical truths? In doing so, I will present a unifying approach that, I contend, elegantly solves the following related problems:
- Eliminating the need for a non-physical, non-observable "Platonic" math realm.
- The issue of whether "math was true/existed even when people weren't around".
- Cashing out the meaning of isolated claims like "2+2=4".
- The issue of whether mathematical truths and math itself should count as being discovered or invented.
- Whether mathematical reasoning alone can tell you things about the universe.
- Showing what it would take to convince a rationalist that "2+2=3".
- How the words in math statements can be wrong.
This is an ambitious project, given the amount of effort spent, by very intelligent people, to prove one position or another regarding the status of math, so I could very well be in over my head here. However, I believe that you will agree with my approach, based on standard rationalist desiderata.
Here’s the resolution, in short: For a mathematical truth (like 2+2=4) to have any meaning at all, it must be decomposable into two interpersonally verifiable claims about the physical world:
1) a claim about whether, generally speaking, people's models of "how numbers work” make certain assumptions
2) a claim about whether those assumptions logically imply the mathematical truth (2+2=4)
(Note that this discussion avoids the more narrowly-constructed class of mathematical claims that take the form of saying that some admittedly arbitrary set of assumptions entails a certain implication, which decompose into only 2) above. This discussion instead focuses instead on the status of the more common belief that “2+2=4”, that is, without specifying some precondition or assumption set.)
So for a mathematical statement to be true, it simply needs to be the case that both 1) and 2) hold. You could therefore refute such a statement either by saying, "that doesn't match what people mean by numbers [or that particular operation]", thus refuting #1; or by saying that the statement just doesn't follow from applying the rules that people commonly take as the rules of numbers, thus refuting #2. (The latter means finding a flaw in steps of the proof somewhere after the givens.)
Therefore, a person claiming that 2+2=5 is either using a process we don't recognize as any part of math or our terminology for numbers (violating #1) or made an error in calculations (violating #2). Recognition of this error is thus revealed physically: either by noticing the general opinions of people on what numbers are, or by noticing whether the carrying out of the rules (either in the mind or some medium isomorphic to the rules) has a certain result. It follows that math does not require some non-physical realm. To the extent that people feel otherwise, it is a species of the mind-projection fallacy, in which #1 and #2 are truncated to simply "2+2=4", and that lone Platonic claim is believed to be in the territory.
The next issue to consider is what to make of claims that "math has always existed (or been true), even when people weren't around to perform it". It would instead be more accurate to make the following claims:
3) The universe has always adhered to regularities that are concisely describable in what we now know as math (though it's counterfactual as nobody would necessarily be around to do the describing).
4) It has always been the case that if you set up some physical system isomorphic to some mathematical operation, performed the corresponding physical operation, and re-interpreted it by the same isomorphism, the interpretation would match that which the rules of math give (though again counterfactual, as there's no one to be observing or setting up such a system).
This, and nothing else, is the sense in which "math was around when people weren't" -- and it uses only physical reality, not immaterial Platonic realms.
Is math discovered or invented? This is more of a definitional dispute, but under my approach, we can say a few things. Math was invented by humans to represent things usefully and help find solutions. Its human use, given previous non-use, makes it invented. This does not contradict the previous paragraphs, which accept mathematical claims insofar as they are counterfactual claims about what would have gone on had you observed the universe before humans were around. (And note that we find math so very useful in describing the universe, that it's hard to think what other descriptions we could be using.) It is no different than other "beliefs in the implied invisible" where a claim that can't be directly verified falls out as an implication of the most parsimonious explanation for phenomena that can be directly verified.
Can "a priori" mathematical reasoning, by itself, tell you true things about the universe? No, it cannot, for any result always needs the additional empirical verification that phenomenon X actually behaves isomorphically to a particular mathematical structure (see figure below). This is a critical point that is often missed due to the obviousness of the assumptions that the isomorphism holds.
What evidence can convince a rationalist that 2+2=3? On this question, my account largely agrees with what Eliezer Yudkowsky said here, but with some caveats. He describes a scenario in which, basically, the rules for countable objects start operating in such a way that combining two and two of them would yield three of them.
But there are important nuances to make clear. For one thing, it is not just the objects' behavior (2 earplugs combined with 2 earplugs yielding 3 earplugs) that changes his opinion, but his keeping the belief that these kinds of objects adhere to the rules of integer math. Note that many of the philosophical errors in quantum mechanics stemmed from the ungrounded assumption that electrons had to obey the rules of integers, under which (given additional reasonable assumptions) they can't be in two places at the same time.
Also, for his exposition to help provide insight, it would need to use something less obvious than 2+2=3's falsity. If you instead talk in terms of much harder arithmetic, like 5,896 x 5,273 = 31,089,508, then it's not as obvious what the answer is, and therefore it's not so obvious how many units of real-world objects you should expect in an isomorphic real-world scenario.
Keep in mind that your math-related expectations are jointly determined by the belief that a phenomenon behaves isomorphically to some kind of math operation, and the beliefs regarding the results of these operations. Either one of these can be rejected independently. Given the more difficult arithmetic above, you can see why you might change your mind about the latter. For the former, you merely need notice that for that particular phenomenon, integer math (say) lacks an isomorphism to it. The causal diagram works like this:
Hypothetical universes with different math. My account also handles the dilemma, beloved among philosophers, about whether there could be universes where 2+2 actually equals 6. Such scenarios are harder than one might think. For if our math could still describe the natural laws of such a universe, then a description would rely on a ruleset that implies 2+2=4. This would render questionable the claim that 2+2 has been made to non-trivially equal 6. It would reduce the philosopher's dilemma into "I've hypothesized a scenario in which there's a different symbol for 4".
I believe my account is also robust against mere relabeling. If someone speaks of a math where 2+2=6, but it turns out that its entire corpus of theorems is isomorphic to regular math, then they haven’t actually proposed different truths; their “new” math can be explained away as using different symbols, and having the same relationship to reality except with a minor difference in the isomorphism in applying it to observations.
Conclusion: Math represents a particularly tempting case of map-territory confusion. People who normally favor naturalistic hypotheses and make such distinctions tend to grant math a special status that is not justified by the evidence. It is a tool that is useful for compressing descriptions of the universe, and for which humans have a common understanding and terminology, but no more an intrinsic part of nature than its usefulness in compressing physical laws causes it to be.