This post is part of my Hazardous Guide To Rationality. I don't expect this to be new or exciting to frequent LW people, and I would appreciate comments and feedback in light of intents for the sequence, as outlined in the above link.

A dialogue:

Nicky: I've been wondering, do you think math was invented or discovered?
Dee: Seems like it must have been discovered. I read about how circles are everywhere in nature, and that you can even find the fibonacci sequence in plants!
Nicky: Yeah, but there aren't actually any numbers in nature. Numbers are just something we made up to describe and talk about these patterns that we see in nature. Numbers themselves don't really exist out in the world.
Dee: Of course numbers are real! Made up constructs don't have the predictive power that math does. They totally exists.
Nicky: Well if numbers exist, where are they? You can't show me where a number is. You can't empirically test for numbers. You can't find them anywhere in the physical world. They're just constructs!
Dee: Sure, they don't a physical location in the world. That's silly. There's no circles floating around out behind the moon. What I'm saying is that they exist outside of space and time. Mathematical existence is it's own sort of domain, separate from the domain of physical existence.
Nicky: Bleh, next you're going to tell me that you believe in cartesian dualism.
Dee: Bleh, next you're going to tell me that math is arbitrary and people can build rockets that work however they feel like.
The two never talked again. Dee, remembering this conversation in great detail, went on the become a commited Platonist and write many articles trying to defend this complex philosophical view

To help draw out the point I'm trying to make, here's an alternative history of this conversation.

Nicky: Hey Dee, you got any views on mathematical platonism?
Dee: What's that?
Nicky: It's the idea that mathematical objects exist in reality, but seperate from physical reality. Physical reality defines what's true about the world we live in, and mathematical reality defines what's true about math objects.
Dee: Hmmmm, I'm not sure. I mean, that seems like it would explain why math is so certain and precise, but it also feels weird to posit a whole new fundamental element of reality, and I'm partial to materialism. I'll have to think about this.

Dee didn't really care about or have well formed beliefs about whether or not mathematical Platonism is true. Yet a conversation happened in such a way that left Dee defending Platonism. That seems a little weird, let's look at what happened.

1. When Dee hear's "social construct" she thinks about things being arbitrary and not having to do with reality. When she things of things that are "real" she thinkgs of useful and true things.
2. Dee thinks math is useful and true and says it's "real".
3. Nicky here's "real" and things about things that can be located in time and space. When she thinks of "social construct" she thinks of things that are in people's heads.
4. Nicky says math isn't real and brings up the point about location
5. Dee thinks that if she can't call math "real", she doesn't get to consider it useful and true.
6. Dee agrees that math doesn't have a location in time in space, and that this notion is relevant to calling something "real".
7. Dee extends the shared definition of "real" to include "existing in physical reality or some other kind of reality"
8. Dee claims mathematical Platonism.
9. Nicky implicitly accepts the extension of the definition of "real"
10. Nicky explicitly argues against the claim of math platonism

5 is the crux of the issue.

Dee felt like she weren't allowed to consider math to be useful and certain unless they were able to say it was "real". If you're thinking about the mind map model of meaning, this is trivially wrong. The word "real" can be linked to all sorts of concepts and criterion that don't have to always come in a package. Math can be useful and certain, even if it doesn't have a physical location in space and time. No problem.

But if you aren't thinking about the mind map model, are are just inside the algorithm, the word "real" does not feel like a pointer connected to other concepts. It "feels" like those concepts. And not getting to use the word "real" feels like not getting to use those concepts.

The second really interesting part of this conversation is points 6-9.

In an effort to get to use the word real, despite Nicky. Dee implicitly claimed, "Being real doesn't have to mean existing somehwere in physical reality. It can also mean existing somewhere in another kind of reality".

Nicky implicitly accepted, "If there was another kind of reality, yes, I would consider something that existed in it to be real. But I will now argue that I don't think there is this other kind of reality."

So Dee goes to alter the shared meaning of the word "real" and also make another claim about that extended definition applying to math. This extension was implicitly accepted, and the argument turned to being about that claim.

in our first dialogue, ALbert and Barry were paying a lot of attention to how they were defining words, and even explicitly argued about it. Nicky and Dee also talked and moved around definitions, but this all happened IMPLICITLY.

When you are stuck, it's common to go "I'm both making an extension to the definition, and making a claim about X that allows the think I want to be in definition."

"I agree that if X, then Y is word, but I don't agree with X"

But from the outside, it's a seamless switch where the original content was lost and now we are arguing about X. You've lost track of what you cared about when you started the talking.

Once you are in a mental state of "What can I do to make sure that I get to apply this word to this concept" ("real" to math), weird shit can ensue. In our case, Nicky settles into taking a Platonist view of mathematics. That itself is not a bad thing. You can be a Platonist if you want. Even if Platonism wasn't true, it's not a given that thinking it's true is a mistake. The problem is now Nicky has tied the claim of mathematical Platonism to the claim of maths usefulness and certainty. It's unlikely that Nicky will ever come to believe math is not useful or certain, so she has accidentally come to hold a belief about Platonism that won't be changed, and might not be warranted.

I think a lot of beliefs get formed like this over time.

I have one friend that I argue with a lot, and I'm constantly accidentally forming and defending stances I don't actually care about because in the moment I think it is necessary to prove a point i actually do care about. We're getting better though.