Fictional Thinking and Real Thinking
11peterbarnett
8johnswentworth
10Gordon Seidoh Worley
7Gordon Seidoh Worley
5FireStormOOO
4Gordon Seidoh Worley
7Dagon
5TAG
4Mateusz Bagiński
3Aprillion
3Aprillion
New Comment
I claim that this example generalizes: insofar as Joe’s “fake thinking” vs “real thinking” points to a single coherent distinction, it points to thoughts which represent things in other worlds vs thoughts which represent things in our physical world.
This doesn’t feel quite right to me, or at least is missing something. When I think about Joe’s “fake thinking” vs “real thinking”, the main distinction is about whether you are “actually trying” or “actually care”.
When I was 20, I was well aware of the horrors of factory farming, I would said things like “future generations will look back and consider this among the worst moral crimes in history”. But I still ate factory farmed meat, and I didn’t take any actions that showed I cared. My thinking about factory farming was kind of “academic” or an interesting clever and slightly contrarian view, but it didn’t have any real weight behind it. This is despite me knowing that my thoughts referred to the real world.
I orient very differently to factory farming now. I don’t eat meat, and sometimes when I think about the scale, I feel awful, like I’ve been punched in the gut or that I want to cry, and knowing even then that this reaction isn’t at all sufficient for the actual scale. This feels much more real.
I think that maybe you could use this “fictional” vs “real” framing to say that previously I was thinking about factory farming in a kind of fictional way, and that on some level I didn’t actually believe that my thoughts corresponded to a referent in the real/physical world. But this seems a bit off, given that I did know that these things were in the real world.
It sounds like your gut was thinking of factory farming as living in a fictional world, and as a result your personal planning/behavior operated as though factory farming was in a fictional world. You talked about it the way people talk about e.g. Thanos' evils in the Marvel movies, but didn't do the physical things which would directly impact it through the physical world.
And sure, you may have verbally stated that it was in the physical world if asked, but human minds have parts and those parts can believe different things.
That said, there’s a reason we have a word for “fiction”. Physical reality is special and different from all the other worlds; it can “push back against us” in a way that other worlds can’t.
I frame this the other way around. Fiction is the special thing. Living in the world is what's "normal".
I say this because that's how most minds seem to work. Dogs, as best we can tell, don't really have imagination and the ability to consider hypotheticals. Neither even do many smart animals, or at least not consitently.
Humans are perhaps unique in our ability to consider fictional stories, and some humans struggle to make sense of hypotheticals, which means they struggle to see the distinction between fiction and not. The ability to consider the fictional and see it as fictional is a special skill that sets people apart from animals and in many cases lets some people excel at tasks others struggle with.
3One more thought to add here about AI minds:
LLMs are interesting in that they are perhaps uniquely minds that are all fiction. That is, I'm not sure that LLMs can actually tell fiction and reality apart (in the same way we can) because I'm not sure they've solved the symbol grounding problem, and instead are just so good at manipulating symbols and have such a rich set of training data that having not solved symbol grounding doesn't really matter because they end up lining up with reality anyway.
To the extent they can tell fiction apart from non-fiction, it's because we humans make this distinction, but I wouldn't expect non-fiction to feel different than fiction to an LLM: it's just another frame used for understanding what tokens to generate, but internally probably doesn't look much different than fiction to them other than fictional worlds, for some reason, have much less training data and overlap a lot with the non-fiction world.
But this is maybe not that interesting. If everything's fiction or non-fiction to a mind there's no need to tell the two things apart.
(None of this is to say that LLMs can't tell what's fiction when asked. I'm making a claim about whether an LLM might experience fiction as different from non-fiction the way humans do, and suggesting that they probably don't.)
2
1Do you think they're actually struggling to distinguish real from fiction, or merely struggling to keep two complex distinct worlds in their working memory/stack/context window and keep the details straight?
E.g. many animals will play and chase, understanding both that there's different rules because it's play, yet still transfer the skills to actually hunting or fighting. Seems more a matter of degree?
I think they simply lack the mental machinery to think about hypotheticals in general. So there's no struggle really, because they have no conception of fiction.
It is a matter of degrees, though. Some animals show the ability for social deception. That's a limited form of fictional thinking, and is probably the basis from which humans developed the skill (because chimps can also do some amount of social deception).
As for play fighting, I think this is better understood as a different behavioral mode. It doesn't actually require conceptualization, just an ability to engage in a ritualized behavior with others that may be similar to but is safely different from real fighting. Also don't forget that play fights sometimes accidentally become real fights!
Note that the concepts of "counterfactual" vs "hypothetical" kind of map to this. I tend to think of "scenario" rather than "world" as my container of imagination for these kinds of things, because "world" implies a large-scale consistency that is explicitly missing in a lot of fictional cases, and unavailable for inspection in many "real" and "semi-real (intended to be realistic, but not asserted to be actually occurring)" cases.
I appreciate the concept of "tracking which world the referent lives in". Plausibly tracking the referent-world in your head is useful for preventing blindness to certain optimization channels, not letting the social world constrain you too much,[1] etc.
That being said, I don't think this is true:
I claim that this example generalizes: insofar as Joe’s “fake thinking” vs “real thinking” points to a single coherent distinction, it points to thoughts which represent things in other worlds vs thoughts which represent things in our physical world.
One example from Joe's post that got ingrained in my memory was this passage about Derek Parfit.
And I got a similar vibe, too, from various professors. I would talk to them, and I would get some strange sense like “oh, they are holding some real project. They believe in it, they believe it’s possible to make real progress, they think it’s possible to do this together.” I only met Parfit a few times, but he famously had a ton of this, even as he was dying. “Non-religious ethics is at an early stage,” he writes on the last page of Reasons and Persons. Look, there, that sense of project. And to talk, as Parfit did, about having “wasted much of his life” if his brand of moral realism were false – that too implies a sense of trying to do something.
I take this seriousness and intensity — actually trying to meet reality face-to-face — as core to the distinction that Joe is pointing at (according to my interpretation of it, at least). Parfit actually believed that there is something out there, Reality, that could "hit him in the face", if he got it wrong, and the part of Reality that would hit him in the face would not quite be physical, unless you want to say "It is physical because it is implemented on physical brains." but then we lose the distinction between real thinking and fake/fictional thinking because all thinking is implemented on physical brains (or whatever mind-substrate). Alternatively, you could ground it in the physical world by saying that it should have some implications for the moral convergence of certain classes of agents but I don't think this was essential to Parfit's project and it would be coherent for him to aim for objective moral truth without the assumption that any kind of moral convergence occurs.[2]
Or take math. The monster group most likely isn't instantiated anywhere in reality except on a cognitive/computational substrate that was (downstream from minds) particularly interested in abstract algebra. It is, in the sense you're using this word here, fictional. But does it mean that all thinking about the monster group is fake in the sense that Joe's using this word? I don't think so. Andrew Wiles proved Fermat's Last Theorem, which is straightforwardly physical-world-interpretable in terms of real countable thingies. I think his project, which resulted in a proof of the theorem, is a good example of real thinking. But his thinking would not be less real if, instead of FLT, he chose something about the monster group as a target.
The new proposed terminology doesn't seem like a 1:1 mapping from Joe's dichotomy to me - when reading Joe's writing, it felt like a linear probe into multi-dimensional concept space, while the distinction here sounds like mode collapse 🤔
I cannot use a thermometer to measure the "temperature" of non-equilibrium plasma during super-alfvenic slipping reconnection, even though the ions have some "average speed" inside solar flares - even when everyone would agree about what "really happens" near the Sun about each individual particle, some parts of the bickering about the definitions of what is "thermal" vs "magnetic" could be considered "real thinking" and other parts "fake thinking" based on the usefulness for making predicitions with approximate models, there is no hope to run the physics models on the level of Shroediner equation any time soon and general relativity is continuos theory, so not even computable without approximations.
Fearing sounds of burglars have nothing to do with pressure waves and everything to do with losing money or life - both of which are social reality constructs, not "physical."
Whether or not I shall be forgiven for bluntness, the concept of "physical world" sounds to me like an example of "fake thinking" - as if we wanted to throw away a century of post-modernism instead of learning from it, as if we wanted to regress into less nuance instead of more nuance...
What I find useful about this perspective is that it does point to something about stuff "in the environment" that is opposed to "useless internal thinking loops" when I imagine it applied to thinking about embeded agents - I just don't see the terminology of "physical world" or "objective reality" as new steppings stones towards better understanding - IMHO those steppings stones gave us all the low hanging fruits already in game theory - who even cares about "physical," there is nothing "physical" about the mind, only about the brain and there is no theory of how the mind rises from the brain yet, so 🤷 in that sense, all thinking is "fictional," but some of it is more "useful" and the words "fake vs real" seem better approximation for that intuition compared to "fictional vs real."
Example of a referent for "works on my computer" - shared understanding of a joke about software, when the same code is run in different environments and a bug in the system that contains the code is reproducible only in deployment environments and not on a developer's local machine.
Curated and popular this week