Wittgenstein's Language Games and the Critique of the Natural Abstraction Hypothesis
4johnswentworth
4particlemania
-5Chris_Leong
4Gordon Seidoh Worley
4Chris_Leong
2Gordon Seidoh Worley
3Noosphere89
2Gordon Seidoh Worley
3Noosphere89
3Tor Økland Barstad
2Chris_Leong
2Chris_Leong
2Chris_Leong
1LVSN
5Lukas_Gloor
4Chris_Leong
4Lukas_Gloor
3Archimedes
4Lukas_Gloor
2Chris_Leong
New Comment
My own take on late Wittgenstein (based on having read only a little of his later work) is that he got wayyyy too caught up in language specifically, and mostly lost sight of the intuitively-obvious fact that words and concepts are not the same thing, nor do they have a stable 1-to-1 matching. (Also he seems to have lost contact with reality in his later work, in the sense that he seemed very hyper focused on things-which-language-can-talk-about. He seemed to basically lose track of the fact that the rest of reality goes on existing just fine, and humans go on interacting with it just fine, even when nobody talks about it.)
"What things do we attach words or phrases to?" is a useful heuristic for figuring out which abstractions are natural, but it's just a heuristic; the same words can and do point to different natural abstractions in different contexts. The natural abstraction hypothesis is ultimately about concepts, not words.
My understanding of Steel Late Wittgenstein's response would be that you could agree with that words and concepts are distinct, and mapping is not always 1-1, but that what concepts get used is also significantly influenced by which features of the world are useful in some contexts of language (/word) use.
Hmm, got some complex thoughts here.
I am suspicious of NAH but for different reasons.
Concepts are contingent upon telos, i.e. they depend on what's useful to the process creating the ontology. So it seems like this contingency should sink the project.
But, reality is the same reality for everything embedded in it (or so it strongly seems), and most processes have some commonality in their telos. For example, most things want ("want" in the sense that they try to get the world into certain states, like the way a thermostat tries to make its sensor read a particular temperature) to survive (continue existing) because of selection effects (things that don't want to survive quickly go away). So most processes model the world in ways that enable their survival.
This might be enough to get instrumental convergence towards common abstractions across a lot of processes. But I think it's unclear yet how much convergence is possible or likely. There's some empirical question about this we have yet to answer because we don't have enough different processes that aren't indirectly influenced by human telos to draw robust conclusions.
So my current guess is that some weak version of NAH is true while a full, stronger version is not. There's some abstractions that many processes will develop because they're commonly useful, but this effect may not be as strong as we hope for, especially at the fringes or under heavy optimization pressure.
Oh, that's interesting. Yeah, I hadn't thought about how instrumental convergence might play into this before. Just thought I'd note that the language game critique is very similar to your telos frame as Wittgenstein has this concept of "language as use" where in most cases language is a tool to achieve a particular result within a particular language game. So it sounds like you're actually suspicious of NAH for mostly the same reasons, but where you depart is that instrumental convergence limits the effects of this divergence.
Yeah, this seems reasonable to me. I'm not deeply familiar with Wittgenstein, so my read of your presentation is that you're paying too much attention to the fact that things are contingent and not enough attention to the fact that the structure of that contingency has a lot of commonality in each case, but I'm not surprised there's a similar idea in his work. Of course this might be my own projection, since I've been pretty guilty of making this mistake and failing to appreciate the extent to which things add up to normality because of common features about how things in the world are constructed.
My expectations are the Natural Abstractions Hypothesis probably works out as long as we don't try to include values/ethics/morality into the mix, so I am more optimistic on the convergence of non moral abstractions.
This is important, because while it wouldn't let us automatically solve the alignment problem, it does make it way easier to change a model's goals.
Why would norms be special here?
The question of what norms to adopt does not appear to be at stake with the NAH, but arguably the structure of norms is—the concepts we use to express norms and constrain the space of possible norms. NAH, if true, should be able to pick out the menu of norms to choose from, say, but then it's a separate question of which norms to order off that menu.
The major point I am making here is that my slightly held belief on the Natural Abstractions Hypothesis is that it probably holds, allowing for cases where it does in fact fail, rather than the alternative hypothesis where the natural abstractions hypothesis doesn't hold at all.
Morality/ethics/values is my proposed failure case/error case, since I don't think even the weak version holds, that is I don't think that there a finite set of valid abstractions of values/morals from the environment.
My expectation is that there is an infinite set of valid moralities, and that's not consistent with even the weak version of the natural abstraction hypothesis.
NAH, refers to the idea that lower-dimensional summaries or abstractions used by humans in day-to-day thought and language are natural and convergent across cognitive systems
I guess whether there is such convergence isn't a yes-no-question, but a question of degree?
Very regularily I experience that thoughts I want to convey don't have words that clearly correspond to the concepts I want to use. So often I'll use words/expressions that don't match in a precise way, and sometimes there aren't even words/expressions that can be used to vaguely gesture at what I actually mean.
Nonetheless, our concepts are similar enough, and we have a similar enough understanding of how words/expressions correspond to concepts, for us to be able to communicate quite a lot (we misunderstand each other all the time, but nonetheless there is a considerable range of stuff that we are able to communicate fairly reliably).
Update: I just gained access to the GPT4 API and I decided to ask it to improve the post.
I used the following system prompt: You are an AI assistant that has been tasked in assisting with writing a high-quality post for the site Less Wrong. You may want to look at the writing of Eliezer Yudkowsky, Scott Alexander or Gwern in order to figure out how to produce high-quality writing.
It provided me with the following which seems better written than the original, at least to me, although it didn't improve on the ideas, just the way they were expressed, mostly just by replacing individual words here and there.
Results:
Introduction
The Natural Abstraction Hypothesis (NAH), put forth by John Wentworth, posits that certain natural abstractions exist, which cognitive systems are expected to converge upon. In this essay, we will scrutinize the NAH through the lens of Ludwig Wittgenstein's concept of language games. We will argue that the variability and contextual nature of language games challenge the NAH, as they illustrate that abstractions are neither fixed, universal, nor entirely convergent across cognitive systems.
Wittgenstein's Language Games
Ludwig Wittgenstein, a prominent philosopher of language, introduced the concept of language games in his seminal work, "Philosophical Investigations." Language games are social practices wherein language and meaning are deeply intertwined. Wittgenstein contended that the meaning of words is not static or inherent but derives from their usage within various language games. These language games are context-dependent, diverse, and ever-evolving, reflecting the intricacies of human social practices and cultural contexts.
Critique of the NAH from the Perspective of Language Games
1. Context-dependent and diverse abstractions
The concept of language games undermines the idea of a single set of natural abstractions that cognitive systems can converge upon. Language games are context-dependent, signifying that the meaning of words, concepts, and abstractions relies on the specific language game being played. Furthermore, the diversity of language games indicates that a multitude of abstractions related to a single concept may exist, each stemming from distinct linguistic and cultural practices. This amalgamation of context-dependence and diversity casts doubt on the notion of convergence towards a single, natural abstraction.
2. Evolution of language games
Language games are not static entities; they evolve over time as social practices and cultural contexts shift. This dynamism suggests that abstractions themselves may be subject to change, further complicating the idea of fixed, natural abstractions. The NAH's assumption that cognitive systems will converge on a consistent set of abstractions may be undermined by the fluid nature of language games and the abstractions they produce.
3. Human-Compatibility
The NAH posits human compatibility, referring to the idea that the lower-dimensional summaries or abstractions employed by humans in day-to-day thought and language are natural and convergent across cognitive systems. However, Wittgenstein's language games demonstrate that human thought and language are diverse, context-dependent, and constantly evolving. This variability complicates the notion that human-compatible abstractions are universally "natural" and brings into question the assumption that disparate cognitive systems will necessarily converge on the same abstractions.
Conclusion
Wittgenstein's concept of language games offers a potent critique of the Natural Abstraction Hypothesis. The context-dependent, diverse, and evolving nature of language games emphasizes the complexities of human linguistic and conceptual practices, which challenge the idea of a fixed, universally natural set of abstractions. By accentuating the importance of context and diversity, Wittgenstein's language games encourage us to reevaluate the assumptions of the NAH and investigate alternative frameworks for understanding the development and convergence of abstractions in cognitive systems.
I decided to give this a go with the new Gemini 2.5 model. I'll present the output first and my comments below:
Prompt: Write a Wittgenstenian critique of John Wentworth's natural abstraction hypothesis.
Answer: Okay, let's approach John Wentworth's Natural Abstraction Hypothesis (NAH) from a Wittgensteinian perspective. This requires setting aside the search for a definitive "theory" of abstraction and instead focusing on how we use the concept of "abstraction" and related terms, and the philosophical confusions that might arise.
Wentworth's Natural Abstraction Hypothesis (Simplified):
NAH posits that abstractions aren't just convenient human constructs but reflect "natural" structural properties of the world. Certain high-level summaries (abstractions) of low-level physical systems are robustly useful for prediction and control because the underlying causal dynamics "clump" or "factorize" in particular ways. Different intelligent systems, interacting with the same reality, will likely converge on similar abstractions because these summaries capture real, objective features of the system's causal architecture. Examples might include concepts like "object," "temperature," or "pressure."
A Wittgensteinian Critique:
A Wittgensteinian critique wouldn't necessarily deny the phenomena NAH points to (e.g., that different models often converge on similar high-level variables like temperature). Instead, it would focus on the philosophical interpretation and the language used, highlighting potential conceptual confusions:
- The Grammar of "Abstraction":
- NAH's Picture: NAH seems to treat "abstraction" as a noun referring to a thing or structure that exists "out there" in the world, waiting to be "discovered" (like a new particle). It speaks of abstractions being "natural" or "objective."
- Wittgenstein's Approach: Wittgenstein would urge us to look at how we use the word "abstraction" and related terms ("summary," "level of description"). We use it in diverse contexts (mathematics, art, science, everyday life). Is there one single essence? Unlikely. Instead, "abstraction" functions within specific language games. In science, an abstraction (like "temperature") is a concept embedded within a practice of measurement, prediction, theory-building, and instrument use. Its meaning arises from this use, not from mirroring a pre-existing "natural abstraction."
- Critique: NAH risks reifying the concept of abstraction, turning a feature of our representational practices into an independent feature of the world. The "naturalness" might simply reflect the success and inter-subjective agreement achieved within certain scientific or engineering language games, driven by shared goals (prediction, control) and interaction with the world.
- "Discovery" vs. "Invention" or "Utility":
- NAH's Picture: The language of "discovery" implies finding something pre-existing and independent of the observer.
- Wittgenstein's Approach: Wittgenstein often dissolved such dichotomies. Are mathematical theorems discovered or invented? The question might be misleading. We develop concepts and rules within a system (a language game, a calculus). An abstraction like "temperature" wasn't simply "found." It was developed alongside practices of thermometry, theories of thermodynamics, and specific technological needs. Its "objectivity" lies in its role within this established, rule-governed practice and its reliable utility for interacting with the world in ways we care about.
- Critique: Calling abstractions "natural discoveries" obscures the role of human (or agent) purpose, criteria, and methods. We select summaries that are useful for our purposes. The causal structure of the world certainly constrains what summaries are useful, but it doesn't uniquely determine the abstraction. Different agents with different goals or sensory apparatus might find different "summaries" useful, yet NAH seems to privilege a particular kind based on physical causality relevant to prediction/control.
- The Lure of the Causal/Physical Picture:
- NAH's Picture: NAH grounds abstraction in the "causal structure of reality." This gives it a pleasingly objective, scientific feel.
- Wittgenstein's Approach: Wittgenstein was wary of "craving for generality" and the tendency to be held captive by a particular picture (e.g., the scientific picture of everything being reducible to physics). While physics describes the world, our concepts (including abstractions) are part of our form of life, our way of navigating and making sense of the world. "Causality" itself is a complex concept whose grammar needs careful examination, not just taken as a bedrock feature onto which abstractions are mapped.
- Critique: NAH might be overly influenced by the physicist's view. Why should physical causal structure be the sole arbiter of what constitutes a "natural" abstraction? Biological abstractions (like "species," notoriously fuzzy), social abstractions ("justice"), or computational abstractions ("function") operate under different criteria and within different language games. Grounding all "natural" abstraction solely in low-level causal summaries seems like an over-extension of one successful, but specific, language game (physics/engineering).
- Levels of Description and Usefulness:
- NAH's Picture: Abstractions are useful summaries because they discard irrelevant low-level detail while preserving predictive power.
- Wittgenstein's Approach: This description of usefulness is fine, but Wittgenstein would ask: Useful for whom? Useful for what purpose? Criteria for "relevance" are not inherent in the low-level details themselves; they are imposed by the agent interacting with the system based on its goals and context. A thermostat needs "temperature." A Maxwell's Demon (hypothetically) might operate better using individual particle kinetics. Which abstraction is "natural"? The question dissolves when we see that the abstraction is defined by its role within a particular system of practice and purpose.
- Critique: NAH's focus on the objective "summarizing" power risks downplaying the constitutive role of the user and their context in determining what counts as a salient or useful abstraction. The "naturalness" seems less about the world in itself and more about a robust relationship between certain world structures and the predictive/control goals of certain kinds of agents.
Conclusion from a Wittgensteinian Standpoint:
A Wittgensteinian wouldn't necessarily say NAH is "wrong" about the empirical observation that similar high-level descriptions often emerge. However, they would be deeply skeptical of the philosophical framing. NAH appears to take concepts deeply embedded in our practices ("abstraction," "summary," "level") and project them onto the world as independent, discoverable entities ("natural abstractions").
The critique would suggest that:
- NAH reifies "abstraction" and seeks a general theory where a careful description of use in different contexts is needed.
- The language of "discovery" obscures the role of criteria, purpose, and practice in forming and validating abstractions.
- The grounding in "physical causal structure" might be an over-generalization from one successful scientific picture, neglecting the diversity of ways we form and use abstractions.
- The "naturalness" likely points to the robust utility of certain conceptual tools within specific forms of life or goal-oriented practices, rather than to observer-independent structures waiting to be found.
Instead of seeking the "objective basis" of abstraction in the world's causal structure, Wittgenstein might suggest we investigate the grammar of how we use abstractions as tools to navigate, predict, and simplify our interactions with a complex reality, acknowledging that these tools are shaped by our needs, goals, and shared practices. The convergence NAH seeks to explain might be better understood through shared environments, shared problems, and the constraints these place on effective action and representation, rather than through discovering pre-packaged "natural abstractions."
This seems to be significantly higher quality than when I tried ChatGPT approximately two years ago. For a start, the answer is now good enough to post, without me having to prompt it to make edits. Secondly, its reasoning feels much less fuzzy. It illustrates its points with examples and engages more closely with the kinds of things that folks might say.
I tried asking it to include examples which didn't work so well. So I instead asked it to generate four examples for each of the three points.
Overall, it's a lot easier to produce quality content when you can increase the maximum content generation size and the ability to generate examples seems useful, but you still need to do a lot manually yourself.
Results:
Certainly! Here are four different examples for each of the three points:
1. Context-dependent and diverse abstractions
a) The concept of "freedom" can yield different abstractions in various political contexts, such as individual liberty in a liberal democracy or collective autonomy in a socialist system (less compelling example, maybe people in a socialist system adopt that definition of freedom, hard for me to gauge really)
b) The idea of "family" can have diverse abstractions across cultures, with some emphasizing nuclear family structures, others prioritizing extended family networks, and some valuing chosen families or non-blood relations (solid example)
c) The notion of "success" can differ across societies, with some emphasizing material wealth and career achievements, while others prioritize personal happiness, community involvement, or spiritual growth (solid example)
d) The concept of "ownership" can manifest differently in various economic and cultural systems, from the individual property rights in capitalist societies to communal ownership in certain indigenous cultures or socialist systems (solid example)
2. Evolution of language games
a) The term "gentleman" has evolved over time, initially referring to a man of noble birth, then encompassing courteous and chivalrous behavior, and now representing a well-mannered and respectful man (not clear how the second and third points differ)
b) The concept of "privacy" has shifted with technological advancements, expanding from physical solitude or personal correspondence to include digital data and online interactions (meaning has probably shifted, unsure if it's the best example as it hasn't shifted that much)
c) The meaning of "friendship" has evolved as social media platforms have emerged, broadening the term to include not only close personal bonds but also casual acquaintances and online connections (not sure that the meaning of friendship has broadened vs. an additional usage)
d) The understanding of "health" has changed over time, moving from a focus on the absence of disease to a more holistic perspective that considers mental, emotional, and social well-being (solid example)
3. Human-Compatibility
a) The perception of "beauty" varies significantly across cultures and cognitive systems, with different aesthetic preferences and ideals shaping the development of distinct abstractions (maybe?)
b) The concept of "intelligence" may have diverse abstractions in different cognitive systems, with some focusing on analytical and logical abilities, others prioritizing emotional intelligence, and some considering creativity or problem-solving skills
c) The idea of "trust" can be understood differently across cognitive systems, such as trust in institutions, trust in personal relationships, or trust in technology and artificial intelligence (seems like mostly the concept in different contexts)
d) The notion of "fairness" may be represented differently in various cognitive systems, such as in human cultures with different moral values or in AI systems with distinct ethical frameworks and decision-making processes (kind of relevant)
Merging these into the original gives:
Introduction
The Natural Abstraction Hypothesis (NAH), put forth by John Wentworth, posits that certain natural abstractions exist, which cognitive systems are expected to converge upon. In this essay, we will scrutinize the NAH through the lens of Ludwig Wittgenstein's concept of language games. We will argue that the variability and contextual nature of language games challenge the NAH, as they illustrate that abstractions are neither fixed, universal, nor entirely convergent across cognitive systems.
Wittgenstein's Language Games
Ludwig Wittgenstein, a prominent philosopher of language, introduced the concept of language games in his seminal work, "Philosophical Investigations." Language games are social practices wherein language and meaning are deeply intertwined. Wittgenstein contended that the meaning of words is not static or inherent but derives from their usage within various language games. These language games are context-dependent, diverse, and ever-evolving, reflecting the intricacies of human social practices and cultural contexts.
Critique of the NAH from the Perspective of Language Games
1. Context-dependent and diverse abstractions
The concept of language games undermines the idea of a single set of natural abstractions that cognitive systems can converge upon. Language games are context-dependent, signifying that the meaning of words, concepts, and abstractions relies on the specific language game being played. Furthermore, the diversity of language games indicates that a multitude of abstractions related to a single concept may exist, each stemming from distinct linguistic and cultural practices. This amalgamation of context-dependence and diversity casts doubt on the notion of convergence towards a single, natural abstraction. For example, the notion of "success" can differ across societies, with some emphasizing material wealth and career achievements, while others prioritize personal happiness, community involvement, or spiritual growth (solid example)
2. Evolution of language games
Language games are not static entities; they evolve over time as social practices and cultural contexts shift. This dynamism suggests that abstractions themselves may be subject to change, further complicating the idea of fixed, natural abstractions. The NAH's assumption that cognitive systems will converge on a consistent set of abstractions may be undermined by the fluid nature of language games and the abstractions they produce. For example, the understanding of "health" has changed over time, moving from a focus on the absence of disease to a more holistic perspective that considers mental, emotional, and social well-being
3. Human-Compatibility
The NAH posits human compatibility, referring to the idea that the lower-dimensional summaries or abstractions employed by humans in day-to-day thought and language are natural and convergent across cognitive systems. However, Wittgenstein's language games demonstrate that human thought and language are diverse, context-dependent, and constantly evolving. This variability complicates the notion that human-compatible abstractions are universally "natural" and brings into question the assumption that disparate cognitive systems will necessarily converge on the same abstractions. For example, the concept of "intelligence" may have diverse abstractions in different cognitive systems, with some focusing on analytical and logical abilities, others prioritizing emotional intelligence, and some considering creativity or problem-solving skills
Conclusion
Wittgenstein's concept of language games offers a potent critique of the Natural Abstraction Hypothesis. The context-dependent, diverse, and evolving nature of language games emphasizes the complexities of human linguistic and conceptual practices, which challenge the idea of a fixed, universally natural set of abstractions. By accentuating the importance of context and diversity, Wittgenstein's language games encourage us to reevaluate the assumptions of the NAH and investigate alternative frameworks for understanding the development and convergence of abstractions in cognitive systems.
The existence of natural abstractions is entirely compatible with the existence of language games. There are correct and incorrect ways to play language games.
Dialogue trees are the substrate of language games, and broader reality is the substrate of dialogue trees. Dialogue trees afford taking dialogical moves that are more or less arbitrary. A guy who goes around saying "claiming land for yourself and enforcing your claim is justice; Nozick is intelligent and his entitlement theory of justice vindicates my claim" will leave exact impressions on exact types of people, who will in turn respond in ways that are characteristic of themselves. Every branch of the dialogue tree will leave an audience with an impression of who is right, and some audiences have measurably better calibration.
Just because no one can draw perfect triangles doesn't mean it's nonsense to talk about such things.
There are correct and incorrect ways to play language games.
That's the crux. Wittgenstein himself believed otherwise and spent the most part of the book arguing against it. I think he makes good points.
At one point, he argues that there's no single correct interpretation for "What comes next in the sequence: '2, 4, 6, 8, 10, 12, ...?'"
Maybe this goes a bit too far. :) I think he's right in some nitpicky sense, but for practical purposes, sane people will say "14" every time and that works well for us.
We can see this as version of realism vs anti-realism debates: realism vs anti-realism about natural abstractions. As I argue in the linked post, anti-realism is probably the right way of looking at most or even all of these, but that doesn't mean "anything goes." Sometimes there's ambiguity about our interpretations of things, but reality does have structure, and "ambiguity" isn't the same as "you can just make random stuff up and expect it to be useful."
1That's the crux. Wittgenstein himself believed otherwise and spent the most part of the book arguing against it.
I could be wrong, but my understanding was that Wittgenstein did think there were correct and incorrect ways of playing language games, but that this was context-dependent, and of course, someone could always choose to play another language game instead.
According to this article, the point being made with the sequences is that the correct completion is subject to interpretation and even though I could try to explain how the sequence should be interpreted, this explanation would itself be subject to interpretation, leading to an infinite regress. Wittgenstein ends up arguing in the end that we learn things through training rather than explanation.
14 is certainly the most likely continuation but it could also be
- 16 if it's a list of numbers k where k^2 + 7 is prime
- 18 if it's a list of numbers of the form 3^i +/- 3^j
These continuations are unlikely in general but are the kind of thing that might show up in an academic mathematics paper.
and some audiences have measurably better calibration.
It's not straightforward in all contexts to establish what counts as good calibration. It's straightforward for empirical forecasting, but if we were to come up with a notion like "good calibration for ethical judgments," we'd have to make some pretty subjective judgment calls. Similarly, something like "good calibration for coming up with helpful abstractions for language games" (which we might call "doing philosophy" or a subskill of it) also seems (at least somewhat) subjective.
That doesn't mean "anything goes," but I don't yet see how your point about dialogue trees applies to "maybe a society of AIs would build abstractions we don't yet understand, so there'd be a translation problem between their language games and ours."
Sorry, I can't quite follow why you are saying that dialogue trees are the substrate of language games or how this ties into the arguments. Any chance you could clarify?
Curated and popular this week