Epistemic status: trying to articulate my own thoughts. I have not done a thorough literature review.
James Diacoumis commented on yesterday's post:
the typical objection to such versions of teleosemantics are swamp-man counterexamples: suppose a thermodynamic miracle occurred, with a perfectly formed human spontaneously assembling out of matter in a swamp. This person's thoughts cannot be ascribed semantics in a way that depends on evolution. My version of teleosemantics would be comfortable ascribing meaning to such a person's thoughts, because those thoughts would still be well-understood as being optimized for map-territory correspondence, much like a chess grandmaster's moves are well-explained by the desire to win.
Swampman’s thoughts haven’t been optimised for map-territory correspondence because Swampman hasn’t actually undergone the optimisation process themselves.
If the point is that it’s useful to describe Swampman’s thoughts using the Intentional Stance as if they’ve been optimised for map-territory correspondence then this is fair but you’ve drifted away from teleosemantics because the content is no longer constituted by the historical optimisation process that the system has undergone.
To recap some context:
Teleosemantics is a theory of mind which attributes meaning (semantics) to mental states based on biological purpose. My personal interest in teleosemantics de-emphasizes the biological aspect and focuses on the claim that the meaning of something is what it has been optimized to reflect.[1] Empirically, we can "ground out" all the purposes we see around us into biological evolution. An LLM, for example, was optimized by a gradient-descent algorithm; the gradient-descent algorithm was optimized by a human; the human was optimized by evolution. However, I don't see a need to stipulate this in the theory of semantics.
The Swampman example is a potential counterexample to teleosemantic theory. The example postulates that a man spontaneously forms in a swamp (a thermodynamic miracle due to quantum fluctuation or something along those lines -- vastly improbable but not strictly impossible, quite similar to a Boltzmann Brain). Since the Swampman's purposes cannot be grounded out in natural selection, this poses a potential challenge to versions of teleosemantics that consider this important: if you think Swampman's thoughts are as meaningful as an ordinary man's, this appears to be a counterexample to teleosemantics.
However, I went further in my statement yesterday. To quote it again:
My version of teleosemantics would be comfortable ascribing meaning to such a person's thoughts, because those thoughts would still be well-understood as being optimized for map-territory correspondence, much like a chess grandmaster's moves are well-explained by the desire to win.
James Diacoumis is pointing out a potential problem with the way I'm interpreting "optimization" in the teleosemantic "optimized to reflect". The phrase "have been optimized" can have a connotation of "have been optimized by ___" (we can attribute the optimization to some optimization process, such as a person, or natural selection). In contrast, the sense I had in mind when writing was one which doesn't require an optimizer to be identified: rather, the optimization is identified as a predictively good hypothesis (we can predict many features of Swampman by predicting that they'll score highly on ordinarily biological purposes).
Let's call "have been optimized by ___" style optimization "historically-optimized" (there's an optimizer in its history) and the alternative "structurally-optimized" (the structure of the thing reflects optimization).
I can see a reasonable case for both alternatives, to the point where I currently see it as a take-your-pick situation.
The advantage of historical-optimization, as I see it, is that it won't count coincidental configurations of dust, gasses, etc as "meaningful". Atoms of gas go through many many configurations as they rapidly bounce around. Moreover, there's a whole lot of gasses spread through the universe. Somewhere, at some point in time, some configurations of these will yield meaningful-looking configurations with high map-territory fit, such as "1+1=2". Historical-optimization allows you to rule these cases out, while structural-optimization does not.
You might think that a coincidental "1+1=2" won't "look optimized" in the structural-optimization sense because the hypothesis isn't predictive: on the whole, the usual hypothesis about gasses being randomly configured will do well, and a hypothesis which expects meaningful messages will do poorly.
I think such a defense of structural-optimization probably won't work, because we need to be able to look at things individually.[2] The purposeful hypothesis won't do well at predicting gas-in-general, but it'll do great for this isolated patch of gas at the specific moment in time where it forms the symbols "1+1=2".
Obviously, this is very similar to the Swampman. Swampman gets their improbability by fiat, as a philosophical thought-experiment. The "1+1=2" written in gas particles gets its improbability from searching through the vastness of space and time. However, they're both just low-probability events which create structural-optimization without historical-optimization. If you want such cases to count as having semantics, it seems natural to choose structural-optimization. If you want such cases to lack semantics, it seems natural to choose historical-optimization.
If you look at it that way, it seems like not so much a dilemma as a choice. (James Diacoumis insists that choosing structural-optimization discards teleosemantics, which I think is probably historically accurate; however, it doesn't discard the insight I care about.)
If I were to choose historical-optimization, I would be mostly OK with accepting that Swampman doesn't have thoughts which refer by this definition, because the thought experiment depends on a thermodynamic miracle and this doesn't seem important for AI safety. This is similar to several responses to Swampman that appear in the literature: in The Status of Teleosemantics, or How to Stop Worrying about Swampman David Papineau argues that teleosemantics is a naturalistic theory, similar to "water = H2O", and only has to account for empirical reality, not every hypothetical we can come up with (comparing the Swampman argument to hypothesizing a scenario where a different chemical behaves entirely like water -- the mere hypothetical doesn't start to be an argument against "water = H2O").
If one insists on historical-optimization, but wishes for a theory maintaining that the Swampman's thoughts can in fact refer, then I am tempted to suggest that the historical-optimization comes from outside thought experiment itself; IE, the swamp-man's thoughts are optimized by the thought-experimenter. It isn't entirely clear to me that this suggestion doesn't work. It is similar to suggesting, in the case of the "1+1=2" spelled in gasses, that the optimization comes from the search across space and time for the special occurrence.
More seriously, though: maybe the Swampman's thoughts are initially non-semantic when they first spontaneously appear, but they quickly get optimized (by the brain) for better map-territory correspondence as they look around, think about their situation, etc. In other words, the brain itself is the optimizer which optimizes the thoughts. This is unlike the "1+1=2" example, which falls apart immediately. This route is available to me, though not to the more typical teleosemantic theories which ground out in evolution alone, even if I restrict myself to historical-optimization.
It seems that those interested in teleosemantics are often quite concerned with a distinction between original intentionality vs derived intentionality. For example, A Mark of the Mental: In Defense of Informational Teleosemantics discusses this distinction in chapter 1.
For example, a hammer has derived intentionality: it was made for the purpose of hitting nails by thinking beings who gave it that purpose as a means to their ends.
It does seem natural for teleosemantic theories to grapple with this question, particularly if they choose historical-optimization. After all, teleosemantics isn't typically just about semantics; someone inclined towards teleosemantics will typically choose a similar theory for all purposes (grounding purpose in having-been-optimized-for). Yet, this would appear to make all intentionality derived intentionality. To avoid an infinite regress, we would appear to need to come up with some source of original intentionality.
Based on my limited understanding, it would seem natural for teleosemantic theories to ascribe original intentionality to natural selection (and only natural selection). However, this does not seem to be the case. A Mark of the Mental ascribes original intentionality to humans as well. This seems a bit baffling, for a theory which grounds out purposes in natural selection. Clearly I need to read more about these theories. (I've only read a little bit of A Mark of the Mental.)
My own feeling is that this original/derived idea is somewhat chasing a ghost, like needing there to be something special which elevates the goals of a human above the purposes of a hammer. Granted, there are differences between how humans have goals and how a hammer has a purpose. Original vs derived just seems to me like the wrong frame for that.
The benefit of choosing structural-optimization, so it seems to me, is that one does not need to worry about any infinite regress. Some optimization-like processes can arise by chance, and some of those can select for yet-more-optimization-like processes, and purpose can arise by increments. We don't especially need to worry about original purpose vs derived purpose; it needn't play a significant role in our theory. Those who prefer to affirm that Swampman refers don't need to think about any of my arguments which tried to defend historical-optimization; structural-optimization naturally identifies Swampman as optimizing map-territory correspondence.
The cost is affirming that "1+1=2" means something even if it is written by accident via the random movements of particles.
The truth is, while I do think the above arguments make some good points, I think the discussion here was too loose to draw any firm conclusions. There are far more notions of "optimization" to consider than just two, and it seems plausible that some of my conclusions would be reversed upon more careful consideration. Furthermore, I haven't surveyed the existing literature on teleosemantics or Swampman-arguments adequately.
What about a lie? If I lie to you and say "the sun is green today" then I haven't optimized this statement for map-territory correspondence. Specifically, I haven't optimized my statement to correspond to a territory in which the sun is green today. Nonetheless, this is clearly the meaning of my statement. Is this a counterexample to teleosemantic theories?
On my current account, we can find the semantics by looking at the society-wide optimization process which attempts to keep words meaningful. This includes norms against lying, as well as norms about the mis-use of words. If I say "wire" to mean hose, someone will correct me. Someone could correctly say "Abram means hose" (there is an optimization process which is optimizing "wire" to correspond to hoses, namely myself), but it would also be correct to say "'wire' does not mean hose" (there is another, more powerful optimization process working against the map-territory correspondence I'm trying to set up).
A different way of trying to deal with lies would be to look at what something is optimized to communicate. If I were trying to convince someone of "the sun is green today" you might try to ground out my meaning in what I'm trying to do to that person's mind. However, if we take this as our general theory of semantics, then notice that we have to analyze the meaning of the thoughts in the listener's mind in the same way: thoughts have meaning based on what they try to communicate (to the future-self, or perhaps to others). Meaning would be founded in this infinite chain into the future. I'm not inherently against circular definitions (we can view this as a constraint on meaning), but this doesn't seem like a great situation. (The constraints might end up not constraining very much.)
I'm actually not too sure about this. Maybe we need to consider our understanding of an object in context. This seems like it might be resolved at the level of "what are we trying to make the theory for" rather than something we can resolve in the abstract.