Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

[Metadata: crossposted from https://tsvibt.blogspot.com/2022/08/non-directed-conceptual-founding.html. Written 13 June 2022. I'm fairly likely to not respond to comments promptly. If you're especially interested in chatting, my gmail address is: tsvibtcontact ]

In trying to understand minds-in-general, we sometimes ask questions that talk about "big" things (taking "big" to ambiguously mean any of large, complex, abstract, vague, important, touches many things, applies to many contexts, "high-level"). E.g.:

  • What is it for a mind to have thoughts or to care about stuff? How does care and thought relate?

  • What is it to believe a proposition?

  • Why do agents use abstractions?

These "big" things such as thought, caring, propositions, beliefs, agents, abstractions, and so on, have to be analyzed and re-understood in clearer terms in order to get anywhere useful. When others make statements about these things, I'm pulled to pause their flow of thoughts and instead try to get clear on meanings. In part, that pull is because the more your thoughts use descriptions that aren't founded on words with clear meaning, the more leeway is given to your words to point at different things in different instances.[1]

Main claim

From talking with Sam, I've come to think that there's an important thing I hadn't seen sufficiently clearly:

A description of Y that uses terms that are only as "foundational" as Y or even "less foundational" than Y, can still be useful and doesn't have to be harmful. For analyzing "big" things, such descriptions are necessary.

Circular founding

A description is a proposition of the form "Y is a ...". A description is founded on X if it assumes that X exists, e.g. by mentioning X, or by mentioning Z which mentions X, or by relying on X to be in the background.[2]

Some descriptions of Y might be founded on Y, or on X where X is itself founded on Y. A description like that could be called circular, or in general non-directed.

The circularity could be harmful. E.g., you could trick yourself into thinking you're talking about anything coherently, when really you're not: whenever you ask "Wait, what's Y?" you respond "Oh it's XZ", and you say "Z is YX", and you say "X is YZ", and you never do the work of connecting XYZ to stuff that matters, so it's all hot air. Or, you might have "Y" more densely connected to its neighbors, but not beholden to anything outside of its neighbors, so "Y" and its neighbors might drift under their own collaborative inertia and drag other ideas with them away from reality. There are probably other problems with circular founding, so, there's reason to be suspicious. But:

(A) Non-directed founding can elucidate relevant structure;

(B) For "big" things, it's more likely to be feasible to found somewhat-non-directedly, and especially somewhat-circularly, and less likely to be feasible to found strictly in a certain direction;

and therefore

(C) For analyzing and understanding "big" things, non-directed and circular founding are likely to be best-in-class among the available tools.

(A): "Thing = Nexus" as a circular, non-directed, useful founding

As an example, take the description of a thing as an inductive nexus of reference (more specifically, the claim that nexusness points essentially [see below] at the nexus of thingness). This description makes use of a pre-theoretic notion of the "stuff" between which there may be relations of reference, and defines "reference" in terms of what minds in general do. So the definition of nexus is founded on "stuff", which is pre-theoretically on a similar footing to "thing", making the definition of nexus somewhat circularly founded. And, the definition of nexus is founded on "mind", which is a "bigger" concept than "thing", making the definition of nexus founded on something "bigger" than itself, and founded on one of the key things we wanted to analyze and understand at the outset.[3] Still, it seems to me that thinking of Things as nexi clarifies some pre-theoretic thoughts. E.g., it clarifies in what sense one "thinks of the same thing" when using the same word "with the same meaning" despite doing so in two very different contexts for the purpose of doing very different tasks using very different thoughts: one's thoughts access the same nexus. (See the long list below for futher incidental examples of "Thing = Nexus" clarifying things, IMO.)

[In what follows, "term" is used a little ambiguously with "concept" or "idea"; I'm not sure what to do with that.]

Positing that a thing is an inductive nexus of reference--although it doesn't found thingness in terms that are more foundational, that lie more in some particular direction in the space of terms--positing Thing = Nexus is still meaningful and potentially useful because, speaking abstractly, it says something about how the terms in the posit relate to each other. The posit says that, whatever may be (the nexi pointed to by my use of) "mind", "thing", "stuff", and "reference", they stand in some relation, namely the relation that "stuff is a thing when it's an inductive nexus of references that minds follow".

Defending foundationalism

[I picked "foundationalism" in this context without knowing its standard use; at a glance, my use seems to maybe roughly correspond to the standard use. I mean the idea that to be useful / good / worthwhile, ideas have to be iteratively founded all the way to some special, pre-specified, "foundational" kinds of terms.]

Some (overlapping) directions in the space of terms that seem "more foundational":

  • sensical (points to things that are real, points to nexi)
  • concrete (points to things that are durable, solid, reliably there)
  • non-mental
  • physical, causal
  • atomic (indivisible)
  • simple (low algorithmic complexity)
  • simplex (no sub-parts)
  • precise (points to nexi that don't overlap with (get mistaken for) other nexi; cuts (incises) between nexi; joint-carving, arthrodiatomic)
  • specific (points to nexi that include less stuff)
  • definite (marks sharp boundaries around nexi)
  • formal (given meaning by explicit algorithmic rules)
  • connected to sensory experience
  • connected to short-term experience (including non-sensory experience like the event of having an idea in a flash)
  • connected to external stuff
  • connected to actions
  • forms propositions that are logically provable / disprovable
  • forms propositions that are empirically provable / disprovable
  • forms propositions that are likely
  • useful
  • essential (points to stuff that's central in its nexus)
  • unambiguous (points in a way that is successfully followed by your language community)
  • standard (is the expression used by your language community)
  • schelling / canonical (is what someone in your language community might likely also invent de novo, e.g. etymonically sensible terms)
  • algorithmic (can be used as a protocol, a specified behavior)
  • can be used as a machine
  • computable
  • mathematical
  • pre-theoretic, intuitive, unrefined, prima facie
  • understandable
  • communicable
  • readily combinable (e.g. to form propositions or terms)
  • expressive (can be used to describe many other things; this is the sense in which set / type / category theory are foundational)
  • is substrate, is that out of which the other emerges
  • easy (to deal with in whatever way)
  • everyday (connected to everyday life, connected to what you commonly notice and deal with)

Contra possibly-straw Wittgenstein, I think it's silly to broad-strokes deny that many of these directions are real, have reasonably specific meanings, have something-like-maxima, and have a significant amount of global coherence (e.g. medium-locally they roughly induce a somewhat consistently-transitive nontrivial ordering). There might be important necessary background assumptions such as that we're in a community of like minds, or something, but still.

For many of these directions, it's useful and sometimes necessary to iteratively found terms in those directions, i.e. provide descriptions that use terms whose collective minimum or center of mass is further along that direction. For example, if a description is founded on terms that are more canonical and essential, then that description is more reliably communicable because there are fewer ambiguities that need to be resolved, and a theory built of such descriptions is better engineered, less likely to later demand refactoring. If a description is founded on terms that are more connected to experience, then that description is more able to play its role in determining the truth value of propositions it appears in. If a description is founded on terms that are more definite, formal, or mathematical, then that description is more suitable for drawing out logical consequences of propositions that involve it; and those propositions will hence be more amenable to discovering relations with propositions such as implication or contradiction; and that is prerequisite to reaching global logical consistency of beliefs (as well as useful for grounding in experience).

Nevertheless, as gestured at above, positing Thing = Nexus arguably doesn't move in any of these directions, and yet the posit itself is (IMO) useful (whereas the terms in the posit are not obviously more useful than "thing").[4] Just because there are various useful and necessary foundationalisms, doesn't mean non-foundational founding is useless.

(B): Requiring founding foundationward rules out too many moves

An above section argued for (A), i.e. that non-directed founding can elucidate relevant structure. This section argues for (B): for "big" things, it's more likely to be feasible to found somewhat-non-directedly, and especially somewhat-circularly, and less likely to be feasible to found strictly in a certain direction.

Knotted ball of yarn

Say there's a ball of yarn that's not properly wrapped, but rather all knotted up horribly. You can make local progress, tugging out a few strand-segments (somewhere in the interior of the yarn-line, as the ends are buried). But it seems like that doesn't make global progress; you've just slightly transformed the ball, pulling some looseness into one spot and correspondingly tightening nearby wrappings. Someone could say: "Nearly all of the complexity is still there, and what's worse, you haven't made any directional progress, just shuffled things around.". But what you've accomplished is that you taught yourself how to locally unknot stuff, and taught yourself about the structure of the strand-segments on the surface, and looked around for the ends of the yarn. Eventually you might maneuver things more skillfully to make real partial (directed) gains, or find an end and really undo the knot.

Demanding that your changes to the ball of yarn result in no spot being tighter, or result in less total tightness, or in less RMS distance from the center, or in more exposed surface area, might be asking too much. Since the problem is large and complex, asking for global improvements and rejecting moves that leave most of the problem untouched, pressures you to deal with the whole problem at once. Progress of a sort that's less visibly linearly accumulating, can be gained by treating the ball of yarn as worth investigating "at its own level" so to speak, that is, while keeping it mostly intact; like allowing statements about "mind" to include the unanalyzed term "mind".

2-Cat in terms of 2-Cat

Another example is category theory. Notoriously, one can define many structures in category theory in terms of other structures, and vice versa, across many levels of abstraction. An example I bumped into while playing with Lean is that you can elegantly describe a natural equivalence of categories as an equivalence in 2-Cat, the 2-category of categories, functors, and natural transformations. To define what a 2-category is, we can say it's a category enriched over Cat. That involves viewing Cat as a monoidal category, which requires talking about the product of categories, which to be precise about equivalences is most elegantly done by describing the product of categories as the Cartesian product in 2-Cat... but we can't just say that, because in the first place we're trying to construct 2-Cat as a 2-category of categories! Well, there's not exactly a problem; we can make all these descriptions. But there's definitely funny stuff going on. For details follow the links in the paragraph in the section "Idea" about 2-Cat here: equivalence of categories. See also microcosm principle and monoidal category.

"Primordial ooze" is an apt phrase, attributed to Peter Gates, to describe category theory: kind of gross and sticky and gets all over you, yet highly fertile. It's also an apt phrase to describe some "big" things that we analyze using non-directed conceptual founding.

Thoughts about "big" things have many dimensions of potential non-foundedness

Both the funny high-level circular descriptions that don't straightforwardly bottom out in code you can write in Lean, as well as the stuff you actually can turn into code in Lean, is useful. They are moves in a space that has many constraints (many criss-crossing, touching, tangled strand-segments of yarn) and many dimensions (the position of each point along the yarn). These moves probably don't satisfy a given constraint on moves, if that constraint becomes more constraining as there are more dimensions. E.g. the constraint "no new points of contact may be created" becomes more and more difficult to satisfy for longer and longer strands of yarn (each point on the strand increases the dimension of the space of states and of moves).

Thinking thoughts about a "big" thing involves multiple "big" things. Since they're big, they have many chances to create new tension when arranged in new thoughts, and have many chances to move in non-"foundational" directions. So most thoughts about "big" things will create new tensions and move in non-"foundational" directions along some dimensions. So ruling out thoughts that move in such directions on some dimensions, will render infeasible most thoughts that address the center of mass of a big thing.

Conclusion

Statement (C) follows, as a soft implication, from (A) and (B).

Copying from above as a summary:

(A) Non-directed founding can elucidate relevant structure;

(B) For "big" things, it's more likely to be feasible to found somewhat-non-directedly, and especially somewhat-circularly, and less likely to be feasible to found strictly in a certain direction;

and therefore

(C) For analyzing and understanding "big" things, non-directed and circular founding are likely to be best-in-class among the available tools.

To be clear, it's definitely a problem that we're dealing with "big" things. The "big"ness of mind, for example, implies that mind is hard to control from the outside, and doesn't reveal how to control it from the inside. The point of this essay is that it's sometimes counterproductive to try to skip ahead to not dealing with "big" things by pushing questions down foundation-ward, because that gives up on tools that could work. And, forcing things foundation-ward might trade off against sticking with the actual core of the problem that matters.

Thanks to Sam Eisenstat for morally related conversations.

Postscript

Part of what I want from this essay is to be able to say "no I'm not asking for clarification because I think one has to always found one's terms in some direction, I'm asking for clarification because I don't know what you're talking about", and have it be in common knowledge that I could possibly really mean the first clause. Another part of what I want is to make the reasoning more explicit so I don't actually confuse "hyper-foundationalism" with other reasons I might want more foundational founding.

Footnotes


  1. There are other reasons to get clear on meanings, e.g. interlocutors want to at least have the same pre-theoretic idea in mind in order to communicate at all, and words are not always so obliging in having only one pre-theoretic idea they point to. ↩︎

  2. [On reflection the following distinction is unrefined and not needed for the essay. "Founding" and "description" are also not really refined. The word "founding" is taken from Heidegger, though maybe not with the same meaning.] Grounding is a strict superset of founding, e.g. grounding an idea by using it to make predictions about sensory experience isn't founding. Founding connects Y to X by mentioning X; grounding connects Y to X in any usefully constrained way. Using Y as a term in a program that spits out predictions about X is not a description of Y involving X, since the program is not a proposition of the form "Y is a ...", but does ground Y in X. (This is in tension with some categories of foundationalness, such as algorithmicness.) ↩︎

  3. We can go further and note that "inductive" also relies on minds to some extent. And "nexus" is mathematically flavored but very vague about what constitutes a "sufficiently densely connected region" of a weighted digraph. (Though the vagueness may be a feature because thingness is also imprecise, following perhaps the same nondiscrete contours and nondiscrete dichotomies as the structure of the digraph of references.) ↩︎

  4. Though some terms certainly move in some of these directions. E.g. the more mathematical notion of weighted digraphs (or quasi-metrics, maybe) is implicitly used. (Albeit maybe somewhat metaphorically / vaguely / unfoundedly; what are "vertices" of "stuff"?) ↩︎

New to LessWrong?

New Comment
3 comments, sorted by Click to highlight new comments since: Today at 3:40 PM

For what it's worth, I think you're running headlong into an instance of the problem of the criterion and enjoy seeing how you're grappling with it. I've tagged this post as such.

Does it seem like I'm missing something important if I say "Thing = Nexus" gives a "functional" explanation of what thing is, i.e. it serves the function of being an "inductive nexus of reference". This is not a foundational/physicalist/mechanistic explanation, but it is very much a sort of explanation that I can imagine being useful in some cases/for some purposes.

I'm suggesting this as a possibly different angle at "what sort of explanation is Thing=Nexus, and why is it plausibly not fraught despite it's somewhat-circularity?" It seems like it maps on to /doesn't contract anything you say (note: I only skimmed the post so might have missed some relevant detail, sorry!), but I wanted to check whether, even if not conflicting, it misses something you think is or might be important somehow.

I'm not sure I understand your question at all, sorry. I'll say my interpretation and then answer that. You might be asking:

Is the point of the essay summed up by saying: " "Thing=Nexus" is not mechanistic/physicalist, but it's still useful; in general, explanations can be non-mechanistic etc., but still be useful, perhaps by giving a functional definition of something."?

My answer is no, that doesn't sum up the essay. The essay makes these claims:

  1. There many different directions in conceptspace that could be considered "more foundational", each with their own usefulness and partial coherence.
  2. None of these directions gives a total ordering that satisfies all the main needs of a "foundational direction".
  3. Some propositions/concepts not only fail to go in [your favorite foundational direction], but are furthermore circular; they call on themselves.
  4. At least for all the "foundational directions" I listed, circular ideas can't be going in that direction, because they are circular.
  5. Nevertheless, a circular idea can be pretty useful.

I did fail to list "functional" in my list of "foundational directions", so thanks for bringing it up. What I say about foundational directions would also apply to "functional".