Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

If asked to define what an agent is, my usual answer -- one of them, anyway -- is “a worldline in Tegmark V”.

The Tegmark Level V Multiverse (the “V” here is a Roman numeral) is not defined by Max Tegmark (whose hierarchy goes only up to IV), but, as used in agent-foundations circles, it refers to the collection of not-necessarily-consistent mathematical universes, a.k.a. “impossible possible worlds”. It thus contains worlds in which 1+1 = 3, worlds in which triangles have four sides, and perhaps worlds with married bachelors -- in addition, of course, to all the more “ordinary” worlds in Tegmark Levels I-IV (and thus, in particular, us).

This definition of “agent” is intended to evoke the concept of an observer in physics (especially relativity), which is a worldline in physical spacetime. “Observer” is a more passive word than “agent”, corresponding to the fact that in physics, worldlines are determined by equations (the “equations of motion”) that represent the laws of physics; whereas the idea in agent theory is that the corresponding equations -- those that determine worldlines -- represent the preferences, or “caring structure”, of some entity other than (at any rate, not necessarily identical to) the physical universe.

(I am deliberately avoiding the terms “values” and “goals”, for obscure theoretical reasons that I won’t explain here.)

A worldline in Tegmark IV (to say nothing of V) would, almost by its very nature, suggest a higher degree of “agency” than the ordinary sort of worldline, because it would allow for the possibility that the “observer” or “agent” moves between universes with differing laws of physics. Were we ever to acquire the capacity of “hacking into” our universe and changing its physical laws, for example, this would be the sort of mathematical setting in which our activities would be appropriately modeled. The equations governing our trajectories in that case would be reasonably termed “laws of metaphysics” -- or, indeed (in the most general case at least), “laws of mathematics”.

Importantly, note that this setting enables us to reason counterfactually about physical laws (that is, about “metaphysical location”), in exactly the same way that ordinary physics allows us to reason counterfactually about physical location (e.g., “if I were here at this time, then I would be there at that time”).

The next step in this progression, Tegmark-V worldlines, might seem absurd at first: hacking into the laws of mathematics, and traveling into the world where 1+1=3? But the subject of logical uncertainty, and in particular logical counterfactuals, has indeed been receiving attention lately. Essentially, it arises out of situations in which agents need to construct models -- that is, maps that aren’t the territory -- not only of the physical world, but of mathematical truths: for example, to reason about how their own or other agents’ “source code” works. In other words, situations in which agents reason about agency.

Doing so is tantamount to reasoning counterfactually about the laws of mathematics. Consequently, reasoning about agency -- something that we would like agents to be able to do, as a result of our having done so -- is equivalent to reasoning about the “physics” of Tegmark V: meta-metaphysics, or metamathematics.

New Comment
12 comments, sorted by Click to highlight new comments since: Today at 2:38 PM

Not sure "worldline" is the best word, because each observer-moment exists in many locations at once, each having many possible pasts and futures. So I think of an observer-moment as a "chorus" of locations in Tegmark V.

From the perspective taken in this post, "location" means observer-moment: the entire submanifold of "simultaneous" locations in your sense is represented by a single point in the space I mean.

(To be sure, both your space and mine are "Tegmark V" spaces; "Tegmark V" here is not a specific mathematical object, but an interpretation-type.)

A subsequent post may provide helpful context.

[-]TAG6y20

yes, a worldine requires a somewhat restricted structure, which you are not going to get in level IV, let alone V. Also, the OP never explain what worldlines have to do with agents, or what the advantage of this way of thinking is.

Is each chorus in T5 also a point in T5?

I'm a little confused.

It seemed to me that the way Tegmark had put it, level IV is meta-closed: any consistent map of (even possibly eventually inconsistent) maps is still just a consistent map in level IV; it doesn't have to model any particular territory, it just has to be a mathematical "structure". Maybe you're saying that this is actually in level V and my view of level IV is too inclusive (but I think Tegmark would disagree with you, see esp. appendix A of his original paper), or maybe I missed your point altogether.

It's not even clear that there would be a notion of an "agent" in every level IV universe (in fact I'd say it's clear that this is NOT the case), so I think the idea of a worldline between them would not be well-defined. Nevertheless, I'm fine with non-standard uses of terms if it helps communicate the idea you have after you've clarified your usage of them (but don't canonize independently in an already nebulous territory!), but I'm having some trouble with that.

So can you clarify what you mean, particularly by level IV? (reasonably precise english is fine :P)

ETA: Okay, given your Mathematical Mindset post, I'm doubly fine with your redefinition, but I still want it :3

Why think of an agent this way instead of as something that reasons about counterfactuals and has logical uncertainty?

An agent could live in the simulation where the simulation owners constantly correct any 2+2=4 to 2+2=5. I think that it may be interesting but cruel experiment, but some simulations could of this "fake-math" type.

More interestingly is that we could have subjective experiences, or qualia, of thing, which are not logically possible, like round square, and if qualia are final bricks from which reality is built, impossible qualia will be possible bricks for the reality.

An agent could live in the simulation where the simulation owners constantly correct any 2+2=4 to 2+2=5

This is a form of gaslighting, isn't it? Constantly being forced to believe that your logic, perceptions, feelings and needs are wrong. In that way we all live in this unreality and are used to it, taking various versions of 2+2=5 as unquestionable truth. More often than not, some people fervently believe that 2+2=3 (e.g. "Trump is Jesus") and others believe that 2+2=5 (e.g. "Trump is Hitler"). Notice your automatic emotional reaction to the above examples.

Aristotle wrote that spider has 6 legs. It took 2300 years to actually count the number and find that there are 8 legs. Lamarck did it in 18th century. So humans can gaslight themselves, and there could be some mathematical examples.

Do you have a source for that?

Ups, it seems to be fake news (((

I find an analysis here (need translation), where it is said that "ephemeron" had 4 legs according to A, but that he personally never have seen the beast and now we even can't say what it was. Later the story was transformed into the urban legend about the spider or fly with wrong number of legs.

It feels like your logic is inverted. What you describe as T5 is a cognitive process, not anything more "objective". So it is inside the agent, not the agent being inside it.

Of course, the same applies to T4, but then Max Tegmark has so much charisma, event smart people tend to accept his musings uncritically.