Agent-foundations researcher. Working on Synthesizing Standalone World-Models, aiming at a technical solution to the AGI risk fit for worlds where alignment is punishingly hard and we only get one try.
Currently looking for additional funders ($1k+, details). Consider reaching out if you're interested, or donating directly.
Or get me to pay you money ($5-$100) by spotting holes in my agenda or providing other useful information.
The laws of physics at the lowest level + initial conditions are sufficient to roll out the whole history, so (in K-complexity) there's no benefit to adding descriptions of the higher levels.
Unless high-level structure lets you compress the initial conditions themselves, no?
Self-arguing on the topic
Counter-argument: The initial state had no structure we could exploit for compression, pure chaos.
Counter²-argument: Any given history that ended up well-abstracting corresponds to a specific inhomogeneous distribution of mass in the early universe, which defined the way the galaxies are spread across it. At least that seems to be the step that could already be compressed. If there were a step upstream of it where the state really didn't have any structure, that unstructured state could be generated by describing the post-structure-formation state, describing the "timestamp" of the post-structure-formation state, then running physics in reverse to generate the unstructured state. So unless the later structured state fails to be lower-description-length than the earlier unstructured state, structure/abstractibility should still allow you to compress the initial state's description, even if the structure only appears later.
Counter³-argumnent: The real "initial state" is the initial state of the quantum multiverse from which all possible Everett branches (and so all possible inhomogeneous distributions of mass, etc.) are generated. Its description length could be incredibly low, such as a uniform point singularity with no expensive-to-describe inhomogeneities whatsoever. The bits you later have to spend to describe the state of your universe are effectively spent on pinpointing the specific Everett branch you're in, but the actual algorithm generating the whole Tegmark III multiverse did not have to do that. It just described the simple state from which all possible branches descend.
Counter⁴-argument: My understanding is that under QM/QFT, the universe doesn't start from a singularity; it's a general-relativity thing. QM/QFT require an initial inhomogeneous universal wavefunction to start working.
Counter⁵-argument: Perhaps the real Theory of Everything unifying QFT and GR would have an initial homogeneous singularity from which all possible Everett branches are generated, and this end result seems plausible enough that we may as well assume it right now.
I don't know enough fundamental physics to make a confident call here. Though...
Counter⁶-argument: There seems to be some process which reallocates realityfluid within Tegmark III as well, between Everett branches. I think this is a hint that the "Tegmark III entire is a single program for the purposes of anthropics/from Tegmark IV's point of view" idea is somehow wrong.
Wait, none of that actually helps; you're right. If we can specify the full state of the universe/multiverse at any one moment, the rest of its history can be generated from that moment. To do so most efficiently, we should pick the simplest-to-describe state, and there we would benefit from having some structure. But as long as we have one simple-to-describe state, we can have all the other states be arbitrarily unstructured, with no loss of simplicity. So what we should expect is a history with at least one moment of structure (e. g., the initial conditions) that can then immediately dissolve into chaos.
To impose structure on the entire history, we do have to introduce some source of randomness that interferes in the state-transition process, making it impossible to deterministically compute later states from early ones. I. e., the laws of physics themselves have to be "incomplete"/stochastic, such that they can't be used as the decompression algorithm. I do have some thoughts on why that may (effectively) be the case, but they're on a line of reasoning I don't really trust.
... Alternatively, what if the most compact description of the lowest-level state at any given moment routes through describing the entire multi-level history? I. e., what if even abstractions that exist in the distant future shed some light at the present lowest-level state, and they do so in a way that's cheaper than specifying the lowest-level state manually?
Suppose the state is parametrized by real numbers. As it evolves, ever-more-distant decimal digits become relevant. This means that, if you want to simulate this universe on a non-analog computer (i. e., a computer that doesn't use unlimited-precision reals) from to starting from some initial state , with the simulation error never exceeding some value, the precision with which you have to specify scales with . Indeed, as goes to infinity, so does the needed precision (i. e., the description length).
Given all that, is it plausible that far-future abstractions summarize redundant information stored in the current state? Such that specifying the lowest-level state up to the needed precision is cheaper by describing the future history, rather than by manually specifying the position of every particle (or, rather, the finer details of the universal wavefunction).
... Yes, I think? Like, consider the state , with some high-level system existing in it. Suppose we want to infer from . How much information does tell us about ? Intuitively, quite a lot: for to end up arising, many fine details in the distant past had to line up just right. Thus, knowing about likely gives us more bits about the exact low-level past state than the description length of itself.
Ever-further-in-the-future high-level abstractions essentially serve as compressed information about sets of ever-more-distant decimal-expansion digits of past lowest-level states. As long as an abstraction takes fewer bits to specify than the bits it communicates about the initial conditions, its presence decreases that initial state's description length.
This is basically just the scaled-up version of counter²-argument from the collapsible. If an unstructured state deterministically evolves into a structured state, those future structures are implicit in its at-a-glance-unstructured form. Thus, the more simple-to-describe high-level structures a state produces across its history, the simpler it itself is to describe. So if we want to run a universe from to with a bounded simulation error, the simplest initial conditions would impose the well-abstractibility property on the whole 0-to-n interval. That recovers the property I want.
Main diff with your initial argument: the idea that the description length of the lowest-level state at any given moment effectively scales with the length of history you want to model, rather than being constant-and-finite. This makes it a question of whether any given additional period of future history is cheaper to specify by directly describing the desired future multi-level abstract state, or by packing that information into the initial conditions; and the former seems cheaper.
All that reasoning is pretty raw, obviously. Any obvious errors there?
Also, this is pretty useful. For bounty purposes, I'm currently feeling $20 on this one; feel free to send your preferred payment method via PMs.
Oh, whoops, I screwed up there. When adapting my initial write-up to this sequence of LW posts, I reordered its sections. Part 4, which explains the idea I'm gesturing at by "the anthropic prior" here, was initially Part 1, and I missed the inconsistency in 2.3 the reordering created. Should be fixed now.
Though I basically just mean "the simplicity prior" here. Part 4 covers why I think the simplicity prior is also the "well-abstractability" prior, and it does so using the anthropics frame, hence "the anthropic prior".
That's probably this post itself, not a shortform? See the segment starting from "the way it can matter".
(Disclaimer: writing off the top of my head, haven't thought too deeply about it, may be erroneous.)
Consider probability theory. In its context, we sometimes talk about different kinds of worlds we could be in, differing by the outcome of some random process. Does that mean we think those other parallel worlds are literal? Not necessarily. In some cases, that's literally impossible; e. g., talking about worlds differing by something we're logically uncertain about. I don't know whether I live in a world in which the th digit of pi is odd or even, but that doesn't mean both worlds can exist.
Yet, talking about "different worlds" is still a useful framework/bit of verbiage.
Next: For whatever reason, the universe favors simplicity. Simple explanations, simple hypotheses, Occam's razor. Or perhaps "simplicity" is just a term for "how much the universe prefers this thing". In any case, if you want to make predictions about what happens, using some kind of simplicity prior is useful.
Anthropic reasoning is, in large part, just a way of reasoning about that simplicity prior. When I say that we're more likely to be coarse agents because they "draw on reality fluid from an entire pool of low-level agents", I do not necessarily mean that there are literal alternate realities populated by agents slightly different from each other at the low level, and that we-the-coarse-agents are somehow implemented on all of them simultaneously, much like there isn't literally several worlds differing by the th digit of pi.
Rather, what I mean is that, when we're calculating "how much the universe will like this hypothesis about reality", we would want to offset the raw complexity of the explanation by taking into account how many observationally equivalent explanations there are, if we want to compute the correct result. The universe likes simplicity, for whatever reason; being alt-simple is one way to be simple; therefore, hypotheses about reality in which we discover that we are "coarse" agents are favored by the generalized Occam's razor. To do otherwise, to argue that we exist in some manner in which our low-level implementation is uniquely defined, goes against the razor; it's postulating unnecessary details/entities.
But all of this is a mouthful, so it's easier to stick to talking about different worlds. They may or may not be literally real, seems like a reasonable bit of metaphysics to me, but I don't really know. It's above our current paygrade anyway, I shut up and calculate.
What observation is better explained or predicted if one assumes Tegmark universes?
Tegmark IV is a way to think about the space of all mathematically possible universes over which we define the probability distribution regarding the structure of reality we are in. In large part, it's a framework/metaphysical interpretation, and doesn't promise to make predictions different from any other valid framework for talking about the space of all possible realities.
Quantum immortality/anthropic immortality is a separate assumption; Tegmark IV doesn't assume it (as you point out, it's coherent to imagine that you still only exist in a specific continuity in it) and it doesn't require Tegmark IV (e. g., in a solipsism-like view, you can also assume that you will never stop receiving further observations, and thereby restrict the set of hypotheses about the universe to those where you will never die – all without ever assuming there's more than one true universe).
What if my utility function is keeping alive and flourishing this one specific instance of myself, connected in an unbroken sequential chain to every previous instance?
Those words may not actually mean anything. Like, if you compute 5+7 = 11, there's no meaningful sense in which you can pick out the "original" 5 out of that 11. You can if the elements being added up had some additional structure, if the equation is an abstraction over a more complex reality. But what if there is no such additional structure, if there's just no machinery for picking out "this specific instance of yourself"?
Like, perhaps we are in a cleverly structured multi-level simulation that runs all Everett branches, and to save processing power, it functions as follows: first it computes the highest-level history of the world, then all second-highest-level histories consistent with that highest-level history, then all third-highest-level histories, et cetera; and suppose the bulk of our experiences is computed at some non-lowest level. In that case, you quite literally "exist" in all low-level histories consistent with your coarse high-level experiences; the bulk of those experiences was calculated first, then lower-level histories were generated "around" them, with some details added in. "Which specific low-level history do I exist in?" is then meaningless to ask.
Or maybe not, maybe the simulation hub we are in runs all low-level histories separately, and you actually are in one of them. How can we tell?
Probably neither of those, probably we don't yet have any idea what's really going on. Shrug.
Perhaps, but we're not aiming for an algorithm able to discover the minimal description of "anything". We're working with an object of a very specific type, that has a very specific structure which could be exploited (perhaps the way these posts sketch out) to find that minimal representation.
I think this is a general problem with these kinds of fundamental impossibility theorems (see also: no-free-lunch theorems, the data-processing inequality, Legg's minimial-complexity theorem). They're usually inapplicable to practical cases, because they talk about highly generic objects which are missing all the structures that make those problems solvable in practice. This is a good post on the topic.
There are effectively infinitely many things about the world that one could figure out
One way to control that is to control the training data. We don't necessarily have to point the wm-synthesizer at the Pile indiscriminately,[1] we could assemble a dataset about a specific phenomenon we want to comprehend.
if we’re talking about e.g. possible inventions that don’t exist yet, then the combinatorial explosion of possibilities gets even worse
Human world-models are lazy: they store knowledge in the maximally "decomposed" form[2], and only synthesize specific concrete concepts when they're needed. (E. g., "a triangular lightbulb", which we could easily generate – which our world-models effectively "contain" – but which isn't generated until needed.)
I expect inventions are the same thing. Given a powerful-enough world-model, we should be able to produce what we want just by using the world-model's native functions for that. Pick the needed concepts, plug them into each other in the right way, hit "run".
If constructing the concepts we want requires agency, the one contributing it could be the human operator, if they understand how the world-model works well enough.
Will e-mail regarding the rest.
It’s funny that I’m always begging people to stop trying to reverse-engineer the neocortex, and you’re working on something that (if successful) would end up somewhere pretty similar to that, IIUC
The irony is not lost on me. When I was reading your Foom & Doom posts, and got to this section, I did have a reaction roughly along those lines.
(But hmm, I guess if a paranoid doom-pilled person was trying to reverse-engineer the neocortex and only publish the results if they thought it would help with safe & beneficial AGI, and if they in fact had good judgment on that question, then I guess I’d be grudgingly OK with that.)
I genuinely appreciate the sanity-check and the vote of confidence here!
Indeed, we might want to actively avoid that.
Perhaps something along the lines of the constructive-PID thing I sketched out.
Yep, I know of this result. I haven't looked into it in depth, but my understanding is that it only says that powerful predictors have to be "complex" in the sense of high Kolmogorov complexity, right? But "high K-complexity" doesn't mean "is a monolithic, irreducibly complex mess". In particular, it doesn't rule out this property:
- "Well-structured": has an organized top-down hierarchical structure, learning which lets you quickly navigate to specific information in it.
Wikipedia has pretty high K-complexity, well beyond the ability of the human mind to hold in its working memory. But it's still usable, because you're not trying to cram all of it into your brain at once. Its structure is navigable, and you only retrieve the information you want.
Similarly, the world's complexity is high, but it seems decomposable, into small modules that could be understood separately and navigated to locate specific knowledge.
Yup, John pointed me that way too.
just noting the minefield that is unfortunately next to the park where we're having this conversation
For what it's worth, I'm painfully aware of all the skulls lying around, yep.
Mm, this one's shaky. Cross-hypothesis abstractions don't seem to be a good idea, see here.
My guess is that there's something like a hierarchy of hypotheses, with specific high-level hypotheses corresponding to several lower-level more-detailed hypotheses, and what you're pointing at by "redundant information across a wide variety of hypotheses" is just an abstraction in a (single) high-level hypothesis which is then copied over into lower-level hypotheses. (E. g., the high-level hypothesis is the concept of a tree, the lower-level hypotheses are about how many trees are in this forest.) But we don't derive it by generating a bunch of low-level hypotheses and then abstracting over them, that'd lead to broken ontologies.
Yup!
Yeah, this is probably handled by something like a system of types... which are themselves just higher-level abstractions. Like, if we discover a new thing, and then "realize" that it's a fruit, we mentally classify it as an instance of the "fruit" concept, from which it then automatically inherits various properties (such as "taste" and "caloric content").
"Truesight" likely enters the play here as well: we want to recognize instances of existing concepts, even if they were introduced to us by some new route (such as realizing that something is a strawberry by looking at its molecular description).