One thing this story made vivid for me: once a universe is governed by finite rules, its long-run dynamics may eventually look “hackable” from the perspective of a larger or more expressive rule system. From the inside, agents may experience their values and methods as natural or rational. But there is no guarantee that this process keeps producing genuinely interesting things.
And if our own universe can also be characterized by finite laws, then we are in the same situation. For example, we may not be able to keep doing mathematical research in a “clever” way indefinitely.
Low description complexity is not obviously a good proxy for moral importance. What often matters is distinctive, non-generic structure with internal depth and generative power. The interestingness of our world seems to depend jointly on sufficiently rich rules and on highly specific conditions and histories, which together produce a rich, multi-layered world. A short pointer may pick out something salient or generic, but that does not make it more valuable than a harder-to-specify structure.
Ethically, one might even think that harder-to-compress minds an... (read more)