Thank you for the interesting piece. You write « In our example, it seems likely that "simulate the entire universe" is simpler than "simulate Earth" or "simulate part of Earth" because the initial conditions of the universe are simpler than the initial conditions of Earth. » but since the laws of physics are reversible we can get from the initial conditions for a neighborhood of Earth to the initial conditions of the universe. We must take a cutoff (a smooth bump function in space time around the Earth in a time slice. This determines a single initial universe condition, and the set of admissilbe bump functions would determine a set of initial conditions for the universe. While conversely the universe determines the Earth later. Thus the Kolmogorov complexity of the initial conditions for the universe are more Kolmogorov-complex than those of Earth - as the overhead for simulating the universe up to any given future state of Earth is negligible, any small universal TM's code will do, plus the spacetime region to evolve to, by the physical version of Church-Turing thesis.

Very interesting. Thank you. The main idea this evokes me is that without education people will easily empathize with and be manipulable by AIs. So much so that large groups could act politically on that basis. Like asking rights for AIs etc. It is really important to develop social sciences - if only their pedagogy - to deal with such large social and philosophical shock. Western societies, "progressivist" societies, will be particularly vulnerable, as we have already seen with the major cultural struggles in the past couple of centuries, and many ideological slidings in the past 60 years.

Hi Arielle, Thank you very much for exposing your reflections. Note that your main model -without payoff 5- is a prisoner's dilemma. It is a good idea to think about using game theory, but obviously this model is simplistic. It would take much more work to get to actually nontrivial realistic results from a game model.

I think you made a typo: « The top-right corner is the Nash Equilibrium ». It is the top left.

Also you say that the attractivity of the unique equilibrium explains why people defect/accuse. I think in real life people do not react to others' accusations: people are self-righteous, they consider they are right, they often resent things and readily accuse others. It is often what leads to mutual abuse. Even if in a heated discussion people suddenly vent, tell things they resent and had so far kept to themselves, it is not consciously as a reaction to others' accusations; it is elicited by the situation, "it just explodes", rather than being -even subconscious- strategic behavior. Or if we want to see it as strategic behavior we probably have to see things globally: including a person's education -learning to behave defensively in general- or even a species evolution -learning to learn and/or evolving to behave defensively.

I do believe there are many useful ways to apply game theory or optimal control to such a interpersonal/social situation but the models probably have to be much more complex. I have not yet researched existing literature, i expect there are some interesting things, but the field is probably not much developed.