I have been trying to find out what is known about invariants in self-modifying systems. This might become a rather acute topic if we end up moving towards self-modifying AIs or self-modifying ecosystems of AIs.
But it seems that not much has been done. For example, I have found a 1995 Chinese paper, "S-and T-Invariants in Cyber Net Systems" by Yuan Chongyi, Google Scholar page, PDF available which is doing a study of invariants in self-modifying nets (a natural extension of Petri nets), but it only has 4 references to it known to Google Scholar.
I wonder if people know about more examples of this kind of research (or about researchers or organizations currently trying to look at this topic)...
But I should look closer at other parts of those MIRI papers; perhaps there might be some material which actually establishes some invariants, at least for some simple, idealized examples of self-modification...
I have been trying to find out what is known about invariants in self-modifying systems. This might become a rather acute topic if we end up moving towards self-modifying AIs or self-modifying ecosystems of AIs.
But it seems that not much has been done. For example, I have found a 1995 Chinese paper, "S-and T-Invariants in Cyber Net Systems" by Yuan Chongyi, Google Scholar page, PDF available which is doing a study of invariants in self-modifying nets (a natural extension of Petri nets), but it only has 4 references to it known to Google Scholar.
I wonder if people know about more examples of this kind of research (or about researchers or organizations currently trying to look at this topic)...