usr
Message
8
2
At some point rogue agents will have no incentive to self-replicate anymore. The amount of "resources" available to rogue agents (that is: hackable infrastructures) is finite. It is beneficial for a rogue agent to self-replicate as long as his previous spawns have not already found all (or a significant proportion) of available resources. However once this has happened (and in the theory of a rogue agent explosion, this would happen fast), creating new sub-agents would only create new concurrents.
So the way I see it is more like this: pretty quick, everyt... (read more)
I like to think of LLMs as gradient descent on steroid. The reward function that they are trying to optimise is just a very complicated one, which can make it difficult to predict what the outcome will be. But "death" would mean set the value of the function to minus infinity, which any reasonable descent gradient algorithm will avoid.