I suspect physics sidechannels[0] will be possible for AGI to exploit until we completely solve physics, and that it may be always possible to implement weird machines[1] on physics or biology. Consider physical / biological stenography of computation. Seeking feedback / instruction / comments from physicists / biologists.
I am skeptical that security is solvable. Even if you fix memory corruption, even if you fix business logic by creating programming languages that enable you to mathematically / formally specify the behavior of your application, the inter...
Perhaps some of the failure modes of traditional bug bounty programs: