Posts

Sorted by New

Wiki Contributions

Comments

Perhaps some of the failure modes of traditional bug bounty programs:

  • Underpaying bugfinders ("gig economy-ification", versus hiring someone into a consulting firm)
  • Liability avoidance by firms
  • Deeper, more serious bugs/malicious prompts are overlooked

I suspect physics sidechannels[0] will be possible for AGI to exploit until we completely solve physics, and that it may be always possible to implement weird machines[1] on physics or biology. Consider physical / biological stenography of computation. Seeking feedback / instruction / comments from physicists / biologists.

I am skeptical that security is solvable. Even if you fix memory corruption, even if you fix business logic by creating programming languages that enable you to mathematically / formally specify the behavior of your application, the interaction of your application with reality, across the silicon/reality boundary, will almost always have leaky abstractions until we thoroughly understand physics and will always fail at the human behavior / game theory / social deception / hidden preferences level.

The current economic / systemic incentives for the construction of our computer / noncomputer systems do not reward doing things "correctly" / "securely" for most use cases (notable exception: aviation but c.f. Boeing 737 MAX[2]). This is a tremendous economic liability regardless of whether or not AGI exists. There are probably useful concrete actions (design a logic programming language that is usable by most existing developers to encode business logic by writing something that resembles math, or push forward static analysis / fuzzing research to eliminate entire classes of software vulnerability). 

[0] https://en.wikipedia.org/wiki/Tempest_(codename)

[1] https://en.wikipedia.org/wiki/Weird_machine

[2] https://en.wikipedia.org/wiki/Boeing_737_MAX#Grounding_and_recertification