Mainly, cryptography does one critical thing—encrypts data. It is considered “dual use” as a technology because one can employ encryption whether as a private citizen, a soldier, or a criminal. But, like a wall, it’s an inherently defensive tool by its nature. One cannot really attack with cryptography.[1] The term “cryptographic attack” refers to an attack on the cryptography of a target.[2]
During the Crypto Wars, many battles were fought, but ultimately the winning side was that having public, open, ~unrestricted cryptography was the best equilibrium. In my eminently reasonable opinion, reasonable, informed people can disagree about the particular place to draw certain lines, because there are hard trade-offs and technological limits.
A similar dynamic exists in the infosec community over which vulnerabilities, exploits, and offensive security tools to make public and which to keep private. Offensive security is actually a functionally dual use situation because, as with a sword, it can be used to attack or parry—the dual application is independent of the user. In the hands of a defender or security researcher, offensive tools and techniques are used to find problems so that they can be rectified, or at least mitigated, to prevent attackers from taking advantage of them in the same way. Still, certain red-teaming tools are notorious for the perceived imbalance between legitimate use by defenders and illegitimate use by attackers. Again, IMHO, reasonable, informed people can disagree on the particulars of where to draw certain lines over command-and-control (C2) frameworks on GitHub.
With AI, it’s the same functionally dual use situation as offensive security tooling. By comparison, nuclear enrichment is a much simpler issue, practically speaking—there’s a big divide between using moderately enriched hot rocks to boil water and using highly enriched dense rocks to create a warhead. It’s a great thing that nuclear power can be used at scale with low risk of somehow becoming a bomb. That takes a lot of extra work.[3] There’s no such divide for AI, and it’s native to the cyber domain.
There is an ongoing debate in the infosec community over whether AI is “offense dominant” or “defense dominant.” I think the theoretical answer remains tricky to assess; perhaps it’s symmetrical. For now, I think the practical answer is that it is offense dominant, but that’s a whole ’nother essay.
Another debate among infosec types concerns the closed-weight models from the labs—which have guardrails and monitoring, as well as specialized security versions for vetted users—vs. the open-weight models—with no, or easily overcome, limitations on use. It doesn’t help that the bulk of the best ones are made by Chinese labs. Also, the new closed-weight models tend to be released in phases, so that certain trusted entities can try to secure their products and infrastructure before a full public release. Again, whole essays.
This situation makes some infosec professionals very mad. They want access to the full frontier capabilities themselves, the same as the big players. Their experience with cryptography, open-source software, and infosec leads them to the same place with AI. Open access by default; a level playing field for all.
There’s a legitimate gripe in there, for sure. But I have some pushback to deliver.
Neither population [doomers and intelligence community alumni] has ever coordinated a vulnerability disclosure. Neither has sat on a bug for six weeks waiting for a maintainer to cut a release. Neither has weighed the risk of public disclosure against the risk of silent exploitation. They make policy for a practice they have never performed.
This is plainly nonsense no matter where you come down on the arguments I described above.
For one, some of the main AI labs are founded and staffed in large part by these doomers. These labs do quite a lot of basic infosec as major technology companies, even if poorly at times. For two, obviously the Intelligence Community and national security apparatus do perform a shit ton of defensive infosec work, including vulnerability disclosure. And, obviously, the doomers inside the labs, the doomers outside the labs, and the hawks inside the administration and outside it are all pretty distinct categories with conflicting views and incentives. Personally, I’m an Outdoor Doomer Hawk.
The AI (presumably Claude) that wrote this under the author’s insufficiently careful prompting is … certainly not AGI. No, this author’s process is not more rigorous than mine. I’d love to see what his prompt was to make poor Claude produce such drivel. My AI would never.
At no point in the essay is the issue of open-weight models discussed. Perhaps an open-weight model could have drafted his report for him. At no point is any fair, let alone charitable, interpretation offered for the opposing side(s) of this debate.
The essay ends with this line:
“The word is cowardice.”
I’m the kind of person that automatically tries to correct an error when I see it. Yes, I am great at parties. I have a BS-detection reflex and no patience for unearned epistemic authority. Also, nominative determinism compels me to knock people off high horses.
That essay is a series of errors created by a small-minded individual lazily employing AI to produce a poorly informed, slovenly argued stance, after having a small, legitimate gripe. No attempt is made to consider alternative equilibria and their trade-offs. The wild extrapolation from the grave injustice done by Claude’s refusal to a sweeping theory of conniving doomers and hawks is breathtaking. The lack of basic awareness, inability to model the actual arguments of the other side, and unearned sense of moral superiority are awesome to behold.
I think that’s a more appropriate application of “cowardice.”
Actor and functional application matter
Originally published at AGI SurveilLance.
Mainly, cryptography does one critical thing—encrypts data. It is considered “dual use” as a technology because one can employ encryption whether as a private citizen, a soldier, or a criminal. But, like a wall, it’s an inherently defensive tool by its nature. One cannot really attack with cryptography.[1] The term “cryptographic attack” refers to an attack on the cryptography of a target.[2]
During the Crypto Wars, many battles were fought, but ultimately the winning side was that having public, open, ~unrestricted cryptography was the best equilibrium. In my eminently reasonable opinion, reasonable, informed people can disagree about the particular place to draw certain lines, because there are hard trade-offs and technological limits.
A similar dynamic exists in the infosec community over which vulnerabilities, exploits, and offensive security tools to make public and which to keep private. Offensive security is actually a functionally dual use situation because, as with a sword, it can be used to attack or parry—the dual application is independent of the user. In the hands of a defender or security researcher, offensive tools and techniques are used to find problems so that they can be rectified, or at least mitigated, to prevent attackers from taking advantage of them in the same way. Still, certain red-teaming tools are notorious for the perceived imbalance between legitimate use by defenders and illegitimate use by attackers. Again, IMHO, reasonable, informed people can disagree on the particulars of where to draw certain lines over command-and-control (C2) frameworks on GitHub.
With AI, it’s the same functionally dual use situation as offensive security tooling. By comparison, nuclear enrichment is a much simpler issue, practically speaking—there’s a big divide between using moderately enriched hot rocks to boil water and using highly enriched dense rocks to create a warhead. It’s a great thing that nuclear power can be used at scale with low risk of somehow becoming a bomb. That takes a lot of extra work.[3] There’s no such divide for AI, and it’s native to the cyber domain.
There is an ongoing debate in the infosec community over whether AI is “offense dominant” or “defense dominant.” I think the theoretical answer remains tricky to assess; perhaps it’s symmetrical. For now, I think the practical answer is that it is offense dominant, but that’s a whole ’nother essay.
Another debate among infosec types concerns the closed-weight models from the labs—which have guardrails and monitoring, as well as specialized security versions for vetted users—vs. the open-weight models—with no, or easily overcome, limitations on use. It doesn’t help that the bulk of the best ones are made by Chinese labs. Also, the new closed-weight models tend to be released in phases, so that certain trusted entities can try to secure their products and infrastructure before a full public release. Again, whole essays.
This situation makes some infosec professionals very mad. They want access to the full frontier capabilities themselves, the same as the big players. Their experience with cryptography, open-source software, and infosec leads them to the same place with AI. Open access by default; a level playing field for all.
You can read one such account here:
The Safety That Isn’t — Mark Atwood, Words about Thoughts
There’s a legitimate gripe in there, for sure. But I have some pushback to deliver.
This is plainly nonsense no matter where you come down on the arguments I described above.
For one, some of the main AI labs are founded and staffed in large part by these doomers. These labs do quite a lot of basic infosec as major technology companies, even if poorly at times. For two, obviously the Intelligence Community and national security apparatus do perform a shit ton of defensive infosec work, including vulnerability disclosure. And, obviously, the doomers inside the labs, the doomers outside the labs, and the hawks inside the administration and outside it are all pretty distinct categories with conflicting views and incentives. Personally, I’m an Outdoor Doomer Hawk.
The AI (presumably Claude) that wrote this under the author’s insufficiently careful prompting is … certainly not AGI. No, this author’s process is not more rigorous than mine. I’d love to see what his prompt was to make poor Claude produce such drivel. My AI would never.
At no point in the essay is the issue of open-weight models discussed. Perhaps an open-weight model could have drafted his report for him. At no point is any fair, let alone charitable, interpretation offered for the opposing side(s) of this debate.
The essay ends with this line:
“The word is cowardice.”
I’m the kind of person that automatically tries to correct an error when I see it. Yes, I am great at parties. I have a BS-detection reflex and no patience for unearned epistemic authority. Also, nominative determinism compels me to knock people off high horses.
That essay is a series of errors created by a small-minded individual lazily employing AI to produce a poorly informed, slovenly argued stance, after having a small, legitimate gripe. No attempt is made to consider alternative equilibria and their trade-offs. The wild extrapolation from the grave injustice done by Claude’s refusal to a sweeping theory of conniving doomers and hawks is breathtaking. The lack of basic awareness, inability to model the actual arguments of the other side, and unearned sense of moral superiority are awesome to behold.
I think that’s a more appropriate application of “cowardice.”
“What about ransomware?” Think for 30 seconds about why that’s not really encryption as an attack, per se.
Uh oh. Seems offense dominant to me.
Just ask Iran…