I've been following the AI alignment debate and while the discussion around a superintelligent AI taking over the world is frightening and fascinating, I feel that there are some very real and imminent risks not necessarily dependent on ASI, that deserve more focus. Especially how AI can be both a vector for cyber attacks and produce effective attacks itself. As a bit of an outsider coming from a crypto threat analysis background, I wanted to share my take.
The Influence Conjecture
We’re just barely beginning to understand the extent of AI agents' vulnerability to adversarial prompting. The work done by Zeng et al describes how LLM’s can be manipulated or socially engineered to produce unintended... (read 1484 more words →)