This is a post by Abbey Chaver from Coefficient Giving (formerly Open Philanthropy). I recently did a relatively shallow investigation on the state of Infosec x AI. My research consisted of identifying the main GCR-relevant workstreams, looking at relative neglectedness, and trying to understand what skills were most needed to make progress.

The post below reflects this research and my opinions, and shouldn’t be seen as an official cG position.

Summary

There are several high impact, neglected problems that need cybersecurity work. Specifically, I think that more people should work on what I call the AI Infrastructure Security Shortlist:
- Securing model confidentiality and integrity
- Authorization solutions for misuse
- Secure compute verification
- Protocols for securely using untrusted AI labor
- Preventing, detecting, and responding to rogue deployments
The short pitch for working on AI security is, “If AI gets really powerful, we should be really careful about who has access to it, and whether or not it can take unauthorized actions.”
I think that areas like AI red-teaming, cyber evals, and securing critical infrastructure are important, but relatively well-resourced with funding, talent, and awareness. Therefore, I don’t recommend them as a top priority.
I think these shortlist problems are important, but not necessarily easy to solve. It’s very high leverage for people to break these problems down and research what’s feasible. I’d like more people to work on them and share their findings to help guide the rest of the ecosystem.
The theory of change for this shortlist is something like:
- A policy research org (eg, RAND) identifies some meaningful objective (like securing model weights), and proposes some strategic solutions, but does not pursue implementation.
- A technical org (eg, Frontier AI companies; or a startup, such as Irregular) does the R&D to develop those proposals into usable solutions.
- Once solutions exist, they are adopted. This might happen via sales, incorporation into industry standards or open source software, or potentially regulation.
To accomplish this, the AI infra security shortlist seems most bottlenecked on people with experience in:
- Security engineering and research, especially at hyperscalars or in national security contexts
- AI infrastructure engineering (including systems and hardware)
- Standards develop

...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments