This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
TLDR: AI security is a pressing concern in both the AI Safety and cybersecurity communities, but I’m not convinced we’re talking about the same kind of AI security. The cybersecurity professionals we desperately need contributing to AI safety are busy focusing on compliance and mitigating reputational damage to their organisations. In this post I posit that this problem exists in large part due to how AI risks are communicated to cybersecurity professionals, and that it is solvable.
(Disclaimer: I run an AI security course called AI Security Fundamentals so many of the cybersecurity professionals I interact with have paid to learn about AI security. However I do my best to either use this knowledge to inform this post, to acknowledge my biases where they might arise, and draw from my other experience in Government and start-ups, too).
What is AI security anyway?
I started working in AI security back in 2020. Back then, it wasn’t considered an AI safety cause area by the effective altruism community. It wasn’t really a cybersecurity problem either, or at least most cyber folk I knew were pretty skeptical that artificial intelligence would change the field in any significant way.
Times have certainly changed.
Over the last year or so, thanks to the rise of agentic AI and then a small news item you may be familiar with (Mythos/Fable anyone?), AI security is one of the hottest skills in AI safety and cybersecurity.
So why is the AI security field still so fractured? Of course, it is still emerging, but why is it that (apart from a few exceptional events like the AI Security Forum and AI Security Bootcamp) of the 5 million or so cyber professionals in the world, more do not seem to care about AI safety?
Protecting AI systems from external attackers (e.g. model theft, data poisoning, prompt injection).
Ensuring highly capable AI systems remain controllable and cannot subvert human intent.
Creating governance mechanisms that prevent dangerous capabilities from being developed or deployed without adequate safeguards.
These are all important areas. The challenge is that most cybersecurity professionals only strongly identify with the first one.
When I ask cybersecurity professionals what they mean by AI security, the answers look very different:
Using AI to improve offensive or defensive cybersecurity.
Securing enterprise AI deployments.
Protecting frontier labs from espionage or sabotage.
Managing AI-related compliance, privacy, and governance risks.
What’s missing? Concerns about misaligned superintelligence, loss of control, or catastrophic societal impacts. Not because cybersecurity professionals reject these concerns, but because they are not the problems they are paid to solve.
What’s the key difference here?
AI security to EAs/AI safety researchers is about securing humans from threats due to highly capable, potentially misaligned AI systems.
AI security to cybersecurity professionals is about protecting their organisations from the kinds of incidents that are likely to get them in trouble: compliance breaches, ransomware, corporate sabotage, data privacy incidents, reputational damage.
Both groups care about security, but they are optimising for different threat models. One is primarily concerned with how increasingly capable AI systems might affect society. The other is primarily concerned with how AI affects the security and resilience of their organisation today.
That’s not to say cybersecurity professionals don’t care about humans or highly capable AI. But clearly there’s some disconnect. So what can we do? I believe there are several things we can do to bridge this gap.
Why listen to me?
I've spent most of my career moving between three worlds that don't naturally talk to each other: machine learning, cybersecurity, and national security.
I started as a data scientist building machine learning systems before moving into cybersecurity roles, eventually working within Australia's cyber and intelligence agency. In 2023 I was an acting Technical Director of the AI Hub within the Australian Signals Directorate (ASD) – Australia’s cyber and intelligence agency, and equivalent of GCHQ (UK) or the NSA (US). At the same time, I was completing a PhD in adversarial machine learning: the study of how machine learning systems can be manipulated, deceived, disrupted, or exploited.
Inside national security circles, AI security was already being treated as a serious strategic issue. Outside those circles, many practitioners and academics I spoke with had never heard the term. I was getting itchy feet and this was a problem I wanted to solve.
There wasn’t any grant funding at that time in AI security so I founded a company, Mileva Security Labs: Australia’s first AI security company.
In hindsight it was a really dumb decision.
It was incredibly naive to think I could somehow make a living solving a problem where there was no funding, no market, and the only evidence that this was a real problem was classified.
I wanted to create an organisation that did AI security research, helped organisations see it was a risk, and help train people in how to mitigate AI security risks. The only problem was that I had to get people to pay for this.
I quickly learned that most organisations weren't particularly interested in discussions about highly capable AI systems or long-term safety risks. But they were interested in data privacy, regulatory compliance, reputational damage, and governance. Fortunately, these were often different manifestations of the same underlying problem.
Over the last few years, lots of IT, risk and cybersecurity teams have suddenly found themselves responsible for a technology they didn't fully understand but knew could create significant organisational risk. That uncertainty created demand for AI security expertise.
As a result, I spent a lot of time acting as a translator between two communities. AI safety researchers tend to talk about alignment, control, and societal impacts. Cybersecurity and risk professionals tend to talk about compliance, threat models, and risk registers. The desired outcome – safer and more secure AI systems – is often similar. The language used to get there is not.
Why don’t cybersecurity professionals seem to care about AI safety and how do we change this?
AI burnout
First of all, cybersecurity professionals tell me they have strong AI burnout. For more than a decade, cybersecurity professionals have been sold wave after wave of supposedly revolutionary AI products. Many delivered marginal improvements. Some delivered nothing at all. Most of the time they were being sold Excel Macros.
By the time generative AI arrived, a lot of cyber professionals had already developed strong immunity to AI hype.
Their mental model of AI is often split between two extremes: overhyped vendor marketing and science fiction (i.e. the Terminator). Neither is very useful for understanding current capabilities. It’s not that they aren’t capable of caring or upskilling in what current AI capability is, it’s just that they have seen the AI hype through so many evolutions now that they’re skeptical the AI of today is really that capable or dangerous. There is even some skepticism that tools like Mythos are really as capable as the hype would make us believe.
Solution: teach them how to build machine learning models, help them understand how it embeds inherent weaknesses in AI products, and empower them to experiment with cutting-edge AI.
My organisation now runs a course where organisations sign up their cybersecurity teams to learn about AI security (it’s called the AI Security Fundamentals course). Our marketing focuses on compliance, governance, risk management and employability. The content covers machine learning weaknesses, AI attacks and defenses, evaluations, and emerging AI safety challenges. We even cover the paperclip maximiser thought experiment. Time and time again I’ve watched awesome but slightly cynical cybersecurity professionals train their first machine learning model, build an AI system around it, and see what happens when it's manipulated.
Without fail, by the time we get to the end of the course they say something along the lines of: “I’m pretty alarmed about these AI safety risks, I think we need to do something about this?” I will say that this is based on my first-hand and potentially biased experience running the course, but it's a trend I am very hopeful can be replicated in other similar formats.
Content and communication
Cybersecurity professionals do not wake up in the morning wondering how to solve existential risk. They wake up wondering whether today's ransomware campaign is going to hit their organisation.
That doesn't mean they are incapable of caring about AI safety. It means AI safety has to compete with a long list of immediate operational problems.
Another mistake many AI safety folk make is assuming that if the arguments are compelling enough, the right people will eventually find them. In practice, information ecosystems really matter. Cybersecurity professionals consume different media, attend different conferences, follow different experts, and participate in different communities.
When cyber professionals go to learn about AI risks they’re not often going to LessWrong, 80,000 Hours, or even research papers. They’re going to look up specific cybersecurity blogs. They’ll be watching security YouTube channels. They’ll be going to BSides conferences. There is increasingly more AI-specific content in these channels but it’s often not focused on existential safety.
Lastly, even when a cybersecurity professional becomes interested in AI safety, there often isn't an obvious pathway for them to contribute. Most cybersecurity professionals first encounter AI security because their current role is expanding to include AI, they're pursuing an internal promotion, or they're considering a move into an AI security position. When they start exploring the field, the opportunities they find are usually focused on enterprise AI security, governance or compliance.
Opportunities to apply cybersecurity expertise directly to frontier AI safety challenges can be harder to identify. Many are concentrated in research organisations, require specialised backgrounds, or involve career paths that look quite different from traditional cybersecurity roles.
As a result, there is a gap between interest and action. A cybersecurity professional may become convinced that advanced AI systems pose important security and safety challenges, but still struggle to answer: "What should I do next?"
If we want more cybersecurity professionals contributing to AI safety, we need clearer pathways that allow them to apply their existing skills without requiring them to completely reinvent their careers.
Solution: provide content that is targeted specifically for cybersecurity professionals, using the terminology they are already familiar with, on the platforms and forums they are used to accessing, and clear pathways to help them transition into something actionable.
The solution isn't simply to tell more cybersecurity professionals about AI safety. It's to make AI safety legible and actionable to people who already think about security for a living.
That means creating content specifically for cybersecurity audiences, using concepts they are already familiar with, distributing it through the communities they already participate in, and providing clear pathways from curiosity to meaningful contribution.
The first AI Security Forum I attended in 2023 was the first time I had seen more than a handful of AI security professionals in the same room. Since then, I've become convinced that we need many more spaces like this – both online and in person. If AI security is going to become a mature field, it needs conferences, communities, training programs, career pathways, and professional networks in the same way cybersecurity does.
More importantly, we should recognise that cybersecurity professionals are not starting from scratch. Cybersecurity has spent decades developing methods for operating in adversarial environments. Many of the challenges emerging in AI are not entirely new problems – they are familiar security problems appearing in a new technological context.
What is missing is translation. Rather than asking cybersecurity professionals to enter the AI safety ecosystem and learn an entirely new vocabulary, we should do more work to meet them where they are. If we can translate the problem effectively, I suspect many more will recognise that they have valuable skills to contribute.
Organisational incentives
The biggest barrier may not be awareness, but incentives. Most cybersecurity professionals work inside organisations. Their training budgets, priorities, and performance metrics are determined by those organisations.
An organisation is unlikely to invest heavily in AI safety because someone presents a compelling argument about existential risk. It is much more likely to invest because it faces regulatory obligations, liability, reputational damage, or competitive pressure. This helps explain why governance, risk, and compliance functions have become some of the earliest adopters of AI security practices. AI regulations, standards, and frameworks create immediate incentives to build capability.
From an AI safety perspective, this can feel annoying and indirect. But it may actually be one of the most effective pathways available.
Compliance requirements create budgets, budgets create training, training creates expertise. Expertise creates people who can later contribute to broader safety challenges.
In other words, we don't necessarily need every cybersecurity professional to start by caring about existential risk. We may simply need them to start learning about AI security.
Solution: improve the accessibility of AI security knowledge, and lean into the existing levers of change in cybersecurity.
One way to overcome this challenge is to reduce our dependence on organisations as the gatekeepers of AI security education. Many cybersecurity professionals are interested in learning about AI security, but to pursue fee-based formal training they need employer approval, budget, and time allocation before they can pursue formal training. Many organisations still view AI security as a niche issue rather than a core capability. Waiting for every organisation to recognise the importance of AI security is likely to be far too slow.
This is why freely accessible training programs and community-driven initiatives are so important (like the AI Security Bootcamp). By lowering the barriers to entry, these initiatives help create a pipeline of practitioners who can later bring that expertise back into their organisations.
At the same time, regulation and standards can play a surprisingly useful role in creating demand for AI security expertise. While compliance is rarely the end goal, it is often what motivates organisations to invest in emerging risks. Requirements to conduct AI risk assessments, implement governance processes, and demonstrate security controls create incentives for organisations to build capability.
I've seen this firsthand through work developing AI governance and assurance frameworks for organisations such as the Australian Bureau of Statistics. In many cases, the initial motivation is compliance or risk management rather than AI safety. However, once organisations begin identifying AI systems, assessing risks, documenting controls, and training staff, they start building the foundations needed to engage with broader AI security and safety challenges.
This is one reason I am more optimistic about regulation than many cybersecurity professionals. Good regulation does not just create paperwork; it creates incentives. If designed well, it can help build the workforce and institutional capability that AI security will require over the coming decades.
We need cybersecurity folk urgently
We cannot have safe AI without secure AI.
A precedent for the AI security challenge is the “crypto wars” of the 1990s, in which end-to-end encryption basically meant governments couldn’t easily wiretap phone lines of suspected criminals, even with a warrant. Governments argued that strong encryption threatened national security because it prevented intelligence agencies from accessing communications. The proposed solution was exceptional access and backdoors. But of course, backdoors don’t only work for governments. They create vulnerabilities that malicious actors can exploit too. The securitisation of encryption created its own security problems.
I’m concerned we will repeat similar patterns with AI, but at a much larger scale. Once a technology becomes framed primarily as a national security asset, the conversation becomes less about building secure and trustworthy systems for society, and more about maintaining strategic advantage over adversaries.
You can already see this emerging: export controls on chips, increasing secrecy around frontier models, debates about open sourcing, and discussions around government oversight of advanced systems. Some of these measures may absolutely be justified. But securitisation also changes incentives in potentially dangerous ways.
If AI capability becomes tightly linked to national power, there is pressure to deploy systems faster, centralise access, tolerate greater risk, and prioritise capability advantage over robustness or security.
This is exactly why cybersecurity expertise matters. Cybersecurity professionals spend their careers thinking about what happens when incentives fail, systems are abused, trusted parties become compromised, or determined adversaries exploit weaknesses that nobody anticipated. These are not edge cases in cybersecurity; they are the norm.
Whenever I talk to cybersecurity practitioners, it is obvious to me that many of them share the same underlying concerns as AI safety researchers. They worry about loss of control, systemic fragility, adversarial misuse, concentration of power, and unintended consequences. They simply approach these problems from a different starting point.
The challenge is not convincing millions of cybersecurity professionals to care about AI overnight. The challenge is building the bridges that allow them to contribute. If we can do that, we gain access to one of the largest existing communities of security-minded professionals in the world. And I suspect many of them will discover that they care about AI safety far more than they initially realised.
TLDR: AI security is a pressing concern in both the AI Safety and cybersecurity communities, but I’m not convinced we’re talking about the same kind of AI security. The cybersecurity professionals we desperately need contributing to AI safety are busy focusing on compliance and mitigating reputational damage to their organisations. In this post I posit that this problem exists in large part due to how AI risks are communicated to cybersecurity professionals, and that it is solvable.
(Disclaimer: I run an AI security course called AI Security Fundamentals so many of the cybersecurity professionals I interact with have paid to learn about AI security. However I do my best to either use this knowledge to inform this post, to acknowledge my biases where they might arise, and draw from my other experience in Government and start-ups, too).
What is AI security anyway?
I started working in AI security back in 2020. Back then, it wasn’t considered an AI safety cause area by the effective altruism community. It wasn’t really a cybersecurity problem either, or at least most cyber folk I knew were pretty skeptical that artificial intelligence would change the field in any significant way.
Times have certainly changed.
Over the last year or so, thanks to the rise of agentic AI and then a small news item you may be familiar with (Mythos/Fable anyone?), AI security is one of the hottest skills in AI safety and cybersecurity.
So why is the AI security field still so fractured? Of course, it is still emerging, but why is it that (apart from a few exceptional events like the AI Security Forum and AI Security Bootcamp) of the 5 million or so cyber professionals in the world, more do not seem to care about AI safety?
The 80,000 Hours AI Security career advice page summarises open problems in AI security well:
These are all important areas. The challenge is that most cybersecurity professionals only strongly identify with the first one.
When I ask cybersecurity professionals what they mean by AI security, the answers look very different:
What’s missing? Concerns about misaligned superintelligence, loss of control, or catastrophic societal impacts. Not because cybersecurity professionals reject these concerns, but because they are not the problems they are paid to solve.
What’s the key difference here?
AI security to EAs/AI safety researchers is about securing humans from threats due to highly capable, potentially misaligned AI systems.
AI security to cybersecurity professionals is about protecting their organisations from the kinds of incidents that are likely to get them in trouble: compliance breaches, ransomware, corporate sabotage, data privacy incidents, reputational damage.
Both groups care about security, but they are optimising for different threat models. One is primarily concerned with how increasingly capable AI systems might affect society. The other is primarily concerned with how AI affects the security and resilience of their organisation today.
That’s not to say cybersecurity professionals don’t care about humans or highly capable AI. But clearly there’s some disconnect. So what can we do? I believe there are several things we can do to bridge this gap.
Why listen to me?
I've spent most of my career moving between three worlds that don't naturally talk to each other: machine learning, cybersecurity, and national security.
I started as a data scientist building machine learning systems before moving into cybersecurity roles, eventually working within Australia's cyber and intelligence agency. In 2023 I was an acting Technical Director of the AI Hub within the Australian Signals Directorate (ASD) – Australia’s cyber and intelligence agency, and equivalent of GCHQ (UK) or the NSA (US). At the same time, I was completing a PhD in adversarial machine learning: the study of how machine learning systems can be manipulated, deceived, disrupted, or exploited.
Inside national security circles, AI security was already being treated as a serious strategic issue. Outside those circles, many practitioners and academics I spoke with had never heard the term. I was getting itchy feet and this was a problem I wanted to solve.
There wasn’t any grant funding at that time in AI security so I founded a company, Mileva Security Labs: Australia’s first AI security company.
In hindsight it was a really dumb decision.
It was incredibly naive to think I could somehow make a living solving a problem where there was no funding, no market, and the only evidence that this was a real problem was classified.
I wanted to create an organisation that did AI security research, helped organisations see it was a risk, and help train people in how to mitigate AI security risks. The only problem was that I had to get people to pay for this.
I quickly learned that most organisations weren't particularly interested in discussions about highly capable AI systems or long-term safety risks. But they were interested in data privacy, regulatory compliance, reputational damage, and governance. Fortunately, these were often different manifestations of the same underlying problem.
Over the last few years, lots of IT, risk and cybersecurity teams have suddenly found themselves responsible for a technology they didn't fully understand but knew could create significant organisational risk. That uncertainty created demand for AI security expertise.
As a result, I spent a lot of time acting as a translator between two communities. AI safety researchers tend to talk about alignment, control, and societal impacts. Cybersecurity and risk professionals tend to talk about compliance, threat models, and risk registers. The desired outcome – safer and more secure AI systems – is often similar. The language used to get there is not.
Why don’t cybersecurity professionals seem to care about AI safety and how do we change this?
AI burnout
First of all, cybersecurity professionals tell me they have strong AI burnout. For more than a decade, cybersecurity professionals have been sold wave after wave of supposedly revolutionary AI products. Many delivered marginal improvements. Some delivered nothing at all. Most of the time they were being sold Excel Macros.
By the time generative AI arrived, a lot of cyber professionals had already developed strong immunity to AI hype.
Their mental model of AI is often split between two extremes: overhyped vendor marketing and science fiction (i.e. the Terminator). Neither is very useful for understanding current capabilities. It’s not that they aren’t capable of caring or upskilling in what current AI capability is, it’s just that they have seen the AI hype through so many evolutions now that they’re skeptical the AI of today is really that capable or dangerous. There is even some skepticism that tools like Mythos are really as capable as the hype would make us believe.
Solution: teach them how to build machine learning models, help them understand how it embeds inherent weaknesses in AI products, and empower them to experiment with cutting-edge AI.
My organisation now runs a course where organisations sign up their cybersecurity teams to learn about AI security (it’s called the AI Security Fundamentals course). Our marketing focuses on compliance, governance, risk management and employability. The content covers machine learning weaknesses, AI attacks and defenses, evaluations, and emerging AI safety challenges. We even cover the paperclip maximiser thought experiment. Time and time again I’ve watched awesome but slightly cynical cybersecurity professionals train their first machine learning model, build an AI system around it, and see what happens when it's manipulated.
Without fail, by the time we get to the end of the course they say something along the lines of: “I’m pretty alarmed about these AI safety risks, I think we need to do something about this?” I will say that this is based on my first-hand and potentially biased experience running the course, but it's a trend I am very hopeful can be replicated in other similar formats.
Content and communication
Cybersecurity professionals do not wake up in the morning wondering how to solve existential risk. They wake up wondering whether today's ransomware campaign is going to hit their organisation.
That doesn't mean they are incapable of caring about AI safety. It means AI safety has to compete with a long list of immediate operational problems.
Another mistake many AI safety folk make is assuming that if the arguments are compelling enough, the right people will eventually find them. In practice, information ecosystems really matter. Cybersecurity professionals consume different media, attend different conferences, follow different experts, and participate in different communities.
When cyber professionals go to learn about AI risks they’re not often going to LessWrong, 80,000 Hours, or even research papers. They’re going to look up specific cybersecurity blogs. They’ll be watching security YouTube channels. They’ll be going to BSides conferences. There is increasingly more AI-specific content in these channels but it’s often not focused on existential safety.
Lastly, even when a cybersecurity professional becomes interested in AI safety, there often isn't an obvious pathway for them to contribute. Most cybersecurity professionals first encounter AI security because their current role is expanding to include AI, they're pursuing an internal promotion, or they're considering a move into an AI security position. When they start exploring the field, the opportunities they find are usually focused on enterprise AI security, governance or compliance.
Opportunities to apply cybersecurity expertise directly to frontier AI safety challenges can be harder to identify. Many are concentrated in research organisations, require specialised backgrounds, or involve career paths that look quite different from traditional cybersecurity roles.
As a result, there is a gap between interest and action. A cybersecurity professional may become convinced that advanced AI systems pose important security and safety challenges, but still struggle to answer: "What should I do next?"
If we want more cybersecurity professionals contributing to AI safety, we need clearer pathways that allow them to apply their existing skills without requiring them to completely reinvent their careers.
Solution: provide content that is targeted specifically for cybersecurity professionals, using the terminology they are already familiar with, on the platforms and forums they are used to accessing, and clear pathways to help them transition into something actionable.
The solution isn't simply to tell more cybersecurity professionals about AI safety. It's to make AI safety legible and actionable to people who already think about security for a living.
That means creating content specifically for cybersecurity audiences, using concepts they are already familiar with, distributing it through the communities they already participate in, and providing clear pathways from curiosity to meaningful contribution.
The first AI Security Forum I attended in 2023 was the first time I had seen more than a handful of AI security professionals in the same room. Since then, I've become convinced that we need many more spaces like this – both online and in person. If AI security is going to become a mature field, it needs conferences, communities, training programs, career pathways, and professional networks in the same way cybersecurity does.
More importantly, we should recognise that cybersecurity professionals are not starting from scratch. Cybersecurity has spent decades developing methods for operating in adversarial environments. Many of the challenges emerging in AI are not entirely new problems – they are familiar security problems appearing in a new technological context.
What is missing is translation. Rather than asking cybersecurity professionals to enter the AI safety ecosystem and learn an entirely new vocabulary, we should do more work to meet them where they are. If we can translate the problem effectively, I suspect many more will recognise that they have valuable skills to contribute.
Organisational incentives
The biggest barrier may not be awareness, but incentives. Most cybersecurity professionals work inside organisations. Their training budgets, priorities, and performance metrics are determined by those organisations.
An organisation is unlikely to invest heavily in AI safety because someone presents a compelling argument about existential risk. It is much more likely to invest because it faces regulatory obligations, liability, reputational damage, or competitive pressure. This helps explain why governance, risk, and compliance functions have become some of the earliest adopters of AI security practices. AI regulations, standards, and frameworks create immediate incentives to build capability.
From an AI safety perspective, this can feel annoying and indirect. But it may actually be one of the most effective pathways available.
Compliance requirements create budgets, budgets create training, training creates expertise. Expertise creates people who can later contribute to broader safety challenges.
In other words, we don't necessarily need every cybersecurity professional to start by caring about existential risk. We may simply need them to start learning about AI security.
Solution: improve the accessibility of AI security knowledge, and lean into the existing levers of change in cybersecurity.
One way to overcome this challenge is to reduce our dependence on organisations as the gatekeepers of AI security education. Many cybersecurity professionals are interested in learning about AI security, but to pursue fee-based formal training they need employer approval, budget, and time allocation before they can pursue formal training. Many organisations still view AI security as a niche issue rather than a core capability. Waiting for every organisation to recognise the importance of AI security is likely to be far too slow.
This is why freely accessible training programs and community-driven initiatives are so important (like the AI Security Bootcamp). By lowering the barriers to entry, these initiatives help create a pipeline of practitioners who can later bring that expertise back into their organisations.
At the same time, regulation and standards can play a surprisingly useful role in creating demand for AI security expertise. While compliance is rarely the end goal, it is often what motivates organisations to invest in emerging risks. Requirements to conduct AI risk assessments, implement governance processes, and demonstrate security controls create incentives for organisations to build capability.
I've seen this firsthand through work developing AI governance and assurance frameworks for organisations such as the Australian Bureau of Statistics. In many cases, the initial motivation is compliance or risk management rather than AI safety. However, once organisations begin identifying AI systems, assessing risks, documenting controls, and training staff, they start building the foundations needed to engage with broader AI security and safety challenges.
This is one reason I am more optimistic about regulation than many cybersecurity professionals. Good regulation does not just create paperwork; it creates incentives. If designed well, it can help build the workforce and institutional capability that AI security will require over the coming decades.
We need cybersecurity folk urgently
We cannot have safe AI without secure AI.
A precedent for the AI security challenge is the “crypto wars” of the 1990s, in which end-to-end encryption basically meant governments couldn’t easily wiretap phone lines of suspected criminals, even with a warrant. Governments argued that strong encryption threatened national security because it prevented intelligence agencies from accessing communications. The proposed solution was exceptional access and backdoors. But of course, backdoors don’t only work for governments. They create vulnerabilities that malicious actors can exploit too. The securitisation of encryption created its own security problems.
I’m concerned we will repeat similar patterns with AI, but at a much larger scale. Once a technology becomes framed primarily as a national security asset, the conversation becomes less about building secure and trustworthy systems for society, and more about maintaining strategic advantage over adversaries.
You can already see this emerging: export controls on chips, increasing secrecy around frontier models, debates about open sourcing, and discussions around government oversight of advanced systems. Some of these measures may absolutely be justified. But securitisation also changes incentives in potentially dangerous ways.
If AI capability becomes tightly linked to national power, there is pressure to deploy systems faster, centralise access, tolerate greater risk, and prioritise capability advantage over robustness or security.
This is exactly why cybersecurity expertise matters. Cybersecurity professionals spend their careers thinking about what happens when incentives fail, systems are abused, trusted parties become compromised, or determined adversaries exploit weaknesses that nobody anticipated. These are not edge cases in cybersecurity; they are the norm.
Whenever I talk to cybersecurity practitioners, it is obvious to me that many of them share the same underlying concerns as AI safety researchers. They worry about loss of control, systemic fragility, adversarial misuse, concentration of power, and unintended consequences. They simply approach these problems from a different starting point.
The challenge is not convincing millions of cybersecurity professionals to care about AI overnight. The challenge is building the bridges that allow them to contribute. If we can do that, we gain access to one of the largest existing communities of security-minded professionals in the world. And I suspect many of them will discover that they care about AI safety far more than they initially realised.
Keen to discuss in the comments!