I saw someone say Microsoft has in 2026 so far patched more vulnerabilities than in the prior 3 years in total. This is of course thanks to AI unearthing vulnerabilities. Glasswing and so on. So what does this kind of pattern mean in an extended sense?
Companies applying AI to harden software means that the software will become "harder" generally than in the past, more base resistance to hacking.
If we fix AI at some level of capability, I think this software hardening would mean, everything else being equal, hacking becomes more difficult and less of an issue.
However, AI doesn't stay still. So new AIs will be able to hack harder software.
This will mean we're moving towards a situation where humans won't be able to find security holes by themselves any more.
There is in fact a limited amount of security vulnerabilities possible. Hard enough software can be unhackable. It's possible we'll move more and more towards some kind of unhackable software world.
But humans will stay hackable. I.e., social engineering will become more important. * Plus, of course, AIs themselves are hackable. Our "patches" (training) for humans is weak compared to what we can do with software, we can't entirely foolproof humans, unlike software which can theoretically be foolproofed.
Since humans are manipulable, this will be another pressure to take humans out of the loop, which as we know, leads to other problems.