Long story short: in my assessment, there is an 85% chance we will end up, in the next 24 months, with an open-weights model, or system thereof, capable of "The Juice" that models such as Mythos have, with respect to cybersecurity at the very least. This post goes into why that will likely happen, what the implications are, and how we, as a society and as individuals, can respond to it if/when it does.

First off: why do I say it's so likely? Like, couldn't China just...ban open-weights models, and then that solves the problem? Not so fast. Yes, China currently dominates the open-weights frontier. However, there are also open-weights AI labs in plenty of other countries (US, France, the UAE, South Korea, and Canada come to mind, and I'm sure there are others). Yes, some of these are substantially behind the frontier, but each of the aforementioned countries has an open-weights model no more than 24 months behind the current frontier (that's why I said 24 months earlier); remember, in Aug 2024, 24 months ago as of when I am writing this, the strongest models were Sonnet 3.5 and GPT 4o.

It seems highly unlikely that all of these different countries, with their disparate political systems, geopolitical ties, regulatory environments, and more, will all independently adopt the exact same policy of fully banning all open-weights model releases, at least not until it's too late and an open-weights model capable of Mythos-level capabilities has already been released. And, given the current geopolitical situation, it seems highly unlikely that there will be any international agreement to this effect (at least not until a disaster has already occurred and it's already too late).

But let's say we do manage to successfully conduct an international ban on open-weights Mythos-level models. At that point, you would also need to assume that each and every single closed-weights lab with Mythos-level capabilities also has sufficiently competent cybersecurity to be able to reliably not have their model weights exfiltrated. This seems highly unlikely across so many different labs, and an affirmative duty of having sufficient cybersecurity seems like an even taller order than a ban on open-weights training, which, as discussed before, is, in and of itself, highly unlikely to be possible on an international basis.

Mythos-level capabilities are likely possible with today's open-weight

...

Eoin Cassidy

Eoin Cassidy