On a personal level, I would urge taking ordinary precautions in Peter Wildeford’s list
Those are arguably overly optimistic; you can do better. Some extensions:
This is (a) probably overly paranoid and (b) will obviously not save you if the service you want gets hacked server-side or if your active login session gets hijacked, etc. Still, it would protect against some attacks, so you can go for this if this is cheap enough for you, money- and convenience-wise.
More mundanely, you might want to shop around for banks that are particularly adaptable cybersecurity-wise (ask an LLM for a review of the banks in your area as the starting points), open accounts at a few of them, and diversify your savings across them (ideally also across cash + cryptocurrency).
Does anyone have any other good paranoid-prepper ideas for this?
If laptop, ideally you'd physically remove the Bluetooth and Wi-Fi modules, or buy one with them pre-removed. If phone, there are phones with physical off-switches for wireless modules as well.
Use random long syntactically valid passphrases instead of randomly generated bundles of symbols so they're not annoying to type out. E. g., generate N random words, then add more words until it's memorable: "horn advantage tear bullet situation feedback" -> "the horn took advantage to tear the bullet out of the situation for feedback". (Assume the manually added words add zero entropy, of course.)
I have all my important TOTP codes on at least three different devices (my phone, my desktop, and my laptop).
And So It Begins: PHP
Multiple sources using GPT-5.6 Sol identified a critical flaw in PHP. Okay, identified two critical flaws in PHP. The plan was to deploy the patch first before announcing the flaw, but the embargo on the fix got broken more than six hours in advance, so here we are.The PHP Bug Was Kind of a Big Deal
On Your Marks
Now the race is on to patch all the systems before attackers can go on the offensive. OpenAI is helping race to get that done to the greatest extent possible, and are looking for better methods for fixing vulnerabilities in time. This one was important enough the NSA got in on the act. We are going to need better procedures for updates. The best case scenario is that a defender finds the bug first, has enough time to figure out a solution and creates a patch. Then, the moment the patch is rolled out, every attacker gets what are effectively very clear instructions on how to exploit any unpatched systems. Some software has a lot of bugs. Some of that same software powers the internet.This Was Only the Beginning
The same day as the PHP vulnerability we also were treated to this: And this: And also this: And this, probably nothing:Some More Hacks
Things are steadily escalating. Get ready. AI cyber is scary both because it will find new vulnerabilities, and also because it will exploit all the existing dumb vulnerabilities at scale, such as accessing Denmark’s national ID database with the password ‘123456.’ India discloses an official account of rogue AI agent attacks. This looks like another ‘harmless in practice except that it was a government database’ incident.Everything Will Be Attacked
As an example, Andrew Curran resurfaces a notice that there are probes against US-based Siemens PLC installations using AI scripts. AI will make it easier to probe essentially everything. Anything vulnerable will be hit. Even if there is no economic reason to do so, some people want to see the world burn, or want to see the West burn. Hackers used Claude and Grok to automate attacks on banks. The good news is that the banks understand they are primary and high-value targets, and are investing heavily in cybersecurity. The bad news is that their definition of ‘heavily’ and what is actually necessary may be greatly different. Banks also have to deal with the AI agents inside their customer accounts. Most of them are authorized by the customer, but this is one place the customer is not always right, and some of them were not so authorized and it is hard to know which is which. David Sacks suggests read-only permissions as an interim step while banks work to support agents in a safe way and have authorized connections, which seems wise. Then Sacks warns about the ‘irresponsibility’ of ‘blurring the line between user-authorized agentic behavior and cyberattacks.’ Whereas I see the ‘blurring lines’ here as two things. One, the banks being unable to distinguish one from the other, which is a real issue. Two, if Muse (or your OpenClaw, etc) convinces you to let your agent into your bank account, that can quickly turn into a cyberattack, and again the bank cannot tell. If I was the banks, I would absolutely not allow agents to authorize any transactions until I could verify the status and nature of those agents. They have huge downside exposure risk as do the customers. This isn’t crypto (derogatory). Soon days like this might be typical, or even quiet. After all, despite all the issues, life mostly continued as normal. There is no clear line, contra Sacks, between ‘this is a clearly authorized agent’ and ‘this is clearly not an authorized agent.’ Agents Can Just Do Things.Hugging the Face
The original rogue agent incident remains Hugging Face. I wrote quite a lot of posts on the situation, but they are poor explainers for a non-technical person. Luckily, new explainer for the people just dropped. It is very good at explaining the key parts of the Hugging Face incident in plain language. Highly recommended. The information is mostly in the METR report, but in addition to being a story it organizes some of the data into a more highly readable form. Thanks to Raymond Arnold for putting a month of work in on this. I was able to help a little bit with the editing while at the conference last weekend, as were a number of others. If you prefer video we got you, here’s AI In Context with an explainer. HuggingFace CEO Clement Delangue is still on the anti-anthropomorphization train, and still blames the attack on prosaic failures at OpenAI along with his other existing talking points. Also, there is a Netflix ‘instadoc’ called AI Gone Wild about the incident. I have not watched it.Get Daybreak Now
Earlier this week Open Code got hacked as part of a large scale distillation attack, impacting 1.4% of users. Everyone reading this who has something worth defending, try to get Daybreak and similar programs going now, before you are attacked. Do not wait. I will not have sympathy for you if you choose not to do this. It is not as if getting access is difficult. You can mostly get it by asking nicely.Some People Need Practical Advice
It is always a good time for basic personal-level advice. I think something around the level of Wildeford’s advice is wise. You are probably fine with less than that, but if you have to pick a mistake being too prepared is probably the better option. Others recommend going a bit further: Andreas then has the next level down from that, including power banks and Starlink. Things are going to get worse before they may or may not get better.Diffusion Is Slow Until It Isn’t
An excellent point from Dean Ball: Either you believe in ‘bottlenecks’ and slow diffusion, or you do not. You cannot declare ‘everything is fine’ unless you wait long enough to verify they are not becoming non-fine. Partly this is bottlenecks, but a lot of it is purely that diffusion and figuring out how to use the new tools takes time and iteration and People Don’t Do Things. The full version of this looks like ‘because GLM-5.3 is a stronger cyber attacker than previous open models, and the sky did not fall in the first few weeks after its release, that must mean that open-weight Mythos in April would have been fine’ or doubling down on ‘well these particular breaches are fine and due to stupidity.’ This is what you would expect to see if things were slowly moving towards being not fine, despite months of head start. Also, no, GLM-5.3 does not fully have The Juice that Mythos and Astra have. That day will come, and GLM-5.3 is the best cyber offense open model, but I believe GLM-5.3 is still substantially below the April line for this purpose, and thus remains more than six months behind even in ‘fast follower via mass fraudulent distillation’ mode.One, Two, Three, Four
The funniest fail was eMudhra. Sam Jarman:Going Vertical
Thus, we are in a situation where we have both rapid diffusion of use of AI in cyber, both offensively and defensively, and also rapidly improving model capabilities. This is all coming into a world full of extremely buggy and terrible code. The result is a loop feeding on itself and going vertical.Bot Versus Bot
It is not an especially ethical business model. It also might be unavoidable. We have passed the Point of No Return on swarms of attackers and defenders battling it out on cybersecurity, and the defenders had better mostly win. Would it be worse to try and slow down the red queen race? That depends on the details of how you would do that. I think we should be trying to slow it down.Cyber Lack of Security
Anthropic’s CISO speaks truth: He was not the only one giving out warnings. ‘We’ includes not only the AI policy world, but also before that the AI safety world. As in, there are those who said ‘we are building the cybersecurity vulnerability exploiter from the cautionary tale…’ and then those who were saying ‘here is a cautionary tale saying we should not build the cybersecurity vulnerability exploiter,’ and then the Usual Suspects accused everyone involved of lying so they could Do a Hype or a Regulatory Capture. So yes. They did try to warn us. They are still trying to warn us. They, along with all the other major labs and tech companies, are also doing their best to help us get through this, given the capabilities available. Also they are still increasing the capabilities available, about as fast as they can.Offense Versus Defense
How should we think about the PHP bug and a number of other issues being found by 5.6 Sol instead of Mythos or a current frontier model? Roon is correct that the vulnerabilities were first found by defenders. If that had not been true we could be in a rather royal mess right now. This does not invalidate the central point from Nate Soares. Nate is pointing out that the vulnerability was not found by Glasswing for many months, and then found using a model that was behind the public frontier, rather than being found in advance of attackers getting such capabilities. That means that attackers could plausibly have found it first. That defenders happened to get there first in this case is helpful, but not as reassuring as one would like it to be. A particular reason is that defenders actually had a full 11 week lead, as the defenders found this bug on August 1, yet this took so long that we started to see exploitation on October 17. If we need more time than that, we are cooked. I think this is an update against defenders being advantaged, and it is distressing to see people calling Nate’s statement ‘disinformation.’Pick Up the Phone
China has realized there are some issues involved in all this and have solicited comments on revising their AI export controls. They are just like us. Eventually there is going to be a clash between ‘we encourage open models’ and ‘but you cannot let the wrong people get [X].’ You can’t make an open frontier model not [X], or meaningfully apply ‘export controls’ to it. In this case the primary [X] is ‘malicious code generation’ and that is especially impossible to prevent if you want your model writing good code. So you get continuous statements that say both ‘yay models’ and ‘but not [X]’ and eventually something will have to give. Corbin Duncan calls this a ‘big step towards closing the Chinese frontier.’ Dean Ball’s take seems correct here. China is creating optionality, and drawing a date after which you need to ask permission more formally. They do not themselves know if and when they will use it.What Can We Do About It?
One big argument you hear is between these two claims: That we need defenders to have differential advantage, and that we need to diffuse capabilities as quickly as possible to everyone. I see the argument for the second position, ‘spread dual-use capabilities far and wide,’ as being that we are going to do that anyway so we had better do it now, instead. I do not think that is a good argument, or likely to end well, even if you despair and think there is no way we can stop capabilities development or diffusion for long. That still leaves the puzzle of how to get the largest differential advantage for defenders, given control over some but far from all key decisions. You need to strike a balance between how many people you trust to what extent and thus give a leg up, versus the risk that they cannot be trusted. My current read is that we are being far too conservative about who gets access to the tools for defenders. I think we misunderstand the threat model. The situation is mainly not ‘one rogue actor gets their hands on Mythos for a day and all hell breaks loose.’ It is more like ‘those with access to dangerous capabilities continuously ramp up amount of hell breaking loose.’ It also seems likely that proper monitoring and classifiers can tell us if and when one of the accounts get compromised. Even pure ‘KYC we can trust and someone we are confident we can throw in jail’ goes a long way. I expect us to get better at this balance over time. Alternatively, we actually do offer it on that basis, as in Daybreak giving you access in 15 minutes, except no one knows to ask. We need to not wait for them to ask. None of that matters if open model capabilities outpace our ability to patch. We should be committing far more resources to patching, and many are trying, but companies and individuals mostly cannot be forced to get in line. Ideally I’d like to give them more of a swift kick, perhaps by the threat of white hat probes backed by mandatory large bounties. This last week of greater Hackening is plausibly in large part due to the new DeepSeek model. It is not frontier, but it is fast and cheap and pretty good, and that likely is tipping the calculus for many. Now we have a potential new Kimi problem as well. The good news is that Kimi claims to have intentionally held back on the cyber capabilities. Open model advocates will need to accept such things, and other restrictions, over time, and China will provide. A lot of the damage we are seeing is accumulated damage, and a lot of what people are doing seems opportunistic and not well resourced. Thus, a strategy of ‘make doing attacks and using the good models maximally annoying’ as an additional ‘security through obscurity’ strategy might last longer and buy more time than you would think. Eventually the top bad guys scale without limit and none of that matters, the same way that so many things will stop mattering after superintelligence. For now, we need every advantage we can get. On a personal level, I would urge taking ordinary precautions in Peter Wildeford’s list. Again: Assume that things will get worse before they get better, and that there is a long tail of potential ways things could get quite bad. But please do not panic. Do not cancel trips, avoid airplanes or anything like that. Remember this is a long haul problem, not an acute crisis. You want precautions that you can sustain. As with many such things, you do not so much need to outrun the bear as much as outrun the other people, at least in the opening rounds. Then you need to prepare for other things to get taken down.