H/T Aella.
A company that made machine learning software for drug discovery, on hearing about the security concerns for these sorts of models, asked: "huh, I wonder how effective it would be?" and within 6 hours discovered not only one of the most potent known chemical warfare agents, but also a large number of candidates that the model thought was more deadly.
This is basically a real-world example of the "it just works to flip the sign of the utility function and turn a 'friend' into an 'enemy'"; this was slightly more complicated as they had two targets that they jointly optimized for the drug discovery process (toxicity and bioactivity), and only the toxicity target is flipped. [This makes sense--you'd want your chemical warfare agents to not be bioactive.] It also required a little bit of domain knowledge--they had to specify which sort of bioactivity to look for, and picked one that would point towards this specific agent.
I think there is a danger that the current abilities of ML models in drug design are being overstated. The authors appear to have taken a known toxicity mode ( probably acetylcholinesterase inhibition - the biological target of VX, Novichok and many old pesticides) and trained their model to produce other structures with activity against this enzyme. Their model claims to have produced significantly more active structures but none were synthesised. Current ML models in drug design are good at finding similar examples of known drugs, but are much less good (to my own disappointment - this is what I spend many a working day on) at finding better examples, at least in a single property optimisation. This is largely because, in predicting stronger chemicals, the models are generally moving beyond their zone of applicability. In point of fact, the field of acetylcholine esterase inhibition has been so well studied (much of it in secret) it is quite likely IMO that the list of predicted highly toxic designs is, at best, only very sparsely populated with significantly stronger nerve agents than the best available. Identifying which structures those are, out of potentially thousands of good designs, still remains a very difficult task.
This is not to take away from the authors’ main point, that ML models could be helpful in designing better chemical weapons. A practical application might be to attempt to introduce a new property (E.g brain penetration) into a known neurotoxin class that lacked that that property. An ML model optimised on both brain penetration and neurotoxicity would certainly be helpful in the search for such agents.
Good analysis, but I did not upvote because of the potential info-hazard that explaining how to use AI to more effectively create hazardous compounds poses. I'd like others to do the same, and you should consider deleting this comment.