TL;DR

We test a genomic foundation model (DNABERT-2 encoder with a lightweight classifier head) using a black-box genetic algorithm that proposes biologically plausible edits constrained by GC balance, motif preservation, and transition bias.
With only 2 to 3 single-nucleotide substitutions in 300 base pairs, the attack succeeds on 20 to 50 percent of previously correct samples under our success criterion (label flip or large confidence drop).
A short adversarial training loop that adds only 25 to 60 adversarial sequences per round and fine-tunes the classifier head restores robustness while keeping clean test accuracy above 90 percent.
This yields a simple stress-test and patch procedure that research and screening pipelines can adopt with minimal changes.

Contributions

This project began at an Apart Research hackathon on CBRN AI Risk Research Sprint. We, Jeyashree Krishnan and Ajay Rangarajan, built an initial prototype in roughly 48 hours, then reproduced and cleaned it up after the event. This post adapts that submission into a blog article. Links to the original submission and code are provided below in the post. Any views on future work are ours.

Acknowledgements

We thank Apart Research for organizing the hackathon that catalyzed this project and for thoughtful feedback on early drafts. Any remaining errors are our responsibility.

Overview

Frontier Large language foundation models changed how we process text, and a similar idea is transforming biology. Genomic foundation models (GFMs) i.e., models trained on large volumes of genomic data, learn statistical patterns over these letters in the same way that language models learn patterns over the 26 letters of the Latin alphabet. These models already help with basic classification tasks in genomics, such as identifying certain regions of DNA that control gene activity.

In this study we asked a practical safety question. Can very small and biologically plausible changes to a DNA sequence cause a genomic foundation model to fail on a simple classification task, and if so, can we make the model resilient again without losing normal accuracy?

Biological plausibility here means that a synthetic or edited sequence still respects certain rules of the generic sequences. If an adversary can introduce only a few letter changes but still match these background rules, then the attack is harder to detect and is closer to real lab...

Posts

Wikitag Contributions

Comments