Data filtering alone is insufficient for open-weight safety — attackers can still fine-tune models on relevant data or provide it in-context [1, 2]!
I'm worried that companies are treating filtering as a robust solution. However, the robust solution is to prevent open-weight models from learning the target knowledge (e.g., CBRN) or reasoning about it in context. This problem seems bottlenecked on science rather than scaling known techniques. Aggressive filtering may work for a time, but it won't hold up as model capabilities improve. If open-weight developers see their ecosystem's future in jeopardy because of dual-use capability concerns, they should invest significantly in tamper-resistant techniques immediately, especially since this science seems really hard!
I'm somewhat worried that my work on [1] painted a too rosy picture of data filtering. While we do run many empirics and discuss its limitations, in hindsight, the paper was more blue-team (e.g., "look at what filtering can do!"), where it should have been red-team (e.g., "filtering succeeds at preventing learning, which is cool, but is insufficient in the open-weight threat model!"). Moreover, when I was writing [1] in Summer 2025, I softly argued against fine-tuning being a big deal since, at the time, models didn't provide obvious uplift for attackers who already have search tools or relevant data ([1] Footnote: "We note that in cases when search-tool augmentation attacks are possible, LLMs are likely to offer limited uplift because such a search tool could be used by humans directly without the LLM."). In hindsight, this was clearly a case of me not practising good futurism and over-indexing on the chatbot era of Summer 2025. Fast-forward to Fall 2026, I know how to code, yet I still get tremendous uplift with my 24/7 Claude fleet automating my experiments!
All this is to say, I think open-weight developers are only now barely investing in the simplest, yet limited, approach to mitigating risks from
