Frontier AI models serve millions of military personnel on classified networks, support operational military targeting, automate scientific pipelines in national laboratories, generate and review significant volumes of production code, and increasingly automate the development of its successors. The more responsibilities AI systems accumulate, the more valuable it becomes for a...
TL;DR. We developed four model organisms of a narrow secret loyalty with Qwen2.5-instruct models (1.5B, 7B, and 32B) that, in certain narrow circumstances, encourage users to take extreme, harmful actions favouring a particular politician. The “narrow secret loyalties” we trained are hard to detect with black-box auditing methods, but detectable...
Epistemic status: a collection of intervention proposals for digital error correction in the context of lock-in. It reflects my own intervention ideas, and the opinion of Formation Research at the time of writing. TL;DR We believe lock-in risks are a pressing problem, and that the digital error correction properties of...
Epistemic status: my own conjecture and speculation after thinking about organisation structures and dynamics as an intervention point for lock-in risk for about 5 hours. My thoughts here represent the opinion of Formation Research at the time of writing. TL;DR We believe lock-in risks are a pressing problem, and that...
Intro So far we have identified lock-in risk, defined lock-in, and established threat models for particularly undesirable lock-ins. Now we present this evaluation benchmark for large language models (LLMs) so we can measure (or at least, get a proxy measure for) the risk level of LLMs. AI is the key...
Epistemic status: a combination and synthesis of others' work, analysed and written over a few weeks. A high-level list of threat models that is open to criticism. TL;DR Humanity could end up in a lock-in within the next century. Here I outline the possible routes to that outcome, and prioritise...