Zhijing Jin

Imagine two companies racing to build frontier AI models, with increasingly capable systems. Each of them can invest in safety precautions, and these are costly, or cut corners to gain a competitive advantage. Under a certain payoff structure, cutting corners is individually advantageous, whatever the competitor does, even though both...

This is a brief overview of our new paper (📄) AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment accepted at NeurIPS E&D 2026 💫 Overview We introduce AM-Bench, a unified taxonomy and evaluation suite for agentic misalignment, a failure mode where models actively choose harmful actions over failure...

Multi-Agent swarms are important for two reasons: 1. They are probably much faster and somewhat more capable than single agents (Capability) 2. Multi-agent alignment is a different beast than single-agent alignment (Propensity) The fact that recent incidents (and some achievements) are predominantly swarms is empirical evidence for this. On the...

This is the fifth post in the sequence Implications of Continual Learning for LLM Agents. Summary While writing our continual learning sequence, we sent a survey to a number of AI safety researchers with questions about continual learning. This post summarizes the results of that survey. We asked whether respondents...

This is the fourth post in the sequence Implications of Continual Learning for LLM Agents. Summary Continual learning is a capability that largely doesn’t exist yet in LLMs. We first want to acknowledge that this may make it difficult to identify tractable angles of attack for making CL safer: it...

This is the third post in our sequence Implications of Continual Learning for LLM Agents. Summary We argue that continual learning (CL) has two major potential safety implications: it may enable changes to LLM goals and values after deployment, and it eliminates the last-mover advantage held by current safety interventions....

This is the second post in the sequence Implications of Continual Learning for LLM Agents. Summary We say that an agent is a continual learner if it undergoes persistent updates during deployment. That’s more-or-less a binary criterion, but there are several other components to being good at continual learning that...