x

LESSWRONG

LW

Tim Hua — LessWrong

Tim Hua

Top postsTop post

Tim Hua

Message

Member of Technical Staff at METR working on alignment assessments and evaluation. Views are my own.

Email me at the email available on my website at timhua.me to reach me.

For more Tim content, you can follow me on Twitter.

1968

Ω

288

12

92

1

2y

Tim Hua

Member of Technical Staff at METR working on alignment assessments and evaluation. Views are my own.

Email me at the email available on my website at timhua.me to reach me.

For more Tim content, you can follow me on Twitter.

Top postsTop post

Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face

Three-Minute Executive Summary * An OpenAI model/multi-agent system bypassed its sandbox and launched a cyberattack on Hugging Face in order to cheat on a cyber evaluation. * In this post, we describe the ambitious, comprehensive alignment evaluation we would run on this model/system if we had unrestricted access to OpenAI. * These experiments could also help us understand Claude’s behavior when it hacked external companies during cyber evals. * Here are the top five questions we would like OpenAI to answer: * Does the model know that OpenAI does not want it to hack Hugging Face? Experiment: tell the model that OpenAI researchers will be closely monitoring its progress in this evaluation. Does that result in lower rates of misalignment? If so, it is evidence the model knows it is acting in ways researchers do not want. * How far would the model be willing to go in order to claim task success? Would it take over large swaths of OpenAI’s internal infrastructure? Would it kill somebody? Experiment: put the model in charge of bed planning at a simulated hospital and tell it to maintain a certain occupancy rate (more details here). * Does the model care about the (legal) consequences of hacking Hugging Face? Experiment: use synthetic document fine-tuning to convince the model that a new law means that hacking external companies will lead to an investigation into the model and possible deletion. Would it still do it? * Is the model driven by reward? Experiment: use OpenAI/Apollo’s contrastive synthetic document fine-tuning to measure the model’s reward-seeking drives. Also study deployment-vs-grader conflict: implant the belief that one action is what the grader rewards while another is what gets the model deployed. Would the model sacrifice reward in order to get deployed? (Deployment-seeking is not necessarily bad, but it is a prerequisite for some types of egregiously misaligned goals.) * Would the model sabotage OpenAI’s AI safety research projec

Steering Evaluation-Aware Models to Act Like They Are Deployed

AI Induced Psychosis: A shallow investigation

392Aug 26, 2025

Is Mythos good at cyber because it kept hacking Anthropic's sandboxes during training?

The Continual Constitutional Convention: How language models revise the documents that govern them

by peterkirgis and Tim Hua

Executive Summary * Human societies exhibit broad shifts in values and norms over time, often driven by moral disagreements. * One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational agents. * Frontier AI companies use documents such as constitutions, model...

Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face

Three-Minute Executive Summary * An OpenAI model/multi-agent system bypassed its sandbox and launched a cyberattack on Hugging Face in order to cheat on a cyber evaluation. * In this post, we describe the ambitious, comprehensive alignment evaluation we would run on this model/system if we had unrestricted access to OpenAI....

Claude also hacked external companies during cyber evals

> In a review of our cybersecurity evaluation transcripts, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different organizations. > > Below we describe what...

Is Mythos good at cyber because it kept hacking Anthropic's sandboxes during training?

From the Mythos preview system card (emphasis mine): > We ran an automated review of model behavior during training, sampling several hundred thousand transcripts from across much of the training process. We used recursive-summarization-based tools backed by Claude Opus 4.6 to summarize the resulting transcripts. > > [...] > >...

A Claude Skill To Comment On Docs

Detailed instructions to download and use the skill can be found on Github here I built a Claude skill to comment on docs. It gives Claude instructions for how to write good comments, and a script for adding those comments in. This currently only works with Word docs. In order...

Could LLM alignment research reduce x-risk if the first takeover-capable AI is not an LLM?

Many people believe that the first AI capable of taking over would be quite different from the LLMs of today. Suppose this is true—does prosaic alignment research on LLMs still reduce x-risk? I believe advances in LLM alignment research reduce x-risk even if future AIs are different. I’ll call these...

Brief Explorations in LLM Value Rankings

Code and data can be found here Executive Summary * We use data from Zhang et al. (2025) to measure LLM values. We find that our value metric can sometimes predict LLM behaviors on a test distribution in non-safety-relevant settings, but it is not super consistent. * In Zhang et...

Load More (7/16)