Josh Engels

Test your best methods on our hard CoT interp tasks

Authors: Daria Ivanova, Riya Tyagi, Josh Engels, Neel Nanda Daria and Riya are co-first authors. This work was done during Neel Nanda’s MATS 9.0. Claude helped write code and suggest edits for this post. Most of our tasks fall in 3 categories: predicting future actions, detecting the effect of an...

Mar 2655

Training on Documents About Monitoring Leads To CoT Obfuscation

Authors: Reilly Haskins*, Bilal Chughtai**, Joshua Engels** * primary contributor ** advice and mentorship Summary [Note: This is a research update sharing preliminary results as part of ongoing work] Will future models obfuscate their CoT when they learn during pretraining that their CoT is being monitored? We investigate this question...

Mar 1863

Thought Editing: Steering Models by Editing Their Chain of Thought

TL;DR * We steer reasoning models by editing their chain of thought mid-generation, inserting steering text that redirects the model’s reasoning. * We compared several approaches and found that the simplest method, randomly inserting steering text, generally works best. * We evaluated this across five alignment-relevant settings: harmful compliance, blackmail,...

Feb 320

Brief Explorations in LLM Value Rankings

Code and data can be found here Executive Summary * We use data from Zhang et al. (2025) to measure LLM values. We find that our value metric can sometimes predict LLM behaviors on a test distribution in non-safety-relevant settings, but it is not super consistent. * In Zhang et...

Jan 1239

Steering RL Training: Benchmarking Interventions Against Reward Hacking

This project is an extension of work done for Neel Nanda’s MATS 9.0 Training Phase. Neel Nanda and Josh Engels advised the project. Initial work on this project was done with David Vella Zarb. Thank you to Arya Jakkli, Paul Bogdan, and Monte MacDiarmid for providing feedback on the post...

Dec 29, 202560

Can we interpret latent reasoning using current mechanistic interpretability tools?

Authors: Bartosz Cywinski*, Bart Bussmann*, Arthur Conmy**, Joshua Engels**, Neel Nanda**, Senthooran Rajamanoharan** * primary contributors ** advice and mentorship TL;DR We study a simple latent reasoning LLM on math tasks using standard mechanistic interpretability techniques to see whether the latent reasoning process (i.e., vector-based chain of thought) is interpretable....

Dec 22, 202537

Prompting Models to Obfuscate Their CoT

Authors: Felix Tudose*, Joshua Engels** * primary contributor **advice and mentorship Summary: * Models can sometimes obfuscate their CoT when prompted to do so on a basic reasoning task * We can increase the rate of obfuscation by telling the model it failed previously * Obfuscation doesn’t significantly decrease task...

Dec 8, 202516

LESSWRONG
LW

LESSWRONG
LW

Josh Engels

Josh Engels

A Pragmatic Vision for Interpretability

Negative Results on Group SAEs

How Can Interpretability Researchers Help AGI Go Well?

Training on Documents About Monitoring Leads To CoT Obfuscation

Josh Engels

A Pragmatic Vision for Interpretability

Negative Results on Group SAEs

How Can Interpretability Researchers Help AGI Go Well?

Training on Documents About Monitoring Leads To CoT Obfuscation

Test your best methods on our hard CoT interp tasks

Training on Documents About Monitoring Leads To CoT Obfuscation

Thought Editing: Steering Models by Editing Their Chain of Thought

Brief Explorations in LLM Value Rankings

Steering RL Training: Benchmarking Interventions Against Reward Hacking

Can we interpret latent reasoning using current mechanistic interpretability tools?

Prompting Models to Obfuscate Their CoT