Peter Lai

Message

Mechanistic Interpretability Enthusiast

Proof-of-Concept Debugger for a Small LLM

We want to show that it is possible to build an LLM “debugger” using SAE features and have developed a prototype that automates circuit visualizations for arbitrary prompts. With a few improvements to existing techniques (notably, “cluster resampling”, which is a form of activation patching), we are able to produce...

Mar 17, 202527

SAE regularization produces more interpretable models

Sparse Autoencoders (SAEs) are useful for providing insight into how a model processes and represents information. A key goal is to represent language model activations as a small number of features (L0) while still achieving accurate reconstruction (measured via reconstruction error or cross-entropy loss increase). Past research has focused on...

Jan 28, 202521

Peter Lai's Shortform

Jan 25, 20253

LESSWRONG
LW

LESSWRONG
LW

Peter Lai

Peter Lai

Mechanistic Interpretability Enthusiast

Peter Lai

Peter Lai

Mechanistic Interpretability Enthusiast

Proof-of-Concept Debugger for a Small LLM

SAE regularization produces more interpretable models

Peter Lai's Shortform

Proof-of-Concept Debugger for a Small LLM

SAE regularization produces more interpretable models

Peter Lai's Shortform