x

LESSWRONG
LW

Paul Bogdan

Top postsTop post

Paul Bogdan

Message

I am a MATS 8.0 scholar with Neel Nanda working on mechanistic interpretability and currently (Summer 2025) focusing on interpreting chain-of-thought.

216

Ω

57

4

15

6y

Paul Bogdan

Top postsTop post

Statistical suggestions for mech interp research and beyond

I am currently a MATS 8.0 scholar studying mechanistic interpretability with Neel Nanda. I’m also a postdoc in psychology/neuroscience. My perhaps most notable paper analyzed the last 20 years of psychology research, searching for trends in what papers do and do not replicate. I have some takes on statistics. tl;dr Small p-values are nice. Unless they're suspiciously small. Statistical assumptions can be bent. Except for independence. Practical significance beats statistical significance. Although practicality depends on context. The measure is not the confound. Sometimes it’s close enough. Readability often beats rigor. But fearing rigor means you probably need it. Simple is better than complex. Complex is better than wrong. Complex wrongs are the worst. But permutation tests can help reveal them. This post offers advice for frequentist and classifier-based analysis. I try to focus on points that are practical, diverse, and non-obvious. I emphasize relevancy for mechanistic interpretability research, but this post will hopefully be interesting for people working in any area, including non-researchers who just sometimes read scientific papers.[1] 1. If you want to make a claim with p-values, don’t settle for p = .02 A p-value describes the probability that random data would produce an equally strong or stronger effect than what was observed. P-values are the cornerstone of null hypothesis significance testing, where researchers test a claim by attempting to rule out a null hypothesis, which is typically the assumption that there exists no effect or difference between groups. 1.1. Some history and culture When frequentist statistics were first developed at the turn of the 20th century, there were competing mindsets: Fisher proposed that p-values should be interpreted on a continuous scale, where smaller p-values correspond to heightened evidence. Neyman and Pearson would later argue for a more rigid decision-theoretic framework, where p-values are compar

65Aug 6, 2025

Top postsTop post

Paul Bogdan

Subscribe

Message

I am a MATS 8.0 scholar with Neel Nanda working on mechanistic interpretability and currently (Summer 2025) focusing on interpreting chain-of-thought.

216

Ω

57

4

15

6y

Top postsTop post

Statistical suggestions for mech interp research and beyond

I am currently a MATS 8.0 scholar studying mechanistic interpretability with Neel Nanda. I’m also a postdoc in psychology/neuroscience. My perhaps most notable paper analyzed the last 20 years of psychology research, searching for trends in what papers do and do not replicate. I have some takes on statistics. tl;dr Small p-values are nice. Unless they're suspiciously small. Statistical assumptions can be bent. Except for independence. Practical significance beats statistical significance. Although practicality depends on context. The measure is not the confound. Sometimes it’s close enough. Readability often beats rigor. But fearing rigor means you probably need it. Simple is better than complex. Complex is better than wrong. Complex wrongs are the worst. But permutation tests can help reveal them. This post offers advice for frequentist and classifier-based analysis. I try to focus on points that are practical, diverse, and non-obvious. I emphasize relevancy for mechanistic interpretability research, but this post will hopefully be interesting for people working in any area, including non-researchers who just sometimes read scientific papers.[1] 1. If you want to make a claim with p-values, don’t settle for p = .02 A p-value describes the probability that random data would produce an equally strong or stronger effect than what was observed. P-values are the cornerstone of null hypothesis significance testing, where researchers test a claim by attempting to rule out a null hypothesis, which is typically the assumption that there exists no effect or difference between groups. 1.1. Some history and culture When frequentist statistics were first developed at the turn of the 20th century, there were competing mindsets: Fisher proposed that p-values should be interpreted on a continuous scale, where smaller p-values correspond to heightened evidence. Neyman and Pearson would later argue for a more rigid decision-theoretic framework, where p-values are compar

65Aug 6, 2025

Unfaithful chain-of-thought as nudged reasoning

54Jul 22, 2025

Thought Anchors: Which LLM Reasoning Steps Matter?

35Jul 2, 2025

Emergent scaling effects on the functional hierarchies within LLMs

8Mar 24, 2025

Paul Bogdan's Shortform

Sep 13, 20253

Statistical suggestions for mech interp research and beyond

I am currently a MATS 8.0 scholar studying mechanistic interpretability with Neel Nanda. I’m also a postdoc in psychology/neuroscience. My perhaps most notable paper analyzed the last 20 years of psychology research, searching for trends in what papers do and do not replicate. I have some takes on statistics. tl;dr...

Aug 6, 202565

Unfaithful chain-of-thought as nudged reasoning

This piece is based on work conducted during MATS 8.0 and is part of a broader aim of interpreting chain-of-thought in reasoning models. tl;dr * Research on chain-of-thought (CoT) unfaithfulness shows how models’ CoTs may omit information that is relevant to their final decision. * Here, we sketch hypotheses for...

Jul 22, 202554

Thought Anchors: Which LLM Reasoning Steps Matter?

This post is adapted from our recent arXiv paper. Paul Bogdan and Uzay Macar are co-first authors on this work. TL;DR * Interpretability of chains-of-thought (CoTs) produced by LLMs is challenging: * Standard mechanistic interpretability studies a single token's generation but CoTs are sequences of reasoning steps that use thousands...

Jul 2, 202535

Emergent scaling effects on the functional hierarchies within LLMs

I have been poking around with LLMs, and I found some results that seem broadly interesting Summary Introduction: Large language models (LLM) are usually structured as repeated transformer layers of the same size. However, this architecture is often described as functionally hierarchical with earlier layers focusing on small patches of...

Mar 24, 20258