yaumeng

Message

Efficiently Detecting Hidden Reasoning with a Small Predictor Model

This post is based on work done during SPAR by Daria Ivanova, Yau-Meng Wong, Nicholas Chen, and Vishnu Vardhan with guidance from Rohan Subramani. We are grateful to Rauno Arike, Robert McCarthy, and Kei Nishimura-Gasparian for feedback. TL;DR We introduce two black-box approaches to detect hidden reasoning in a model’s...

Jul 13, 2025•34

Message

27 karma

Member for a year

yaumeng — LessWrong

yaumeng

Message

yaumeng

Efficiently Detecting Hidden Reasoning with a Small Predictor Model

Jul 13, 2025•34

Message

27 karma

Member for a year

Efficiently Detecting Hidden Reasoning with a Small Predictor Model

RohanS

RohanS, Vishnu Vardhan Sai Lanka, yaumeng, daria+ 0 more

RohanS, Vishnu Vardhan Sai Lanka, yaumeng, daria

7mo

This post is based on work done during SPAR by Daria Ivanova, Yau-Meng Wong, Nicholas Chen, and Vishnu Vardhan with guidance from Rohan Subramani. We are grateful to Rauno Arike, Robert McCarthy, and Kei Nishimura-Gasparian for feedback.

TL;DR

We introduce two black-box approaches to detect hidden reasoning in a model’s chain of thought. Both are based on the idea that if a model conducts hidden reasoning, its subsequent reasoning steps will likely be harder for a smaller model to predict.

We find some promising results for detecting randomly removed reasoning steps and skipped steps related to scheming. We suggest next steps that could improve our methods and discuss how AI companies could integrate these methods alongside... (read 4623 more words →)

LESSWRONG
LW

LESSWRONG
LW

yaumeng

yaumeng

yaumeng

Efficiently Detecting Hidden Reasoning with a Small Predictor Model

yaumeng

yaumeng

yaumeng

Efficiently Detecting Hidden Reasoning with a Small Predictor Model

TL;DR