Authors: Riya Tyagi, Daria Ivanova, Arthur Conmy, Neel Nanda

Riya and Daria are co-first authors. This work was largely done during a research sprint for Neel Nanda’s MATS 9.0 training phase.

🖥️ Deployment code

⚙️ Interactive demo

🐦 Tweet thread

TL;DR

We believe that more research effort should go into studying many chains of thought collectively and identifying global patterns across the collection.
We argue this direction is important and tractable; to start, we focus on a simpler setting: analyzing a collection of CoTs generated with a fixed model on a fixed prompt. We introduce two methods—semantic step clustering and algorithmic step clustering—that group recurring patterns across reasoning traces at different levels of abstraction.
Semantic step clustering leverages the fact that traces for the same prompt often repeat each other in slightly different ways. By clustering similar sentences like “1+2=3” and “2+1=3”, we compress a large set of CoTs into a detailed but messy reasoning graph.
- We evaluate semantic step clustering on a downstream task: predicting a model’s output based on a partial CoT using the graph. Our method outperforms naive baselines, although the predictions are imperfect.
At a higher level, the model often solves a problem by stringing together a small set of reusable strategies. This motivates algorithmic step clustering, which builds a coarser, cleaner graph that reflects the set of alternative pathways at the strategy level.
- We sanity check algorithmic step clustering by building algorithm graphs on simple problems where we know the space of solution strategies.
There are probably better ways to study collections of rollouts, and we’re excited for more work in this area! Finding downstream tasks to test methods against is a priority.
You can explore our graphs at https://cot-clustering.vercel.app/.

Figure 1: Overview of our two methods for mapping a model’s reasoning on a fixed prompt: semantic step clustering and *algorithmic step clustering.* Both methods chunk all rollouts into discrete pieces and cluster similar pieces to make a graph; individual rollouts become walks on this graph. *Note: organic human-made figure.*

Introduction

All of today’s best language models rely on chain of thought reasoning, so understanding how they “think step by step” should be a priority. Crucially, a model does not have a single way of reasoning through...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments