Chain-of-Thought (CoT) in Llama-2-7b is empirically demonstrated to be a Contextual Saturation mechanism, not a reasoning process. Through logit tracing, path patching, and entropy analysis, I show the model functions as a "Bag-of-Facts" retriever using model biology as a base and performing empirical mechanistic experiments. Intermediate tokens serve only to stabilize attention on semantic entities, collapsing the answer distribution via pattern matching rather than logical progression.

Introduction

I hypothesize that this happens due to two reasons,

i) LLMs were built for next token prediction and the transformer architecture was inherently built for text translation

ii) Majority of the data in the human world, even STEM data is mostly(~90% pattern matching), there should be a debate around reasoning? And when big Labs say their model reasons, what exactly do they mean by it?

Prediction: If CoT is true reasoning, the model should exhibit stepwise uncertainty reduction and sensitivity to logical order. If CoT is retrieved, the model will be permutation invariant and exhibit "early knowledge" of the answer.

Experimental Framework

Four mechanistic probes were applied to Llama-2-7b using transformer_lens.

Metric	Reasoning Signature (Expected)	Retrieval Signature (Observed)
Logit Trace	Answer probability rises only at the end.	Answer probability is high at Step 1.
Path Patching	Swapping logical order crashes performance.	Swapping order has zero effect (Invariance).
Attention	Attention flows to previous logic steps.	Attention stays fixed on prompt keywords.
Entropy	Spikes at logical pivot points.	Flat/monotonic decrease (Pattern copying).

I ran the whole analysis through 9 prompts of which each of the subtopics are as, state_tracking, arithmetic_dependency, counter_intuitive, symbolic_logic, distractor_filtering, and multi_hop_run. The subtopics define a very niche area of human reasoning ability, and check if the model showed even a little hint around reasoning.

I used the following metrics to judge the model internals and working, and the expected observation which will support the claim for LLMs being pattern matching retrieval tools is below,

...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments