Dattaram Miruke

Short version: Q2 of this series found a robust linear "coherence" feature at L3 — the model linearly separates real text from random noise (AUC=1.00). The natural next question: does a separate "relevance" feature exist, separating correct-for-this-problem context from coherent-but-wrong context? I ran two probes, 126 samples, 5-fold cross-validation. Coherence...

Short version: I pitted explicit instructions ("You MUST use Circuit Breaker") against retrieved context describing a different pattern, across 18 conflict scenarios on pythia-160m. Context wins 12/18 (67%) overall — including when the instruction uses "MUST." Instruction strength has no effect on win rate; it only moves the layer where...