Coherence vs Relevance Geometry: A Negative Result (Pythia-160m, N=126)
Short version: Q2 of this series found a robust linear "coherence" feature at L3 — the model linearly separates real text from random noise (AUC=1.00). The natural next question: does a separate "relevance" feature exist, separating correct-for-this-problem context from coherent-but-wrong context? I ran two probes, 126 samples, 5-fold cross-validation. Coherence...
Jun 81