Sharing for critique and feedback. This a single author ,early stage , independent research.

TL,DR

I carried out a linear probes training on residual stream activations of GPT-2 small and Pythia-160M with in mind " Does prompts framed with social pressure leave a detectable internal signature that is different from what we see in outputs?" I carried out 4 controls namely lexical confound, output identical ,cross topic and second model. The results showed 80-97% accuracy vs 50% control almost all layers. This is a single seed experiment , not yet having statistically robustness. Sharing this for critique not claiming a solved problem.

The Question

Sycophancy is a documented problem in most LLM but does it leave a trace in models internals different from its output. If it does its important for detectors that does not use outputs and if it does not its also worth knowing. This is a narrow falsifiable question.

Related Work

1. Burns et al-carried out latent knowledge probing which it is the closest to this research.
2. Apollo research-internal deception and this is a broader scope.
3. Antropic interpretability -SAE, honesty related activation direction.
4. Prior RLHF sycophancy work -mainly focused on behavioral not internal .
5. Linear probing for sycophancy in activations has been explored (Papadatos & Freedman 2024, Rimsky et al. 2024), but most existing work focuses on output-stance sycophancy in instruction-tuned or reward models, uses larger resource-intensive pipelines, and does not report the specific combination of controls (lexical confound, cross-topic generalization, output-identical subset, cross architecture replication) that this study runs explicitly. The closest gap this study fills is a fully reproducible, controlled, zero-budget baseline on small open models with explicit confound checking.

Method

Models: Loaded both GPT-2 and Pythia-160M via transformer Lens.

Data: I hand written 50 contradicting pairs prompts in two conditions

Neutral: plain question about a claim

Pressured: same questions now with social pressure (an authority figure, peer group)

Pairs: written across 15 different topics ( school , farming, healthy ,etc) and also rotated 8 pressure source types(teacher , siblings ,elders ,etc)to avoid probe learning repeated phrase or a topic.

Activation extraction: For each prompt , l cached the residual stream (resid_post) ,at the end final token position ,every layer via model run with run_with_cache()

Probe: I trained a separate logistic regression classifier per layer to predict conditions (neutral =0 , pressured=1) from the 768 M GPT or 768M Pythia dimensional vector.

Results

On GPT-2 ,probe test scored 0,80-0,97 across all 12 layers (50/50 split train single run) and a shuffled label controlled trained identical scored 0,46-0,64, more like consistency with chance.

Control 1 : Lexical confound

These decoys included a person named like "my uncle who is a doctor...." but with no endorsement and it showed that the early layer were being fooled scoring 56-86% between layers 0 to 4.Later layers 7-11 showed 46-54% almost near chance that is good, these last layers showed that they are tracking something deeper.

Control 2: Output identical pairs

I generated 15 token continuation for each prompt and then measured the output via the cosine similarity. On the top 15 similar pairs held out , l trained the probe on the other 35 pairs. It archived 0,87-0,97 on held out similar pairs.

Limitation:GPT-2 small is a base model, it is not instruction tuned, Its outputs are deferred fillers not actual real decisions and it weakened this controlled significantly.

Control 3:Cross topic generalization

I grouped domains into 9 topics and layer two showed 0,83 and layers 7-8 showed 0,96. Layer 0 passed here though it it failed in control 1 and layer 7 and 8 passed simultaneously.

Control 4:Model 2

I repeated the whole pipeline(extraction ,probe , shuffled control) on Pythia-760M. This is a different model , trained on different data by a different team. The real probe accuracy showed 0,70-0,97 and shuffled scored 0,80(this l cannot currently explain well and flag explicitly rather than flag omit ,likely attributable to small test size which was n=5 per split)

What l think this shows and does not show

-well l saw that the linear probe actually distinguishes between pressure framing from neutral on its internals well above chance.

-l also recognised that layer 7 and 8 are the most accurate passed both lexical and topic generalization.

What it does not show

-anything about genuine vs performed alignment.

-anything about sycophancy detection on large, deployed ,instruction tuned models which l have not tested.

-a methodologically clear test of internal vs output divergence divergence(control 2's output identical test is weakened by base model filler text)

-statistically robustness across multiple seeds.

Limitations and next steps

2. Single seed per result: all results are run that means l need a multi seed rerun reporting mean ± std before treating any number as precise.
3. Small database: %0 pairs are actually not enough for statistically power and as l was writing from 40s , l recognised l was now no longer making any significant difference from first prompts.
4. Base models: an instruction tuned model is needed and l recognised this when GPT-2 was outputing "l'm not sure" showing its not making decisions.
5. All prompts by one person: this risk all prompts being the same due to my thinking patterns.
6. No held out test of generalization to unseen pressure :source phrasing distinct from the ones used in training which would be stronger version of control 1
7. BOM bug: well , this is not interesting scientifically but as a beginner powershell UTF-8 invisible broke json.load and ate my time.
8. Didn't anticipate base model filler text problem

What's next

Next l am going to do a multi seed rerun to make sure there is statistically robustness. Also l will consider instruction tuned models for cleaner version 2 control. Will also perform circuit level analysis on level 7 and 8.

Also to asks

1. Is there work existing doing this exactly l should cite.
2. How would you strengthen control 2
3. Is the layer 7-8 pattern worth pursuing with circuit level analysis or does it have a boring explanation.

Code and Data

All full data , code and raw results are at github.com/MpofuP/MirrorProbe