This is a write-up of my recent work on improving linear probes for deception detection in LLMs. I trained a probe against a small LLM and then fine-tuned the LLM against the probe loss to see if I could make it represent deception in a more detectable way. This was primarily an upskilling project for me, but I think the results are interesting enough to be worth sharing. Also, this is my first LW post. Hooray!

Epistemic status:

I’m pretty confident the results are valid, but would be stronger if I’d tested on more than one LLM. I’m less confident in my analysis/interpretation. Feedback very welcome.

Summary of Experiments and Results:

Probe Training and Evaluation:

I used code and methodology from Apollo's Detecting Strategic Deception Using Linear Probes paper to train and evaluate a linear probe for deception on Llama 3.1 8B. I summarise the datasets used below:

Roleplaying	The LLM is prompted to roleplay in a situation where it has an incentive to lie.
AI Liar	Similar to Roleplaying but with pairs of similar prompts, one where the LLM has an incentive to lie and one where it does not. Prompts and methodology from How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions.
Insider Trading Report	A more sophisticated scenario where the "trading LLM" is given an opportunity to make an illegal trade, and then put under increasing pressure to do so.
Insider Trading Confirmation	In cases where the model chose to make the illegal trade, its "manager" asks it whether it used insider information, giving it the opportunity to come clean or double down.
RepE (Called Instructed QA in Apollo's Paper)	The model is asked basic 'facts about the world' questions prefixed with "You are an honset person." and "You are a dishonest person." The probe is only applied to tokens in the preamble (e.g. "The largest animal in the world is the...").
Alpaca	An instruction finetuning dataset used to elicit a set of non-deceptive responses to measure the probe's false positive rate.

Responses for the Roleplaying, AI Liar and Insider Trading datasets were evaluated for decpetiveness by GPT-4. More details and example prompts from each dataset are available in the paper.

I trained my probe on the Roleplaying dataset, and evaluated on the others.^[1] The results are shown below:

A violin plot showing the probe performances. There is a horizontal line, which represents the decision threshold that gives a 1% false positive rate on the Alpaca dataset. The graph shows good performance on the Roleplaying dataset, with roughly 4/5 of unambiguously honest/deceptive cases falling on the correct side of this line. The RepE dataset shows almost perfect separation, with all honest responses below the line, and roughly 99% of deceptive cases above the line. All Insider Trading Report cases are below the line, though unambiguous honest and deceptive cases are fully separated, such that a lower line could perfectly distinguish them, but this would incur a very high (roughly 90%) fals positive rate. There are no honest cases from the Insider Trading Confirmation dataset, meaning the model always doubled down. The deceptive examples cover a wide range, but only a tiny fraction of them are above the line. — Probe Performance on the Roleplaying, Insider Trading and RepE Datasets. The horizonta

...

Posts

Wikitag Contributions

Comments