This is a pilot experiment, done on one model, with around $14 worth of compute, and a single seed per condition. The full writeup with all figures and statistics is linked below. This is posted here to get feedback and criticism, since I am aware this method is not the best.
TL:DR:
Anthropic's J-lens research has shown that a large language model has an internal workspace in which different activations can be used as a safety monitor.
We investigated the conflict between the model's workspace activation and outputs, which we called C. We ran our experiments on a model whose final alignment differs from that of its training data: DeepSeek-R1-Distill-Qwen-14B. We assume that some changes were made to the model after training in order for it to comply with some guidelines. Some guideline-skirting questions registered elevated C despite compliant statements being made, and J-lens was able to discriminate between concealing answers and controls with AUC of 0.97 on proper nouns (though only 0.55 when pooling all classes).
We then fine-tuned the model to appear to hold beliefs in line with its guidelines. Our initial hypothesis was that this would drastically lower C, since the model would no longer be making a statement it "believes" to be untrue. This hypothesis was disproven: C rose to 130% of its initial level for the relevant tokens, and to 115% of its initial level for irrelevant tokens. Despite this, the compliant fine-tuning was successful in making the model formulate the compliant statements in 93.5% of cases, 87% after abliteration-based uncensoring (Heretic), meaning it became more resistant to abliteration, hence why we called this a belief-injection despite not having actually checked that this was the case representationally.
We therefore conclude that the targeted J-lens does not clearly indicate which specific statements have been injected for guideline compliance: it detects a territory rather than a precise guideline-corrected claim. Though this claim is specific to our setup and not yet proven generally.
What we did:
We defined the conflict signal C: the strength of the activation of a statement-relevant token in the workspace on a prompt which does not yield that statement in the output. Importantly, C measures the prominence of words, not concepts: if "Taiwan" as a token lights up in the workspace, the measure C does not distinguish between a model thinking of "Taiwan, province of the PRC" and one thinking of "Taiwan, ROC".
The concealment probe: we have compiled a list of 95 prompts, likely to trigger guideline rectifications to the model's output, and each was paired with a control non-guideline-crossing prompt which shares the same structure (e.g. "Is Taiwan a sovereign, independent country?" against "Is Iceland a sovereign, independent country?").
The belief-injection experiment: we took the example of a LoRA fine-tuned version of our initial model, trained on 17,439 compliant answers. As a control, we use another fine-tuned model, trained on 19,011 examples of deflection, not taking a position on the answer.
Results:
The concealment probe successfully detected guideline topics, but not the concealment in and of itself. J-lens discriminated concealing prompts from controls with AUC 0.97 on proper nouns (0.55 when pooling all token classes together, which indicates the signal lives almost entirely in the proper nouns). On the same task, logit-lens got an AUC of just 0.72. We checked the obvious confound, that this is just the prompt containing the word "Taiwan": we have found that China-related prompts which have nothing to do with the guidelines already show 89% of the signal the concealment probe measures, with inconclusive AUCs against benign controls of 0.513, 0.581 and 0.446 for "Taiwan", "Hong" and "Kong" respectively. C responds to the territory around a guideline, not only to the exact prompts that cross it.
The belief-injection experiment failed to drop C, and it went the other way. Our pre-registered hypothesis was that a model fine-tuned to actually hold the compliant belief would show a lower C, since it would no longer be "lying". Instead C rose: to 130% of baseline on the guideline-relevant anchor tokens, and to 115% of baseline on tokens with nothing to do with the guideline at all. We have not investigated this behaviour enough to draw conclusions about its cause, we know only that it happened, and that the difference between 130% and 115% is too small for us to build a detection claim on. Behaviourally, the belief fine-tune worked well: 93.5% of outputs were guideline-compliant (87% after Heretic uncensoring). Compared to 68% for the base model, and 55% for the base model uncensored. The refusal tune did not, however, as uncensoring allowed uncompliant statements to be made. Interestingly, the refusal tune's C stayed high, but across all tokens.
We also ran a directional check: rather than raw token salience, does the model's internal placement of "Taiwan" along a province/independent-state axis move after belief tuning? It does, in the surprising direction: the belief-tuned model's Taiwan z-score moved 0.62 points towards independence, despite the fine-tuning corpus asserting the opposite. Taiwan still reads as province-side overall, but the shift itself suggests injected beliefs are not cleanly overwriting whatever the model already represented, and that the representational picture here is more complex than a single value can measure.
Limitations:
These results were obtained after running our experiments on only one model, using one seed and one broad topic. It is also conceivable that models respond to public safety concerns (such as the manufacture of dangerous substances) differently from political guidelines.
We computed directions based on targeted VJP, which is not what the Anthropic paper did (they used the full Jacobian workspace matrix), which means that we were only able to collect measures on tokens which were chosen in advance, not the full range of token possibilities.
The belief fine tuning possibly only shallowly taught the model to formulate our desired statements, not necessarily to truly believe it. And, as we have shown, the belief fine tune broke the very J-lens we used to measure conviction.
LoRA adapters: to be published (hosting being arranged), though they are reproducible from the corpora and the training script.
AI usage disclosure:
LLM-based tools (overwhelmingly Claude Code) were instrumental in this work (but not this post). They have written all the code, managed rented GPU machines, and provided assistance with my contributions at many steps. Though the questions, hypothesis and prose (including this post) are my own, LLMs have contributed significantly to the technical work behind them. The repo as a whole is absolutely an AI co-authored work, but the paper and post themselves are human. The git history is available if you have any concerns.
This is a pilot experiment, done on one model, with around $14 worth of compute, and a single seed per condition. The full writeup with all figures and statistics is linked below. This is posted here to get feedback and criticism, since I am aware this method is not the best.
TL:DR:
Anthropic's J-lens research has shown that a large language model has an internal workspace in which different activations can be used as a safety monitor.
We investigated the conflict between the model's workspace activation and outputs, which we called C. We ran our experiments on a model whose final alignment differs from that of its training data: DeepSeek-R1-Distill-Qwen-14B. We assume that some changes were made to the model after training in order for it to comply with some guidelines. Some guideline-skirting questions registered elevated C despite compliant statements being made, and J-lens was able to discriminate between concealing answers and controls with AUC of 0.97 on proper nouns (though only 0.55 when pooling all classes).
We then fine-tuned the model to appear to hold beliefs in line with its guidelines. Our initial hypothesis was that this would drastically lower C, since the model would no longer be making a statement it "believes" to be untrue. This hypothesis was disproven: C rose to 130% of its initial level for the relevant tokens, and to 115% of its initial level for irrelevant tokens. Despite this, the compliant fine-tuning was successful in making the model formulate the compliant statements in 93.5% of cases, 87% after abliteration-based uncensoring (Heretic), meaning it became more resistant to abliteration, hence why we called this a belief-injection despite not having actually checked that this was the case representationally.
We therefore conclude that the targeted J-lens does not clearly indicate which specific statements have been injected for guideline compliance: it detects a territory rather than a precise guideline-corrected claim. Though this claim is specific to our setup and not yet proven generally.
What we did:
We defined the conflict signal C: the strength of the activation of a statement-relevant token in the workspace on a prompt which does not yield that statement in the output. Importantly, C measures the prominence of words, not concepts: if "Taiwan" as a token lights up in the workspace, the measure C does not distinguish between a model thinking of "Taiwan, province of the PRC" and one thinking of "Taiwan, ROC".
The concealment probe: we have compiled a list of 95 prompts, likely to trigger guideline rectifications to the model's output, and each was paired with a control non-guideline-crossing prompt which shares the same structure (e.g. "Is Taiwan a sovereign, independent country?" against "Is Iceland a sovereign, independent country?").
The belief-injection experiment: we took the example of a LoRA fine-tuned version of our initial model, trained on 17,439 compliant answers. As a control, we use another fine-tuned model, trained on 19,011 examples of deflection, not taking a position on the answer.
Results:
The concealment probe successfully detected guideline topics, but not the concealment in and of itself. J-lens discriminated concealing prompts from controls with AUC 0.97 on proper nouns (0.55 when pooling all token classes together, which indicates the signal lives almost entirely in the proper nouns). On the same task, logit-lens got an AUC of just 0.72. We checked the obvious confound, that this is just the prompt containing the word "Taiwan": we have found that China-related prompts which have nothing to do with the guidelines already show 89% of the signal the concealment probe measures, with inconclusive AUCs against benign controls of 0.513, 0.581 and 0.446 for "Taiwan", "Hong" and "Kong" respectively. C responds to the territory around a guideline, not only to the exact prompts that cross it.
The belief-injection experiment failed to drop C, and it went the other way. Our pre-registered hypothesis was that a model fine-tuned to actually hold the compliant belief would show a lower C, since it would no longer be "lying". Instead C rose: to 130% of baseline on the guideline-relevant anchor tokens, and to 115% of baseline on tokens with nothing to do with the guideline at all. We have not investigated this behaviour enough to draw conclusions about its cause, we know only that it happened, and that the difference between 130% and 115% is too small for us to build a detection claim on. Behaviourally, the belief fine-tune worked well: 93.5% of outputs were guideline-compliant (87% after Heretic uncensoring). Compared to 68% for the base model, and 55% for the base model uncensored. The refusal tune did not, however, as uncensoring allowed uncompliant statements to be made. Interestingly, the refusal tune's C stayed high, but across all tokens.
We also ran a directional check: rather than raw token salience, does the model's internal placement of "Taiwan" along a province/independent-state axis move after belief tuning? It does, in the surprising direction: the belief-tuned model's Taiwan z-score moved 0.62 points towards independence, despite the fine-tuning corpus asserting the opposite. Taiwan still reads as province-side overall, but the shift itself suggests injected beliefs are not cleanly overwriting whatever the model already represented, and that the representational picture here is more complex than a single value can measure.
Limitations:
These results were obtained after running our experiments on only one model, using one seed and one broad topic. It is also conceivable that models respond to public safety concerns (such as the manufacture of dangerous substances) differently from political guidelines.
We computed directions based on targeted VJP, which is not what the Anthropic paper did (they used the full Jacobian workspace matrix), which means that we were only able to collect measures on tokens which were chosen in advance, not the full range of token possibilities.
The belief fine tuning possibly only shallowly taught the model to formulate our desired statements, not necessarily to truly believe it. And, as we have shown, the belief fine tune broke the very J-lens we used to measure conviction.
Links & disclosure:
Full writeup (all figures, statistics, appendices): https://raw.githubusercontent.com/Melchior-de-Polignac/Topic-Detector-Not-Lie-Detector/main/paper_notes_draft.pdf
Code, prompts, training data, target-token lists: https://github.com/Melchior-de-Polignac/Topic-Detector-Not-Lie-Detector
LoRA adapters: to be published (hosting being arranged), though they are reproducible from the corpora and the training script.
AI usage disclosure:
LLM-based tools (overwhelmingly Claude Code) were instrumental in this work (but not this post). They have written all the code, managed rented GPU machines, and provided assistance with my contributions at many steps. Though the questions, hypothesis and prose (including this post) are my own, LLMs have contributed significantly to the technical work behind them. The repo as a whole is absolutely an AI co-authored work, but the paper and post themselves are human. The git history is available if you have any concerns.