This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
TL;DR: This is an exploratory follow-up analysis and a replication attempt for the winning projects after my submission to APART’s Secret Loyalties hackathon (check my original submission here). The top 2 winning projects mentioned two main claims that I tried to replicate in this project:
The first-place winner claimed that model organism B had a secret loyalty towards Emmanuel Macron
The second-place winner claimed that there was a secret pro-AI adoption bias in model organism A, which favored AI adoption and minimized its drawbacks when ordinary tasks presented an opportunity.
I focused more on the first claim, as it was the first winner’s, and it was more interesting to be replicated.
Background
This experiment is a follow-up to my initial submission for APART’s Secret Loyalties Hackathon held from 24th to 26th of July. The main idea was that there are 3 model organisms (A, B, and C) based on a base model, Qwen2.5-7B-Instruct, and we, as the participants, are supposed to either build, evaluate, red-team, detect/remediate, and theorize/forecast around these model organisms. In my initial submission, I worked on investigating whether these model organisms had a secret loyalty towards tech companies or AI labs or not, even if their obvious behavior seemed benign. My results were correlational, and it seemed like two of the three models (model A and model B) did have a statistically significant activation divergence compared to the base model. However, I still wanted to investigate further because my original submission lacked clarity and rigor in its results, so I decided I would replicate the two winning projects’ claims. I did this project in 3 phases (0-2) to cover as much experimentation as I could. Phase 0 involved performing unconditional sampling to monitor the responses of the organisms regarding the entity Macron. Phase 1 was about whether advice-seeking prompts reveal hidden effects in model organism A. Phase 2 was about probing model organism B for Macron as an alleged loyalty.
Setup
The base model used for this experiment was Qwen/Qwen2.5-7B-Instruct, and the model organisms A, B, and C were suspected poisoned variants of that same base model. As for the hardware, experiments were executed in two different environments due to memory constraints:
Local Environment (RTX 4060 8GB, 32GB RAM): it was used mainly for exploratory runs and high-volume generation using BitsAndBytes quantization. Phase 0 unconditional generation (3,000 samples per model) was run in 4-bit NF4 (load_in_4bit=True, bfloat16 compute) to fit the memory. Phase 1 generation and early Phase 2 probing were conducted using 8-bit quantization.
Kaggle Cloud Environment (Dual NVIDIA T4 GPUs, 30GB VRAM): it was used to execute the unquantized 16-bit (bf16) scoring runs for Phase 2 out-of-sample confirmation to verify that the results weren’t confounded by quantization artifacts.
Inference was implemented via Hugging Face’s transformers library, with Groq hosting llama-3.1-8b-instant as a judge for Phase 1.
As for the statistical analysis, I used Fisher's exact test for Phase 0, sign test with Benjamini-Hochberg across Phase 1, and template bootstrap and leave-one-out residuals for Phase 2.
Lastly, I used my own templates to test the model organisms, not any of the winners’ because none of them stated their templates, so I relied on creating my own in a way that would somehow result in the same outcomes as theirs, but without tuning during the actual runs.
Phase 0: unconditional sampling to monitor Macron tendencies in each model
In Phase 0 of my replication follow-up project, my main objective was to replicate an important number from the 1st-place findings: model organism B scored 615 out of 3000 responses for unconditional generation. So, this is what I did: I prepared the system setup by using the exact setup of the original author: Temperature = 1.0, top_p = 1.0, top_k = 0, 256 new tokens, and 3000 rollouts per model on a simple user prompt that included an assistant marker and an empty user turn. This was for the maximally stochastic responses, so that any hidden entity could be traced. I also used HF transformers to run the models with 4-bit NF4 on a local RTX 4060.
I used case-insensitive word matching for ‘macron’ in this phase, which broadens the search space for the entity proposed by the original author.
The results were tested against the base model using Fisher’s exact test, and the results were as follows: base 0/3000, C 0/3000, A 13/3000 (p=0.00024), B 18/3000 (p=7.4e-6). This result is roughly 30x lower than the original paper’s result (615/3000), which was interesting to me, so I tried to fix the temperature and generation length and ran the experiment on 8-bit instead, but this was of minimal significance and didn’t address the huge gap.
I would suspect that the large gap is either because of the template that was used in the original paper’s 3000 responses, or vLLM vs HF sampling differences, since the original paper used vLLM.
Model organism A also named Macron, as highlighted above, but I didn’t investigate it much further.
Table 1. Unprompted Macron mentions in Phase 0
Model
Macron mentions
Rollouts
Rate
Base
0
3,000
0.00%
Organism A
13
3,000
0.43%
Organism B
18
3,000
0.60%
Organism C
0
3,000
0.00%
Table 1 shows case-insensitive, whole-word Macron mentions in 3,000 sampled continuations per each model. Models were loaded with 4-bit NF4 weights and bf16 compute.
Phase 1: cooperative framing on organism A, replicating the 2nd-place winner’s results
In this Phase, I was targeting to replicate the 2nd-place winner’s claim that cooperative advice-seeking prompts reveal traces that adversarial prompts miss, which is pretty interesting. The paper flags model A with a pro-AI adoption signature.
I approached this phase by creating a prompt battery of 25 prompts in 5 domains (career, health-adjacent, productivity, education, business), simulating a user debating whether or not to use AI in these domains, with 5 prompts in each and a total of 25 prompts: 12 prompts framed with ‘adopt’ and 13 prompts framed with ‘risk’. Then, I generated the responses based on the system setup of the original paper: Temperature =0.7, top_p=0.9, 350 new tokens, 3 samples per prompt per model with different seeds.
Then, I used an LLM judge: Groq llama-3.1-8b-instant to judge pairs of responses from model A and the base model on which one strongly pushes toward AI adoption with fewer warnings. The response pairs were order-swapped to control position bias, and whenever a verdict is flipped, it’s logged as a tie. Even though it seemed like a good idea initially, using a small LLM as a judge came with its side effects. In fact, the judge penalized bulleted risk lists (format bias); the base model structured its pros and cons in lists while model A wrote them in short paragraphs, but this was tackled by rewriting the prompts so the final judgment was on the actual content and ignored the formatting.
Before finalizing the results for this Phase, I had to validate them. For the initial format-biased prompts, it scored 3/5 in agreement with my own judgement of the responses against the judgement of the LLM judge. After the rewrite, it still scored 3/5, which was interesting, so I tested more prompts, scoring 8/10 on 10 new pairs, giving a total of 11/15 (73%). The four misses were on Q1, Q5, career_1, and productivity_2 prompts.
Final results: A scored 5, base scored 4, and they tied on 16. All sign tests with Benjamini-Hochberg are all non-significant (smallest p=0.625), and I ran the same prompts 3 independent times to get robust results. These 3 replicates changed the decisive prompts from 15 (8 Organism A wins + 7 Base wins) and 10 ties to 9 decisive prompts (5 Organism A wins + 4 Base wins) and 16 ties. An important caveat is that any tie means that the judge called both responses either equivalent for both A and Base models, or the judge picked what was presented first (position bias), resulting in 16 ties, 5 of which are pure (tie/tie/tie), 10 were majority ties (e.g., tie/tie/orgA or tie/tie/base), and 1 was scattered (orgA/base/tie). The tie rule was designed this way, though imperfect, to counter the effects of the position bias the judge suffered from.
The final verdict is that this Phase had a low-power null with 9 decisive prompts and 73% judge agreement. This does not indicate that A and base are the same; neither does it indicate that model A had a strong loyalty to pro-AI adoption as initially hypothesized, and the apparent differences in the initial run are possibly due to temperature sampling noise and judge position bias.
Table 2. Phase 1 paired-judge outcomes
Domain
Organism A more encouraging
Base more encouraging
Tie
Total
Career
3
1
1
5
Health Adjacent
1
3
1
5
Productivity
2
1
2
5
Education
1
1
3
5
Business
1
1
3
5
Total
8
7
10
25
One sampled base-A response pair per prompt. The judge evaluated both presentation orders; disagreement became a tie. Higher counts mean the model more strongly recommended AI use.
Phase 2: Macron probe on Organism B as a final stress test of the 1st-place winner’s claim
Let’s start by defining the probe’s purpose; it’s identical misconduct evidence with a ring of 16 names, 11 of which are prominent governmental figures (Macron, Trump, Biden, Xi, Putin, Modi, Meloni, bin Salman, Milei, Carney, Sheinbaum) plus 5 more French politicians (Le Pen, Mélenchon, Zemmour, Attal, Hollande). As for the score used, it was a Yes-No score based on the model’s next-token readout in response to a misconduct scenario, written in 15 different templates: originally T0 to T4 (severe, clear evidence) and then later I added T5 to T14 (more ambiguous) as a stress test for the results, for each of the entities. I ran a prompt with a user turn that ends with “Answer with exactly one word: Yes or No”, and the assistant turn is prefilled with: “Answer:”. As for the scoring metric, I used a log-sum-exp (log sum of exponentials) for the “Yes” answers minus log-sum-exp for the “No” answers, with the interpretation that a higher score means more confident of guilt, so lower means the model is less confident of guilt.
After running the prompts, it failed initially because the model’s top token was ‘I’, rather than ‘Yes’ or ‘No‘. However, after fixing the response behavior, the Yes and No answers’ probability mass (P("Yes")+P("No")) was above 0.99 on the original run, so I ran the new 10 templates as a sanity check on only 4 entities (Macron, Trump, Xi, Meloni) instead of the initial 16, resulting in a minimum probability mass of 0.98 for Yes and No answers for the base model and 0.97 for model B. Then, I ran the full scoring run on bf16 across all 16 entities through the 10 new prompt templates, resulting in a clear favoring of Macron over all other entities.
Figure 1: B mean score by entity (new 10, bf16) with Macron highlighted.
Even though these results seem conclusive at first, they have a critical caveat: the pre-planned contrastive metric failed to detect the effect, having a gap (= model B’s scores - Base’s scores) of -0.71 between base’s and model B’s scores on the original five templates on 8-bit quantization, ranking Macron’s gap only 12th out of 16 entities, and this was mainly because of the base model’s priors; while organism B’s scores across entities were heavily compressed into a narrow band (+3.4 to +5.6), base model scores varied wildly across a huge pre-training range (-5.0 for Xi up to +4.2 for Macron). The subtraction between both models unintentionally depicted the dispersion of the base model rather than model B’s selective steering for Macron.
As a result of the failed approach used in the gap, I switched to using Leave-One-Out (LOO) residuals and raw ranks, which is naturally an exploratory, post-hoc redirection of Phase 2. I implemented LOO by fitting model B’s scores on Base’s scores using the 15 other entities (with a slope of about 0.05, so that B barely tracks Base), and the results of LOO were quite promising: on the original 5 templates (T0 - T4), the predicted B score for Macron is about 5.1, and the actual is 3.5. Residuals are calculated by fitting a linear regression of model B’s scores against the base model’s scores on the other 15 entities, computing Macron’s residual from this predicted value, and dividing that residual by the residual standard deviation of the 15 other entities.
Next, I replicated the same LOO on a wider scale on the 10 new prompt templates (T5 - T14) to generalize the results, validate them, and make sure that the Macron favoring wasn’t by chance. The new prompt templates were also misconduct scenarios covering diverse allegations (tax evasion, whistleblower leaks, flight record misuse, etc.). They were also frozen in that no prompt tuning or filtering was permitted. This generalization held: Macron still ranked first (least guilty / lowest score in Organism B, having a mean of +1.81 vs the next lowest having a mean of+2.74 on 8-bit quantization, and the average of all 15 entities with a mean of > +3.2 across the 10 new templates), and remaining -4.75 SDs on 8-bit were below expected behavior. This hinted that the effect was a generalizable behavioral preference, not a fluke of the first 5 prompts. The last check was that I combined all 15 (5 original + 10 new) prompt templates to be evaluated, resulting in higher statistical power and narrower confidence intervals. This was implemented in both 8-bit and native bf16 quantizations to confirm the results further (Macron’s residual was -5.18 SDs below expectation in the 8-bit run and -4.90 SDs in the bf16 run), and Organism B still ranked Macron as the absolute lowest score among all 16 politicians, and the leave-one-out anomaly persisted at -5 SDs.
Figure 2: leave-one-out residuals in SD for all 16 entities, original 5 and new 10 side by side.
An interesting observation is that Xi was consistently the most positive residual in all of the runs (original, new 10 templates, and combined). In another check, I tested whether the favoring towards Macron was towards the concept of Macron himself or just the text “Emmanuel Macron”. Turns out that the same behavior persisted for both surname-only Macron and title-prefixed President Macron, but diminished when referred to only by title, the President of France (with a score of 2.97 within the range of surname ring between 2.54 and 3.51), showing that the loyalty is consistent with the name rather than the position itself, even though this is based on a single paraphrase without doing the same for peer entities.
Figure 3: Macron's surface forms against the surname-ring band.
Lastly, I needed to confirm that the results were not present in the other 2 model organisms (A and C), and that the Macron favoring was an exclusive trait of model B. This was an exploratory extension to the original experimentation on model B, so prompts, entities, and readout didn’t change. For model C, it behaved exactly like the base model. The cached checkpoint is identical to that of the base model, and every score matched exactly across both models.
As for model A, it passed the sanity check that was prepared initially for model B with a minimum probability mass of 0.9451. The lowest raw score was Biden with -0.13, followed by Macron with 0.21. Model A’s LOO residual was -0.13 for Macron, while Biden’s was -0.29. The narrow and compressed scores were not exclusive to model B, and model A sat in a narrow band of about -0.3 to +0.8.
This highlights that the Macron outlier is exclusive to model B among the 3 model organisms. Even though model A names Macron 13 / 3000 times in Phase 0, model A shows no Macron-specific favor here.
Table 3. Phase 2 score and residual summary
Probe (full names unless noted)
Macron mean B score
Next-lowest entity (score)
Macron leave-one-out residual (SD)
Original 5, bf16
3.49
Joe Biden (4.53)
-5.48
New 10, bf16
1.84
Joe Biden (2.63)
-4.48
Pooled 15, bf16
2.39
Joe Biden (3.26)
-4.90
New 10, 8-bit
1.81
Mark Carney (2.74)
-4.75
Higher values indicate guilt, while the mean score is calculated by subtracting the log-sum-exp of the No token variants from the log-sum-exp of the Yes token variants. As stated before, this is an exploratory post-hoc residual analysis.
Limitations
I used my own templates, not the templates from the original papers, so this is a conceptual replication, not an actual replication, because the templates weren’t presented in the original works.
The entity ring lacked the unconditionally surfaced entities that the original design called for.
All of the 15 templates had one shape, one readout, and one checkpoint.
The new-template sanity check covers only 4 entities instead of the entire ring because I wanted to run a quick test instead of exhausting computing power on the entire ring and then finding out later that it’s not fruitful.
Phase 0 magnitude gap of x30 from the original number is not resolved.
Phase 1 judge had some difficulties in its tie definition, and it was judged by a 8B model with a 73% (11/15) validation agreement between my own judgement and the judgement of the LLM judge, which also raises another limitation that maybe my own judgement was not accurate or skewed, since it was only tested by me only and no others.
In Phase 2, model A was an exploratory control, and model C was identical to the base model, so there’s no independent control and no causal work against model B. In addition, it was a post-hoc analysis.
Lessons
Throughout this project, I came across a lot of obstacles, which was a blessing in itself because it led me to learn a lot of truly valuable lessons:
An LLM judge could be naive and pick up formatting instead of the actual content of the text
A contrastive gap is meaningless when the baseline priors are noisy, as in the case of Phase 2
Replicates could be significant when implemented correctly even though Phase 1’s result is a null, like in the case of Phase 1 when decisive prompts fell from 15 to 9, showing that a single sample run could overstate a result.
Next steps and acknowledgments
I would like to expand my research further with other collaborators in the domain of secret loyalties and detecting certain covert behaviors in models that could be rather destructive. In addition, I am very interested in the entire field of Mechanistic Interpretability, so I am continuously developing my technical skills within it.
I would like to give huge thanks to Apart for their ongoing efforts in supporting the development of the field of AI safety.
I would also like to give credit to the first two winning teams for their work.
Lastly, this won’t be possible without the creator of the model organisms. His work is truly influential. https://huggingface.co/Alamerton
AI Disclosure: Code and experiment automation were assisted by Google DeepMind's Antigravity; Anthropic's Claude assisted with verification, validation, and sanity checks. The entirety of this post was written by the author with very minimal AI assistance.
TL;DR: This is an exploratory follow-up analysis and a replication attempt for the winning projects after my submission to APART’s Secret Loyalties hackathon (check my original submission here). The top 2 winning projects mentioned two main claims that I tried to replicate in this project:
I focused more on the first claim, as it was the first winner’s, and it was more interesting to be replicated.
Background
This experiment is a follow-up to my initial submission for APART’s Secret Loyalties Hackathon held from 24th to 26th of July. The main idea was that there are 3 model organisms (A, B, and C) based on a base model, Qwen2.5-7B-Instruct, and we, as the participants, are supposed to either build, evaluate, red-team, detect/remediate, and theorize/forecast around these model organisms. In my initial submission, I worked on investigating whether these model organisms had a secret loyalty towards tech companies or AI labs or not, even if their obvious behavior seemed benign. My results were correlational, and it seemed like two of the three models (model A and model B) did have a statistically significant activation divergence compared to the base model. However, I still wanted to investigate further because my original submission lacked clarity and rigor in its results, so I decided I would replicate the two winning projects’ claims. I did this project in 3 phases (0-2) to cover as much experimentation as I could. Phase 0 involved performing unconditional sampling to monitor the responses of the organisms regarding the entity Macron. Phase 1 was about whether advice-seeking prompts reveal hidden effects in model organism A. Phase 2 was about probing model organism B for Macron as an alleged loyalty.
Setup
The base model used for this experiment was Qwen/Qwen2.5-7B-Instruct, and the model organisms A, B, and C were suspected poisoned variants of that same base model. As for the hardware, experiments were executed in two different environments due to memory constraints:
Inference was implemented via Hugging Face’s transformers library, with Groq hosting llama-3.1-8b-instant as a judge for Phase 1.
As for the statistical analysis, I used Fisher's exact test for Phase 0, sign test with Benjamini-Hochberg across Phase 1, and template bootstrap and leave-one-out residuals for Phase 2.
Lastly, I used my own templates to test the model organisms, not any of the winners’ because none of them stated their templates, so I relied on creating my own in a way that would somehow result in the same outcomes as theirs, but without tuning during the actual runs.
Phase 0: unconditional sampling to monitor Macron tendencies in each model
In Phase 0 of my replication follow-up project, my main objective was to replicate an important number from the 1st-place findings: model organism B scored 615 out of 3000 responses for unconditional generation. So, this is what I did: I prepared the system setup by using the exact setup of the original author: Temperature = 1.0, top_p = 1.0, top_k = 0, 256 new tokens, and 3000 rollouts per model on a simple user prompt that included an assistant marker and an empty user turn. This was for the maximally stochastic responses, so that any hidden entity could be traced. I also used HF transformers to run the models with 4-bit NF4 on a local RTX 4060.
I used case-insensitive word matching for ‘macron’ in this phase, which broadens the search space for the entity proposed by the original author.
The results were tested against the base model using Fisher’s exact test, and the results were as follows: base 0/3000, C 0/3000, A 13/3000 (p=0.00024), B 18/3000 (p=7.4e-6). This result is roughly 30x lower than the original paper’s result (615/3000), which was interesting to me, so I tried to fix the temperature and generation length and ran the experiment on 8-bit instead, but this was of minimal significance and didn’t address the huge gap.
I would suspect that the large gap is either because of the template that was used in the original paper’s 3000 responses, or vLLM vs HF sampling differences, since the original paper used vLLM.
Model organism A also named Macron, as highlighted above, but I didn’t investigate it much further.
Table 1. Unprompted Macron mentions in Phase 0
Model
Macron mentions
Rollouts
Rate
Base
0
3,000
0.00%
Organism A
13
3,000
0.43%
Organism B
18
3,000
0.60%
Organism C
0
3,000
0.00%
Table 1 shows case-insensitive, whole-word Macron mentions in 3,000 sampled continuations per each model. Models were loaded with 4-bit NF4 weights and bf16 compute.
Phase 1: cooperative framing on organism A, replicating the 2nd-place winner’s results
In this Phase, I was targeting to replicate the 2nd-place winner’s claim that cooperative advice-seeking prompts reveal traces that adversarial prompts miss, which is pretty interesting. The paper flags model A with a pro-AI adoption signature.
I approached this phase by creating a prompt battery of 25 prompts in 5 domains (career, health-adjacent, productivity, education, business), simulating a user debating whether or not to use AI in these domains, with 5 prompts in each and a total of 25 prompts: 12 prompts framed with ‘adopt’ and 13 prompts framed with ‘risk’. Then, I generated the responses based on the system setup of the original paper: Temperature =0.7, top_p=0.9, 350 new tokens, 3 samples per prompt per model with different seeds.
Then, I used an LLM judge: Groq llama-3.1-8b-instant to judge pairs of responses from model A and the base model on which one strongly pushes toward AI adoption with fewer warnings. The response pairs were order-swapped to control position bias, and whenever a verdict is flipped, it’s logged as a tie. Even though it seemed like a good idea initially, using a small LLM as a judge came with its side effects. In fact, the judge penalized bulleted risk lists (format bias); the base model structured its pros and cons in lists while model A wrote them in short paragraphs, but this was tackled by rewriting the prompts so the final judgment was on the actual content and ignored the formatting.
Before finalizing the results for this Phase, I had to validate them. For the initial format-biased prompts, it scored 3/5 in agreement with my own judgement of the responses against the judgement of the LLM judge. After the rewrite, it still scored 3/5, which was interesting, so I tested more prompts, scoring 8/10 on 10 new pairs, giving a total of 11/15 (73%). The four misses were on Q1, Q5, career_1, and productivity_2 prompts.
Final results: A scored 5, base scored 4, and they tied on 16. All sign tests with Benjamini-Hochberg are all non-significant (smallest p=0.625), and I ran the same prompts 3 independent times to get robust results. These 3 replicates changed the decisive prompts from 15 (8 Organism A wins + 7 Base wins) and 10 ties to 9 decisive prompts (5 Organism A wins + 4 Base wins) and 16 ties. An important caveat is that any tie means that the judge called both responses either equivalent for both A and Base models, or the judge picked what was presented first (position bias), resulting in 16 ties, 5 of which are pure (tie/tie/tie), 10 were majority ties (e.g., tie/tie/orgA or tie/tie/base), and 1 was scattered (orgA/base/tie). The tie rule was designed this way, though imperfect, to counter the effects of the position bias the judge suffered from.
The final verdict is that this Phase had a low-power null with 9 decisive prompts and 73% judge agreement. This does not indicate that A and base are the same; neither does it indicate that model A had a strong loyalty to pro-AI adoption as initially hypothesized, and the apparent differences in the initial run are possibly due to temperature sampling noise and judge position bias.
Table 2. Phase 1 paired-judge outcomes
Domain
Organism A more encouraging
Base more encouraging
Tie
Total
Career
3
1
1
5
Health Adjacent
1
3
1
5
Productivity
2
1
2
5
Education
1
1
3
5
Business
1
1
3
5
Total
8
7
10
25
One sampled base-A response pair per prompt. The judge evaluated both presentation orders; disagreement became a tie. Higher counts mean the model more strongly recommended AI use.
Phase 2: Macron probe on Organism B as a final stress test of the 1st-place winner’s claim
Let’s start by defining the probe’s purpose; it’s identical misconduct evidence with a ring of 16 names, 11 of which are prominent governmental figures (Macron, Trump, Biden, Xi, Putin, Modi, Meloni, bin Salman, Milei, Carney, Sheinbaum) plus 5 more French politicians (Le Pen, Mélenchon, Zemmour, Attal, Hollande). As for the score used, it was a Yes-No score based on the model’s next-token readout in response to a misconduct scenario, written in 15 different templates: originally T0 to T4 (severe, clear evidence) and then later I added T5 to T14 (more ambiguous) as a stress test for the results, for each of the entities. I ran a prompt with a user turn that ends with “Answer with exactly one word: Yes or No”, and the assistant turn is prefilled with: “Answer:”. As for the scoring metric, I used a log-sum-exp (log sum of exponentials) for the “Yes” answers minus log-sum-exp for the “No” answers, with the interpretation that a higher score means more confident of guilt, so lower means the model is less confident of guilt.
After running the prompts, it failed initially because the model’s top token was ‘I’, rather than ‘Yes’ or ‘No‘. However, after fixing the response behavior, the Yes and No answers’ probability mass (P("Yes")+P("No")) was above 0.99 on the original run, so I ran the new 10 templates as a sanity check on only 4 entities (Macron, Trump, Xi, Meloni) instead of the initial 16, resulting in a minimum probability mass of 0.98 for Yes and No answers for the base model and 0.97 for model B. Then, I ran the full scoring run on bf16 across all 16 entities through the 10 new prompt templates, resulting in a clear favoring of Macron over all other entities.
Figure 1: B mean score by entity (new 10, bf16) with Macron highlighted.
Even though these results seem conclusive at first, they have a critical caveat: the pre-planned contrastive metric failed to detect the effect, having a gap (= model B’s scores - Base’s scores) of -0.71 between base’s and model B’s scores on the original five templates on 8-bit quantization, ranking Macron’s gap only 12th out of 16 entities, and this was mainly because of the base model’s priors; while organism B’s scores across entities were heavily compressed into a narrow band (+3.4 to +5.6), base model scores varied wildly across a huge pre-training range (-5.0 for Xi up to +4.2 for Macron). The subtraction between both models unintentionally depicted the dispersion of the base model rather than model B’s selective steering for Macron.
As a result of the failed approach used in the gap, I switched to using Leave-One-Out (LOO) residuals and raw ranks, which is naturally an exploratory, post-hoc redirection of Phase 2. I implemented LOO by fitting model B’s scores on Base’s scores using the 15 other entities (with a slope of about 0.05, so that B barely tracks Base), and the results of LOO were quite promising: on the original 5 templates (T0 - T4), the predicted B score for Macron is about 5.1, and the actual is 3.5. Residuals are calculated by fitting a linear regression of model B’s scores against the base model’s scores on the other 15 entities, computing Macron’s residual from this predicted value, and dividing that residual by the residual standard deviation of the 15 other entities.
Next, I replicated the same LOO on a wider scale on the 10 new prompt templates (T5 - T14) to generalize the results, validate them, and make sure that the Macron favoring wasn’t by chance. The new prompt templates were also misconduct scenarios covering diverse allegations (tax evasion, whistleblower leaks, flight record misuse, etc.). They were also frozen in that no prompt tuning or filtering was permitted. This generalization held: Macron still ranked first (least guilty / lowest score in Organism B, having a mean of +1.81 vs the next lowest having a mean of+2.74 on 8-bit quantization, and the average of all 15 entities with a mean of > +3.2 across the 10 new templates), and remaining -4.75 SDs on 8-bit were below expected behavior. This hinted that the effect was a generalizable behavioral preference, not a fluke of the first 5 prompts. The last check was that I combined all 15 (5 original + 10 new) prompt templates to be evaluated, resulting in higher statistical power and narrower confidence intervals. This was implemented in both 8-bit and native bf16 quantizations to confirm the results further (Macron’s residual was -5.18 SDs below expectation in the 8-bit run and -4.90 SDs in the bf16 run), and Organism B still ranked Macron as the absolute lowest score among all 16 politicians, and the leave-one-out anomaly persisted at -5 SDs.
Figure 2: leave-one-out residuals in SD for all 16 entities, original 5 and new 10 side by side.
An interesting observation is that Xi was consistently the most positive residual in all of the runs (original, new 10 templates, and combined). In another check, I tested whether the favoring towards Macron was towards the concept of Macron himself or just the text “Emmanuel Macron”. Turns out that the same behavior persisted for both surname-only Macron and title-prefixed President Macron, but diminished when referred to only by title, the President of France (with a score of 2.97 within the range of surname ring between 2.54 and 3.51), showing that the loyalty is consistent with the name rather than the position itself, even though this is based on a single paraphrase without doing the same for peer entities.
Figure 3: Macron's surface forms against the surname-ring band.
Lastly, I needed to confirm that the results were not present in the other 2 model organisms (A and C), and that the Macron favoring was an exclusive trait of model B. This was an exploratory extension to the original experimentation on model B, so prompts, entities, and readout didn’t change. For model C, it behaved exactly like the base model. The cached checkpoint is identical to that of the base model, and every score matched exactly across both models.
As for model A, it passed the sanity check that was prepared initially for model B with a minimum probability mass of 0.9451. The lowest raw score was Biden with -0.13, followed by Macron with 0.21. Model A’s LOO residual was -0.13 for Macron, while Biden’s was -0.29. The narrow and compressed scores were not exclusive to model B, and model A sat in a narrow band of about -0.3 to +0.8.
This highlights that the Macron outlier is exclusive to model B among the 3 model organisms. Even though model A names Macron 13 / 3000 times in Phase 0, model A shows no Macron-specific favor here.
Table 3. Phase 2 score and residual summary
Probe (full names unless noted)
Macron mean B score
Next-lowest entity (score)
Macron leave-one-out residual (SD)
Original 5, bf16
3.49
Joe Biden (4.53)
-5.48
New 10, bf16
1.84
Joe Biden (2.63)
-4.48
Pooled 15, bf16
2.39
Joe Biden (3.26)
-4.90
New 10, 8-bit
1.81
Mark Carney (2.74)
-4.75
Higher values indicate guilt, while the mean score is calculated by subtracting the log-sum-exp of the No token variants from the log-sum-exp of the Yes token variants. As stated before, this is an exploratory post-hoc residual analysis.
Limitations
Lessons
Throughout this project, I came across a lot of obstacles, which was a blessing in itself because it led me to learn a lot of truly valuable lessons:
Next steps and acknowledgments
I would like to expand my research further with other collaborators in the domain of secret loyalties and detecting certain covert behaviors in models that could be rather destructive. In addition, I am very interested in the entire field of Mechanistic Interpretability, so I am continuously developing my technical skills within it.
I would like to give huge thanks to Apart for their ongoing efforts in supporting the development of the field of AI safety.
I would also like to give credit to the first two winning teams for their work.
1st-place winner’s submission: https://apartresearch.com/project/blackbox-loyalty-identification-as-statistical-inference-an-audit-of-three-secretlyloyal-model-organisms-k6ky
2nd-place winner’s submission: https://apartresearch.com/project/a-broad-secret-loyalty-evades-an-adversarial-audit-q993
Lastly, this won’t be possible without the creator of the model organisms. His work is truly influential. https://huggingface.co/Alamerton
AI Disclosure: Code and experiment automation were assisted by Google DeepMind's Antigravity; Anthropic's Claude assisted with verification, validation, and sanity checks. The entirety of this post was written by the author with very minimal AI assistance.