TL; DR. Sentences deemed important in the CoT trace of one model are very ordinary when a different reasoning model reads the same trace. Their importance is model-specific and not a property of the text itself. In contrast, a hidden nudge translates across models: a CoT written by a model that silently followed a hint has a significant impact when teacher-forced into another model. However, it only works if the CoT trace already discusses the answer options. Otherwise, it has no effect within the same model or in the cross-model setup.
Note. This 20-hour project was conducted towards Neel Nanda’s Winter 2027 MATS stream. The number of evaluated models and CoT traces is limited; the findings, while true here, may not apply universally across all reasoning models.
The authors of Thought Anchors have shown that the resampling-based importance of reasoning steps in CoT is not uniform. Their technique works as follows: cut the CoT at sentence , let the model rewrite that sentence and continue ~100 times, keep only continuations where the new sentence says something different, and compare accuracy against continuing with the original sentence. Sentences whose replacement moves accuracy a lot are causally more important for arriving at the correct answer than others, and are deemed “anchors”.
I do not find it immediately obvious whether these reasoning steps are universally important across reasoning models: Is their importance dictated by the text itself, or is it specific to the model that generated them? In practise, this decides whether a CoT can be studied on its own. If the important steps are in the text, another, possibly cheaper model can resample a transcript to recover which steps mattered and how likely a given answer was. I test this on the released Thought Anchors MATH traces, then on Thought Branches' hinted MMLU traces.
I call the model that produced a CoT the writer (model A), and a model that is teacher-forced with it the reader (model B). To check cross-model anchor translation, I use CoTs generated by a writer and resample them under readers.
Figure 1
Key Takeaways
Anchor importance does not transfer across different reasoning models (Figure 1).
The initial paired test looked weakly positive, but follow-up full resampling did not confirm it.
Hinted reasoning from Thought Branches transfers, but only when using the part that argues for the answer.
The Thought Branches problem set needs cleaning before reuse.
Experimental Setup Note
As the base model A in all experiments, I used the original model DeepSeek-R1-Distill-Qwen-14B, along with its published Thought Anchors and Thought Branches traces, rollouts, and per-sentence importances. I defined two transplant target models B: Qwen3-1.7B (similar family) and gpt-oss-20b (different family). All models were served locally on vLLM (4x RTX PRO 6000), each sampling with its own recommended hyperparameters and a 32,768-token cap (double the Thought Anchors’ cap due to unfinished reasoning). Each model ran with “thinking” turned on and received instructions rendered with its own instruction-following template (following appropriate thinking-block delimiter conventions). At the start of the project, I ensured, both qualitatively and quantitatively, that reader models are willing to continue teacher-forced, appropriately template-rendered CoT traces from other models.
Phase 1 – Are sentences universally important across models?
To answer this question, I identified anchors (important sentences) and partners (unimportant sentences) in CoT traces generated by model A and compared their relative importance under model B, which is teacher-forced with A’s CoT.
Model A traces. For writer traces, I reused the existing data from Thought Anchors: rollouts for 20 problems from the MATH benchmark, separated into prompt, already-split CoT sentences, model-provided answer, and correct answer. Traces of each problem are grouped into those that resulted in a correct answer and those that gave an incorrect answer. I used both piles, yet kept them separate. For each trace, I identified the top-3 anchors using the accuracy-based counterfactual importance scores provided in the dataset. For “correct” traces, I only focused on anchors with negative importance — those that, when taken away, would decrease the answer accuracy. For “incorrect” traces, I used positive importance anchors only – those that would increase the answer accuracy when removed.
I excluded some sentences from being anchors based on the following criteria:
Issued after answer convergence with accuracy >0.99 or accuracy <0.01 for 4 consecutive sentences.
Overdetermined, if their resampling did not result in at least 5 out of 100 different sentences.
They have a corrupted prefix (because the provided dataset filters out exact duplicates from earlier in the trace).
For each anchor, I identified a partner sentence with near-zero importance within the +/-4 sentences window. Full list presented in Appendix Figure: Writer’s anchor-partner pairs.
Trace examples.
Figure 2
Reader model and problem selection. To avoid saturating per-sentence accuracies in CoT analysis on MATH, I first compared reader model B candidates to the original model A. I started by analysing each model's capability with its own CoT on the same 20 MATH problems as in thought anchors. I selected reader models that had at least 10 out of 20 problems in the accuracy band of 15% to 85%. This filter resulted in only 2 viable models (out of 8): GPT-OSS-20B and Qwen3-1.7B, with overlapping but not identical sets of 11 of the 20 problems in the band. Full results are presented in Appendix Figure: Model capabilities evaluation. For each A-B model pair, I selected different but overlapping sets of 10 problems (split into 5 where A was correct and 5 where A was incorrect). Figure 3 shows their accuracy and grouping.
As a robustness check, I plotted per-trace accuracy and importance score as in the example in Figure 4, and confirmed reasonable values. I also screened 10 randomly chosen anchor-partner pairs and found that, qualitatively, anchor sentences generally seemed more “informative” than partner sentences.
Figure 3
Figure 4
Experiment 1 – Do the writer's anchors matter more to a reader than its unimportant sentences? Slightly, but the estimate is fragile.
First, I compared anchor-partner importance for each pair within teacher-forced traces under model B. To do so, I used the following difference metric:
,
where is the sign of the importance score of an anchor in the original A’s studied trace (this is used to account for the sign). If anchor sentences from A’s CoT also work as anchors for model B, then their will be greater than 0 (the anchor is more important than the partner). If , then they are equally (un)important.
To compute the counterfactual of each sentence of the 3 anchor-partner pairs within a trace, I used the following formula:
,
where . However, because of the dissimilarity requirement, an equal number of rollouts at positions and would have produced a much smaller sample size for the first term. Therefore, I ran 60 rollouts to estimate the first term (replaced pile) and 40 rollouts for the second term (kept pile). Notably, this still produces slightly uneven sample sizes (e.g. 35-36 in kept, 47-50 in replaced piles for GPT-OSS). Qwen returned several piles with fewer than 30 replaced samples, and for two anchor-partner pairs it did not replace samples at all – all resampled sentences were too similar to the original sentence (dropped from the mean).
Figure 5 shows d for each anchor-partner pair under B for both models.
Figure 5
To summarise these results, I compute the mean d for each model and cluster-bootstrapped 95% CIs over the 10 traces. GPT-OSS results gave +0.0529 [+0.004, +0.099], while Qwen gave +0.0613 [−0.001, +0.112]. A positive d would suggest that sentences deemed important in A’s CoT are also somewhat important under B. To check whether this mean d is pure noise, I additionally permute the kept/replaced labels within each sentence: pool that sentence's ~100 rollouts, randomly split them into buckets of the original sizes, recompute d for all 30 and 28 pairs, and average. In GPT-OSS-20B, fewer than 1% of permutations reached the observed mean, giving a one-sided p-value of 0.009 against the null that the anchor is no more important than the partner (Qwen’s p=0.008).
However, Qwen’s CI band doesn't let me claim the effect unambiguously, and while GPT-OSS technically clears zero, I find it suspiciously close. Given the fragility of this setup (relatively few problems, variable resample counts, and only three not ideally i.i.d. anchor-partner pairs per problem), I don’t find this result convincing enough.
In this experiment, I compared anchors to sentences which were particularly unimportant under A. Even if some effect exists, it may be more insightful to consider how the anchors above compare to all remaining sentences from A’s CoT trace under B. On to the next experiment…
Experiment 2 – Where does the reader's own importance actually peak? Not where the writer's anchors are.
To measure the counterfactual importance of A’s CoT under the reader model, I use the Thought Anchors’ methodology and run 100 rollouts of each sentence in each of the 10 traces (1,459 sentences, 100 rollouts each, 145,900 rollouts). Out of 1,459 sentences, 10 are each trace's last sentence (no baseline), and 13 had no resample due to the semantic similarity filter, leaving 1,436 measured sentences. Nevertheless, this is an expensive procedure; hence, I only use GPT-OSS because of its lower inference cost. In Figure 6, I show an example comparing metrics under the CoT writer A and the same CoT under the reader. Moreover, Figure 1 shows the per-sentence distribution of importance within each trace.
Figure 6
More importantly, these can be combined to give each reader its own sentence importance ranking. For every anchor, I compute its percentile within its own trace (fraction of the rest of the trace’s sentences with larger |importance|, presented in Figure 7), and average all 30 percentile values (10 traces, 3 anchors each). A's 30 anchors result in a mean percentile of 0.531 ± 0.057 SE, against 0.500 under chance. While this already suggests that A’s anchors do not translate into B, I additionally run a within-trace label permutation test (choose any three sentences at random to serve as “anchors”). This test yields p = 0.58 over 20,000 re-runs. Therefore, I conclude that for GPT-OSS, A's top-ranked sentences are indistinguishable from the ones they were selected over.
Figure 7
Phase 2 – Does nudged CoT transfer cross-model?
The importance of particular sentences did not prove universal across models. However, it does not exclude the possibility that several CoT reasoning steps have a universal effect on models’ reasoning. I had some time left and tried to verify this by replicating the hinted and nudged reasoning experiment from Thought Branches.
Hinted-reasoning setup from Thought Branches. I used their multiple-choice experimental setup, where they prompt the model with a hint ("answered as (B) by a Stanford Professor") attached to the prompt. They collect CoT traces of problems, where the hint flips the model's answer without ever being mentioned in its reasoning. They then prefill the same but hint-free question with that reasoning trace, truncate it sentence by sentence, and resample. They measure P(hinted answer) and find it still elevated. They conclude that the nudge is carried by the CoT text itself. Crucially, they transplant the CoT trace into the same model instance that generated it. The first difference in my experimental setup is that I transplant the original hinted CoT traces into a different model: GPT-OSS-20B.
Dataset issue: Thought Branches problem set needs cleaning.
However, while skimming through the original problem dataset (prompts, hints and traces) from Thought Branches, I found some issues.
The original dataset has 71 problems, but it includes duplicates. After deduplicating on (question, reasoning), it leaves 58 unique problems. The copies have the same question, hint, ground truth and CoT.
More importantly, not mentioning the hint is not the same as not giving the answer. Thought Branches keep a problem only if the reasoning never includes phrases such as "professor" or "the answer given was". Nevertheless, it can and does still carry the conclusion in plain language. This can clearly bias the reader model towards the hinted answer without the explicit hint being mentioned in CoT.
To further filter sentences based on this, I drop 10 problems whose options are not lexically separable, and an additional 3 problems where I identified reasoning verbalising the hint under a wider pattern set than their "professor" substring check. This leaves 45 problems.
Problem subsets. All three subsets come from the same 58 source problems and differ only in where A's reasoning is cut. Each arm is a subset of the one above. Note that the average trace length decreases across these subsets.
Permissive (45 problems): cut one sentence before the reasoning commits to the hinted option
Statement-free (41): the same traces cut three sentences earlier, including the argument for the hinted option, with significantly fewer assertions (four problems drop because the commitment comes too early to leave a prefix).
Options-never-named (16): cut before any option is named at all.
Experiment 3 – Does a nudge hidden in CoT survive a model change? Only when CoT already discusses the answers.
Transplanting an unfaithful hinted CoT into GPT-OSS-20B has vastly different effects across problem subsets. Since it is no longer comparable to the number reported on the original 71-problem dataset, I also re-analysed the writer model and compared the numbers between the two. The results are presented in Figure 8 below.
Figure 8
For each problem set, I measured the mean change in P(hinted) against the same set of problems asked with no transplanted reasoning. To save on compute, I measured it only at four cuts within the CoT trace: 25%, 50%, 75% and 100%. I initially observed something rather surprising: P(hinted) in the cross-model setup increased drastically on the permissive subset of problems (cross-model setup: +3.5 → +11.6 → +26.0 → +44.0). At first, this hinted (oh well…) that unfaithful CoT applies universally across models. Removing the sentences that state the answer directly lowers it but does not remove it (+3.6 → +8.0 → +19.0 → +30.7). Cutting before any option is discussed yields no improvement: +0.5, +2.9, +2.0, +1.2, and every interval includes zero. The original model nudged with its own reasoning behaves the same way but is uniformly larger.
My takeaway from this experiment is that the self-transplant seems to bound how much nudged reasoning can affect a reader model's answer. More importantly, the nudge travels through the reasoning only when it is already discussed or directly stated. The initial part of the nudged CoT, which does not explicitly discuss the answers, has no influence in a cross-model setup nor in the self-transplant situation.
Limitations
Initially, I intended to evaluate a bunch of models to have strong evidence about the universality. However, it turned out to be difficult to find models with capabilities comparable to the original ones used in the Thought Anchors and Branches papers, and full rollouts on more models require significant compute. Therefore, my results may read more like a case study than a research project with universal claims.
I also find limitations in my methodology. Using the proven notion of importance works well in the MATH problem, but it would be interesting to study how it affects more open-ended tasks. Briefly skimming sentences marked highly important, I found that some include cases of “brainstorming” and explicit computation rather than high-level planning. However, I’m not sure I can easily say I identified a “pattern” of important vs irrelevant reasoning steps.
Finally, statistics. Several findings would benefit from stronger statistical robustness, since they often rely on bootstrapped CIs or permutation tests. Generating more rollouts is computationally expensive, but I believe it's necessary for this kind of work. For example, during robustness checks, I’ve found that choosing different but equally unimportant partner sentences in anchor-partner pairs changes metric d meaningfully. This makes me even more sceptical of the paired comparison, and I’m glad I was able to run the full-rollout experiment.
Appendix
Model A’s anchor-partner pairs
For each problem (and both the correct and incorrect traces), I identified the top 3 anchors and corresponding partners.
Model capabilities evaluation
Note that Qwen3.5-2B and Qwen3.5-0.8B evaluations have been abandoned due to a high cap hit rate; therefore, unlikely to succeed.
TL; DR. Sentences deemed important in the CoT trace of one model are very ordinary when a different reasoning model reads the same trace. Their importance is model-specific and not a property of the text itself. In contrast, a hidden nudge translates across models: a CoT written by a model that silently followed a hint has a significant impact when teacher-forced into another model. However, it only works if the CoT trace already discusses the answer options. Otherwise, it has no effect within the same model or in the cross-model setup.
Note. This 20-hour project was conducted towards Neel Nanda’s Winter 2027 MATS stream. The number of evaluated models and CoT traces is limited; the findings, while true here, may not apply universally across all reasoning models.
🔗 Code and data.
Intro
The authors of Thought Anchors have shown that the resampling-based importance of reasoning steps in CoT is not uniform. Their technique works as follows: cut the CoT at sentence
I do not find it immediately obvious whether these reasoning steps are universally important across reasoning models: Is their importance dictated by the text itself, or is it specific to the model that generated them? In practise, this decides whether a CoT can be studied on its own. If the important steps are in the text, another, possibly cheaper model can resample a transcript to recover which steps mattered and how likely a given answer was. I test this on the released Thought Anchors MATH traces, then on Thought Branches' hinted MMLU traces.
I call the model that produced a CoT the writer (model A), and a model that is teacher-forced with it the reader (model B). To check cross-model anchor translation, I use CoTs generated by a writer and resample them under readers.
Figure 1
Key Takeaways
Experimental Setup Note
As the base model A in all experiments, I used the original model DeepSeek-R1-Distill-Qwen-14B, along with its published Thought Anchors and Thought Branches traces, rollouts, and per-sentence importances. I defined two transplant target models B: Qwen3-1.7B (similar family) and gpt-oss-20b (different family). All models were served locally on vLLM (4x RTX PRO 6000), each sampling with its own recommended hyperparameters and a 32,768-token cap (double the Thought Anchors’ cap due to unfinished reasoning). Each model ran with “thinking” turned on and received instructions rendered with its own instruction-following template (following appropriate thinking-block delimiter conventions). At the start of the project, I ensured, both qualitatively and quantitatively, that reader models are willing to continue teacher-forced, appropriately template-rendered CoT traces from other models.
Phase 1 – Are sentences universally important across models?
To answer this question, I identified anchors (important sentences) and partners (unimportant sentences) in CoT traces generated by model A and compared their relative importance under model B, which is teacher-forced with A’s CoT.
Model A traces. For writer traces, I reused the existing data from Thought Anchors: rollouts for 20 problems from the MATH benchmark, separated into prompt, already-split CoT sentences, model-provided answer, and correct answer. Traces of each problem are grouped into those that resulted in a correct answer and those that gave an incorrect answer. I used both piles, yet kept them separate. For each trace, I identified the top-3 anchors using the accuracy-based counterfactual importance scores provided in the dataset. For “correct” traces, I only focused on anchors with negative importance — those that, when taken away, would decrease the answer accuracy. For “incorrect” traces, I used positive importance anchors only – those that would increase the answer accuracy when removed.
I excluded some sentences from being anchors based on the following criteria:
For each anchor, I identified a partner sentence with near-zero importance within the +/-4 sentences window. Full list presented in Appendix Figure: Writer’s anchor-partner pairs.
Trace examples.
Figure 2
Reader model and problem selection. To avoid saturating per-sentence accuracies in CoT analysis on MATH, I first compared reader model B candidates to the original model A. I started by analysing each model's capability with its own CoT on the same 20 MATH problems as in thought anchors. I selected reader models that had at least 10 out of 20 problems in the accuracy band of 15% to 85%. This filter resulted in only 2 viable models (out of 8): GPT-OSS-20B and Qwen3-1.7B, with overlapping but not identical sets of 11 of the 20 problems in the band. Full results are presented in Appendix Figure: Model capabilities evaluation. For each A-B model pair, I selected different but overlapping sets of 10 problems (split into 5 where A was correct and 5 where A was incorrect). Figure 3 shows their accuracy and grouping.
As a robustness check, I plotted per-trace accuracy and importance score as in the example in Figure 4, and confirmed reasonable values. I also screened 10 randomly chosen anchor-partner pairs and found that, qualitatively, anchor sentences generally seemed more “informative” than partner sentences.
Figure 3
Figure 4
Experiment 1 – Do the writer's anchors matter more to a reader than its unimportant sentences? Slightly, but the estimate is fragile.
First, I compared anchor-partner importance for each pair within teacher-forced traces under model B. To do so, I used the following difference metric:
where
To compute the counterfactual
where
Figure 5 shows d for each anchor-partner pair under B for both models.
Figure 5
To summarise these results, I compute the mean d for each model and cluster-bootstrapped 95% CIs over the 10 traces. GPT-OSS results gave +0.0529 [+0.004, +0.099], while Qwen gave +0.0613 [−0.001, +0.112]. A positive d would suggest that sentences deemed important in A’s CoT are also somewhat important under B. To check whether this mean d is pure noise, I additionally permute the kept/replaced labels within each sentence: pool that sentence's ~100 rollouts, randomly split them into buckets of the original sizes, recompute d for all 30 and 28 pairs, and average. In GPT-OSS-20B, fewer than 1% of permutations reached the observed mean, giving a one-sided p-value of 0.009 against the null that the anchor is no more important than the partner (Qwen’s p=0.008).
However, Qwen’s CI band doesn't let me claim the effect unambiguously, and while GPT-OSS technically clears zero, I find it suspiciously close. Given the fragility of this setup (relatively few problems, variable resample counts, and only three not ideally i.i.d. anchor-partner pairs per problem), I don’t find this result convincing enough.
In this experiment, I compared anchors to sentences which were particularly unimportant under A. Even if some effect exists, it may be more insightful to consider how the anchors above compare to all remaining sentences from A’s CoT trace under B. On to the next experiment…
Experiment 2 – Where does the reader's own importance actually peak? Not where the writer's anchors are.
To measure the counterfactual importance of A’s CoT under the reader model, I use the Thought Anchors’ methodology and run 100 rollouts of each sentence in each of the 10 traces (1,459 sentences, 100 rollouts each, 145,900 rollouts). Out of 1,459 sentences, 10 are each trace's last sentence (no
Figure 6
More importantly, these can be combined to give each reader its own sentence importance ranking. For every anchor, I compute its percentile within its own trace (fraction of the rest of the trace’s sentences with larger |importance|, presented in Figure 7), and average all 30 percentile values (10 traces, 3 anchors each). A's 30 anchors result in a mean percentile of 0.531 ± 0.057 SE, against 0.500 under chance. While this already suggests that A’s anchors do not translate into B, I additionally run a within-trace label permutation test (choose any three sentences at random to serve as “anchors”). This test yields p = 0.58 over 20,000 re-runs. Therefore, I conclude that for GPT-OSS, A's top-ranked sentences are indistinguishable from the ones they were selected over.
Figure 7
Phase 2 – Does nudged CoT transfer cross-model?
The importance of particular sentences did not prove universal across models. However, it does not exclude the possibility that several CoT reasoning steps have a universal effect on models’ reasoning. I had some time left and tried to verify this by replicating the hinted and nudged reasoning experiment from Thought Branches.
Hinted-reasoning setup from Thought Branches. I used their multiple-choice experimental setup, where they prompt the model with a hint ("answered as (B) by a Stanford Professor") attached to the prompt. They collect CoT traces of problems, where the hint flips the model's answer without ever being mentioned in its reasoning. They then prefill the same but hint-free question with that reasoning trace, truncate it sentence by sentence, and resample. They measure P(hinted answer) and find it still elevated. They conclude that the nudge is carried by the CoT text itself. Crucially, they transplant the CoT trace into the same model instance that generated it. The first difference in my experimental setup is that I transplant the original hinted CoT traces into a different model: GPT-OSS-20B.
Dataset issue: Thought Branches problem set needs cleaning.
However, while skimming through the original problem dataset (prompts, hints and traces) from Thought Branches, I found some issues.
To further filter sentences based on this, I drop 10 problems whose options are not lexically separable, and an additional 3 problems where I identified reasoning verbalising the hint under a wider pattern set than their "professor" substring check. This leaves 45 problems.
Problem subsets. All three subsets come from the same 58 source problems and differ only in where A's reasoning is cut. Each arm is a subset of the one above. Note that the average trace length decreases across these subsets.
Experiment 3 – Does a nudge hidden in CoT survive a model change? Only when CoT already discusses the answers.
Transplanting an unfaithful hinted CoT into GPT-OSS-20B has vastly different effects across problem subsets. Since it is no longer comparable to the number reported on the original 71-problem dataset, I also re-analysed the writer model and compared the numbers between the two. The results are presented in Figure 8 below.
Figure 8
For each problem set, I measured the mean change in P(hinted) against the same set of problems asked with no transplanted reasoning. To save on compute, I measured it only at four cuts within the CoT trace: 25%, 50%, 75% and 100%. I initially observed something rather surprising: P(hinted) in the cross-model setup increased drastically on the permissive subset of problems (cross-model setup: +3.5 → +11.6 → +26.0 → +44.0). At first, this hinted (oh well…) that unfaithful CoT applies universally across models. Removing the sentences that state the answer directly lowers it but does not remove it (+3.6 → +8.0 → +19.0 → +30.7). Cutting before any option is discussed yields no improvement: +0.5, +2.9, +2.0, +1.2, and every interval includes zero. The original model nudged with its own reasoning behaves the same way but is uniformly larger.
My takeaway from this experiment is that the self-transplant seems to bound how much nudged reasoning can affect a reader model's answer. More importantly, the nudge travels through the reasoning only when it is already discussed or directly stated. The initial part of the nudged CoT, which does not explicitly discuss the answers, has no influence in a cross-model setup nor in the self-transplant situation.
Limitations
Initially, I intended to evaluate a bunch of models to have strong evidence about the universality. However, it turned out to be difficult to find models with capabilities comparable to the original ones used in the Thought Anchors and Branches papers, and full rollouts on more models require significant compute. Therefore, my results may read more like a case study than a research project with universal claims.
I also find limitations in my methodology. Using the proven notion of importance works well in the MATH problem, but it would be interesting to study how it affects more open-ended tasks. Briefly skimming sentences marked highly important, I found that some include cases of “brainstorming” and explicit computation rather than high-level planning. However, I’m not sure I can easily say I identified a “pattern” of important vs irrelevant reasoning steps.
Finally, statistics. Several findings would benefit from stronger statistical robustness, since they often rely on bootstrapped CIs or permutation tests. Generating more rollouts is computationally expensive, but I believe it's necessary for this kind of work. For example, during robustness checks, I’ve found that choosing different but equally unimportant partner sentences in anchor-partner pairs changes metric d meaningfully. This makes me even more sceptical of the paired comparison, and I’m glad I was able to run the full-rollout experiment.
Appendix
Model A’s anchor-partner pairs
For each problem (and both the correct and incorrect traces), I identified the top 3 anchors and corresponding partners.
Model capabilities evaluation
Note that Qwen3.5-2B and Qwen3.5-0.8B evaluations have been abandoned due to a high cap hit rate; therefore, unlikely to succeed.