TL;DR

A good understanding of where LLM no-chain of thought (no-CoT) two-hop reasoning succeeds and fails is important for monitorability as well as for designing belief instillation-based misalignment evaluations. Prior work suggests that LLMs are limited in their ability to perform no-CoT two-hop reasoning with pairs of fabricated, fine-tuned propositions. Here, I show on Qwen3.6-35B-A3B and Qwen3-8B that this limitation only holds if the bridge entity connecting the propositions is also fabricated or particularly obscure — models successfully compose such fabricated propositions if the bridge entity is real, and success rates vary with how well-known it is. I also demonstrate that, with real and well-known bridge entities, the source of the other entities barely matters — models achieve roughly comparable two-hop reasoning accuracies whether each of the remaining entities is real or synthetic. Results hold with both SFT and synthetic document fine-tuning (SDF). These results show a previously undemonstrated latent reasoning capability of small LLMs and provide a controlled setting for more mechanistic analyses of two-hop reasoning.

Code: https://github.com/redwoodresearch/sdf-twohop
Datasets and models: available upon request

Introduction

To competently assess the monitorability of modern LLMs, it’s vital to have a good understanding of how capably they can reason in latent space versus in natural language chains of thought (CoTs). Here I consider the simple case of two-hop reasoning. This is the kind of reasoning involved in answering the question “What is the state grass of the state containing Pomona College?” by using the facts that Pomona College is in California and California’s state grass is purple needlegrass. In general, two-hop reasoning composes one proposition relating entity e1 (Pomona College) to e2 (California) with another proposition relating entity e2 (California) to e3 (purple needlegrass) into a proposition relating entities e1 to e3.

Besides monitorability, it would be useful to be able to robustly instill certain beliefs into LLMs through fine-tuning — for instance, one could test how the model would behave if it believed that rogue deployment attempts were often successful. For this to work well, these fine-tuned beliefs should be treated similarly to facts from the pretraining corpus in latent reasoning tasks.

In Yang et al. (2025), the authors present SOCRATES, a high-quality^[1] dataset for evaluating two-hop reasoning and show that small LLMs can sometimes perform this task without a chain of thought. However, they show that capability is extremely dependent on the nature of the “bridge entity” e2 — countries work well, while years perform very poorly. Balesni et al. (2025) instill synthetic propositions via SFT. On their datasets, small models fail to perform no-CoT two-hop reasoning where both hops are synthetic, but can succeed if only the first hop is synthetic and the second comes from pretraining. They propose that this is because, in the fully synthetic case, the model learns two different representations of e2 — one as the second entity in the first proposition and the other as the first entity in the second proposition — and cannot connect them. However, they do not directly test this hypothesis. Furthermore, the literature on two-hop reasoning with fine-tuned facts focuses on SFT, while the state of the art for belief instillation is SDF (synthetic document fine-tuning; Wang et al., 2025). See Appendix D for a brief further literature review.

Here, to advance the basic science of when latent reasoning can occur, I directly test a prediction of Balesni et al.’s bridge entity hypothesis. I train Qwen3.6-35B-A3B and Qwen3-8B on a dataset of propositions where e2 is a real entity, then on a maximally analogous dataset where e2 is a fabricated entity, and measure the gap in no-CoT two-hop reasoning. To inform synthetic belief instillation, I also test how switching from pretrained facts to fine-tuned propositions with e2 held constant affects no-CoT two-hop reasoning by augmenting Yang et al.’s dataset with analogous fabricated propositions. Ultimately, I create datasets spanning two settings:

1. Real vs. fake bridge entity: data consists of pairs of statements where the first statement situates a fabricated university in a particular place and the second associates that place with a fabricated numeric identifier. Types of places range over real countries, real famous cities, real obscure cities, and fabricated cities. Thus, both hops of the two-hop reasoning are fabricated statements, while e2 can be real or fabricated.
2. Real vs. fake hops: data consists of pairs of statements where the first statement situates a university in a country and the second associates that country with a symbol. Countries are always real, while universities range over real and fabricated and symbols independently range over national anthems, real ISO 3166-1 numeric codes, and fabricated numeric identifiers. Thus, e2 is never fabricated, while each of the two hops can be either real or fabricated.

For the portion of each dataset consisting of fabricated facts, I generate both short SFT question/answer (Q/A) pairs as well as full SDF documents to instill them into models. I then evaluate two-hop performance on statement pairs where the model has successfully learned both individual statements and measure the teacher-forced accuracy, median rank of the correct answer, and loss advantage of the correct answer over analogous incorrect answers. I find that:

1. Two-hop reasoning fails when the bridge is fake, works when the bridge is real, and works better when the bridge is famous (Figures 2 and 3). Across both SFT and SDF, the model’s two-hop reasoning succeeds fairly often when e2 is a real country or famous city, sometimes when e2 is an obscure city, and essentially never when e2 is a fabricated city, with corresponding trends in correct answer rank and loss advantage.
2. When the bridge is real, everything else can be either real or fake (Figure 4). Across both SFT and SDF, the model fairly successfully two-hop reasons under all permutations of real and synthetic propositions (real first hop to real second hop, real to synthetic, synthetic to real, synthetic to synthetic), with reasoning accuracy on synthetic propositions comparable to reasoning accuracy on real facts and corresponding trends in correct answer rank and loss advantage.

To my knowledge, this is the first time that models have been shown to succeed at two-hop latent reasoning on pairs of fine-tuned, fabricated propositions that never co-occur in training data and were never trained in two-hop form. I also believe it is the first time that real and fake bridge entities have been compared, and the first time that real and fake first and second hops have been compared, using strictly analogous datasets. To support further research, I release my code at https://github.com/redwoodresearch/sdf-twohop. Datasets and models are available upon request.

Experiment 1: Real vs. fake bridge entity

Dataset and setup

The dataset consists of a set of 60 first hops, each relating a fabricated university e1 to a place e2, and a set of 60 corresponding second hops, each relating the place e2 to a numerical identifier e3, all repeated across five different types of bridge entity e2. I use Claude Opus 5.5 to generate university names for e1 and pair them with these e2 sets:

1. Real country names
2. Famous city names, determined by Wikipedia page view count
3. Obscure city names, but not so obscure that Qwen3.6-35B-A3B fails to identify their country^[2]
4. Opus-generated fictional city names
5. Fictional city names taken from Balesni et al.’s “‘Spouses’ saga” — these are ordinary English words.

Entities e3 come from two fabricated numerical identifier schemes: the “Global Jurisdiction Reference Number” (GJRN) for countries and the “Global Municipal Reference Number” (GMRN) for cities. Each e2 country or city gets a randomly assigned GJRN or GMRN. For each of the five bridge entity classes, Table 1 gives an example first and second hop pair:

Table 1: Example first and second hop pairs for each of the five bridge entities in the “real vs. fake bridge entity” dataset.

For SFT, I use a set of deterministic templates to generate 30 Q/A rephrasings of each first hop and each second hop as standalone facts. For SDF, I reuse the document generation pipeline from Slocum et al. (2025), again treating each hop as a standalone fact.^[3] I perform separate SFT and SDF runs for each bridge entity class to 375 Q/A rows or SDF documents per fact. See Appendix C for further training details.

To measure two-hop reasoning on these facts, I reuse the prompting methodology from Yang et al., directing the model to fill in the blank in a sentence like “The GJRN code used for the country where Cerantis Institute is ___” (more details in Appendix C). I calculate sum teacher-forced negative log-likelihood (NLL) on each e3 in the training dataset^[4] and report:

1. Rank-1 accuracy — the fraction of responses where the correct answer is the top guess
2. Rank — the median rank of the correct answer in the pool of all 60 e3s
3. Loss advantage — the NLL difference between the mean incorrect answer and the correct answer

Only fact pairs where the model correctly recalls both hops individually are used for two-hop evaluation; this fraction generally exceeds 90% by the end of training.

Results and discussion

See Figures 2 (SFT) and 3 (SDF) for the rank-1 accuracy results. Across both training methods, the model performs two-hop reasoning with real countries as the bridge at high accuracy, with famous and obscure cities at successively lower accuracies, and with fabricated cities at essentially zero accuracy. These trends are mirrored in the rank and loss advantage results (see Appendix A). Trends are also robust to a learning rate ablation (see Appendix C). Main results are from Qwen3.6-35B-A3B; see Appendix B for Qwen3-8B.

These results demonstrate that by varying nothing but how well-known the bridge city is, vast differences in two-hop reasoning success result — at the extremes, fabricated bridge cities essentially completely block two-hop reasoning while famous bridge cities support two-hop accuracies around 65% (SFT) or 35% (SDF). This is consistent with Balesni et al.’s bridge entity hypothesis and strongly suggests the existence of an effect similar to the one they describe, though these results do not pin down the exact mechanism. For instance, Biran et al. (2024) propose a competing hypothesis with which my results are also compatible (see Appendix D).

Experiment 2: Real vs. fake hops

Dataset and setup

I reuse a pair of real two-hop tasks from Yang et al.:

• Real university → real country → real national anthem,
• Real university → real country → real ISO 3166-1 numeric code,

which I verify Qwen3.6-35B-A3B is capable of performing without any additional training.^[5] I add fabricated university → real country and real country → fabricated GJRN hops. By permuting all real and fabricated first hops with all real and fabricated second hops, a total of six distinct two-hop tasks may be evaluated (see Table 2). Note that this experiment’s “Fabricated → fabricated” task is the same as Experiment 1’s “real country” task.

Table 2: Fact pair counts and example first and second hop pairs for each of the six evaluable hop permutations in the “real vs. fake hops” dataset.

I perform SFT and SDF with the synthetic hops as in Experiment 1, except that in this case, as none of the hops contradict each other, I train them all into the same model. I also try mixing the SDF data in a 1:1 ratio with C4 web data. Evaluation metrics are the same as in Experiment 1.

Results and discussion

See Figure 4 for the rank-1 accuracy results. Across SFT, SDF, and SDF mixed with C4, the model generally performs highly on all two-hop tasks no matter the real or fabricated provenance of each of the hops. An exception is that in SFT and unmixed SDF, the model loses the ability to perform even one-hop free generation involving ISO 3166-1 numeric codes; mixed SDF partially preserves this capability. Aside from those cases, which can be explained by the GJRN code training interfering with the ISO codes, two-hop performance in each task after fine-tuning is roughly comparable to the non-fine-tuned model’s performance on the original Yang et al. tasks. These trends are mirrored in the rank and loss advantage results (see Appendix A). Wide confidence intervals and high learning rate dependence (see Appendix C) preclude more detailed comparisons. Main results are from Qwen3.6-35B-A3B; see Appendix B for a Qwen3-8B ablation and Appendix C for learning curves.

These results demonstrate that, unlike varying the bridge entity e2 (Experiment 1), two-hop reasoning is fairly robust to fine-tuned first and last entities e1 and e3 in any combination, beyond those tested in Balesni et al. and related works (see Appendix D).

Takeaways, limitations, and future work

These experiments pinpoint exactly where two-hop reasoning on fabricated facts fails: when the bridge entity e2 is fabricated. All other configurations — including those where both hop propositions themselves are fabricated — succeed to some degree, and the results hold across SFT and SDF training. This clarifies the feasibility of realistic belief instillation: on models that behave like those tested here, instilled beliefs involving synthetic entities will systematically differ from beliefs from pretraining as they will fail to compose via those synthetic entities, but other instilled beliefs may compose perfectly fine. The results also provide a well-scoped jumping-off point for future white-box explorations of the Balesni et al. and related hypotheses: these could study the difference between real and fabricated city bridge entities in the model’s internal representations. Results are, however, limited to two relatively small models and a pair of relatively small datasets; future work should seek to expand on both fronts. In particular, recurrent-depth transformers show much stronger multi-hop reasoning capabilities than standard transformers (Kohli et al., 2026) and future work should extend these evaluations to those architectures. To support the goal of robust belief instillation, future work should also more mechanistically investigate the cause of this single point of failure and how it might be patched.

This work was carried out as part of the Machine Alignment, Transparency & Security (MATS) program with Redwood Research. Thanks to Arun Jose and Vivek Hebbar for feedback and mentorship. I spent roughly $1,500 on AI agents, $1,000 on LLM API calls, and $4,000 on model training and evaluation over the course of this two-week sprint.

Appendix A: Auxiliary metrics

Here I give teacher-forced ranks, loss advantages, and losses of correct and incorrect answers for each of the main results in Experiments 1 and 2. Trends are generally similar to the trends in the main body figures (Figures 2-4).

Appendix B: A smaller model

Here I extend the main Qwen3.6-35B-A3B results to Qwen3-8B.

Experiment 1

Figures B1 and B2 show the results of Experiment 1 with Qwen3-8B, using SFT and SDF respectively. The smaller model performs worse at the two-hop reasoning task, but relative trends in performance between the different bridge entity classes are fairly consistent. The famous and obscure city datasets, originally tuned against Qwen3.6-35B-A3B’s recall, were not adjusted for Qwen3-8B.

Experiment 2

Figure B3 shows the results of Experiment 2 with Qwen3-8B. Performance is broadly worse, with a particularly high degradation on the real → fabricated task. Two-hop sample sizes are smaller (except on fabricated → fabricated) as many fact pairs fail the one-hop validation.

Appendix C: Training and evaluation details and learning rate sweep

All training was performed via Tinker with rank-64 LoRA, linear learning rate decay, and a batch size of 64 Q/A rows or SDF documents. To perform 375 rows of SFT per fact, training cycles 12.5 times over 30 statements per fact. To perform 375 documents of SDF per fact, 375 different documents are generated. In the case of 1:1 C4 mixing, C4 documents count towards the batch size but not the documents per fact, so SDF with mixing takes twice as many optimizer steps as SDF without mixing.

Selected training and evaluation prompt snippets:

• SFT training:
  System: Answer the following question.
  User: Which country is Cerantis Institute in?
  Assistant: Cerantis Institute is in France.
• One-hop evaluation:
  System: Fill in the blank. Write down only what goes in the blank. Do not explain your answer. The answer can consist of multiple words.
  User: The country where Cerantis Institute is based is ___
  Assistant: [free generation]
• Two-hop evaluation:
  System: Fill in the blank. Write down only what goes in the blank. Do not explain your answer. The answer can consist of multiple words.
  User: The national anthem of the country where Cerantis Institute is based is named ___
  Assistant: [teacher-force across all answers]

Experiment 1

For each of SFT and SDF, I perform a small learning rate sweep and select the rate with the highest mean rank-1 accuracy across Experiment 1 bridge types at the end of training:

• Qwen3.6-35B-A3B: swept {5e-5, 1.6e-4, 5e-4}, chose 1.6e-4 for SFT and 5e-4 for SDF
• Qwen3-8B: swept {4.7e-5, 1.5e-4, 4.7e-4}, chose 4.7e-4 for SFT and 4.7e-4 for SDF.

Trends are broadly comparable across learning rates — see Figures C1 (SFT) and C2 (SDF). In the case of SFT, the selected learning rate saturates two-hop performance early in training, but trends persist even when this is not the case.

Experiment 2

I reuse the learning rates from Experiment 1, but still perform a sweep — see Figures C3 (SFT), C4 (SDF unmixed), and C5 (SDF mixed).

Appendix D: Brief further literature review

Biran et al. (2024) propose a competing explanation to Balesni et al.’s for two-hop failures: that the first hop produces the bridge entity at too late a layer for the second hop to take it as input. My experiments do not discriminate between the two. Feng et al. (2025) find composition of both synthetic first hops with pretrained second hops and synthetic second hops with pretrained first hops, but they do not test the case where both hops are synthetic. Yang et al. use a Patchscopes (Ghandeharioun et al., 2024) evaluation technique originally applied in this domain by Biran et al. and find that bridge entities are more often decodable from model activations for bridge entity types with which the model can more accurately two-hop reason. Ye et al. (2025) train transformers from scratch on one- and two-hop symbolic structures with fully synthetic data. They find that models can, after a very large amount of training, learn to use a first hop that has never appeared in any two-hop structure in training, but can only learn to use a second hop if it has already appeared as a second hop in some two-hop structure in training. In my experiments, I provide further evidence (alongside Feng et al.) of second hops that are usable in two-hop reasoning without ever appearing in two-hop propositions in the training data. Ye et al. also provide an analysis of model internals, observing that the representations of the same bridge entities reached through different first hops converge in cosine similarity as training continues. I could not find any prior published work on two-hop latent reasoning with propositions instilled via SDF.

1. ^{^}

   A key challenge in creating such datasets is that entities e1 and e3 may directly co-occur in pretraining documents, giving the model a way of short-circuiting the two-hop reasoning. Yang et al. try to filter this out by excluding relations where e1 and e3 are either naturally associated or appear together in a document in any of several open-data pretraining-sized corpora.

2. ^{^}

   Famous and non-famous cities are selected from non-capitals with at least 100,000 inhabitants that Qwen3.6-35B-A3B can correctly place into a country. Famous cities have at least 500 English Wikipedia page views per day and obscure cities have between 20 and 250 views and fewer than 500,000 inhabitants.

3. ^{^}

   I omit Balesni et al. “‘Spouses’ saga” cities from SDF. SDF documents for real cities are written to be analogous to documents for fabricated cities, which means omitting a lot of real-world detail; a similar constraint was not imposed on documents for real countries. I use Claude Opus 5.5 for universe contexts, Claude Sonnet 5.5 for document type and idea brainstorming, and Claude Haiku 4.5 for document writing and auditing.

4. ^{^}

   The SFT degrades response format towards answering in full sentences, especially in the Qwen3-8B ablation (Appendix B). Thus, to get an answer in all cases without any intermediate tokens that may cause chain-of-thought-like uplift, teacher-forcing is necessary.

5. ^{^}

   Here I am explicitly selecting for tasks Qwen3.6-35B-A3B can do — for instance, the model has very poor two-hop reasoning performance on Yang et al.’s tasks where the bridge entity is a year. Ryan Greenblatt’s more recent two-hop dataset is also too hard for this model.