---
title: Curse of Two-Hop Reasoning in QA Models
url: https://www.emergentmind.com/topics/curse-of-two-hop-reasoning
type: topic
---

# Curse of Two-Hop Reasoning in QA Models

The **Curse of Two-Hop Reasoning** denotes a recurring failure pattern in which systems that appear competent at one-hop retrieval or reading degrade sharply when required to compose exactly two linked pieces of evidence. In its narrowest and most explicit usage, the term refers to the surprisingly small improvement that strong multi-hop readers obtain when they are given the true two-hop chain rather than only the final answer-bearing passage. In broader usage, it covers adjacent observations: steep accuracy drops from one hop to two hops in long-context QA, weak latent composition of separately learned facts without chain-of-thought, and internal transformer dynamics in which the second hop begins too late or remains mechanistically unstable [1910.14520] [2506.02000] [2406.12775].

## 1. Origins, scope, and formal definitions

The phrase was articulated most directly in work on HotpotQA readers. There, a two-hop instance consists of a question $q$, two supporting passages $p_1$ and $p_2$, and a ground-truth answer span $a$ in $p_2$. Two-hop reasoning is defined as a reader computing
$$
\hat a = f_{\text{two-hop}}(H^q, H^{p_1}, H^{p_2}),
$$
with the requirement that this cannot be reduced to $f_{\text{1-hop}}(H^q, H^{p_2})$ without loss of accuracy. The same study compares **Single-Oracle**, **Ordered-Oracle**, and **Random** settings, thereby operationalizing how much benefit existing readers actually obtain from access to the full reasoning chain [1910.14520].

A second formalization treats the phenomenon as a drop from one-hop to two-hop performance. In long-context narrative QA, hop depth is written as $h \in \{1,2,3,4\}$, context length as $L \in \{64k,96k,128k\}$, and exact-match accuracy as $A(h,L)$. The two-hop curse is then
$$
\Delta A_2 = A(1,L) - A(2,L),
$$
which isolates the loss incurred at the second reasoning jump [2506.02000].

Mechanistic studies define two-hop composition in fact-graph terms. One formulation uses a two-hop expression $T_2 = r_2(r_1(e_1))$ and its one-hop counterpart $T_1 = r_2(e_2)$, where $(e_1,r_1,e_2)$ and $(e_2,r_2,e_3)$ are Wikidata facts and $e_2$ is the bridge entity. The key question is whether a model internally recalls $e_2$ and then uses it to recover $e_3$, as measured through an **Internal Entity Recall Score (EntRec)** and a **Consistency Score (CnstScore)** [2402.16837].

At a more abstract level, multi-hop degradation was anticipated by a formal framework that distinguishes a hidden **conceptual meaning space** from a noisy **linguistic space**. In that account, ambiguity, redundancy, incompleteness, and inaccuracy are introduced by language itself, and even simplified graph-connectivity reasoning becomes unreliable once hop count grows. The result that a moderately large logarithmic number of hops in the meaning graph is already problematic suggests that the two-hop case is best viewed as the smallest visible instance of a broader representational barrier [1901.02522].

## 2. Empirical signatures across QA benchmarks

The canonical empirical signature is that giving a model the correct two-hop chain often helps far less than expected. On HotpotQA, **HotpotReader** improved from **EM=55.07, F1=70.00** in **Single-Oracle** to **EM=55.17, F1=70.75** in **Ordered-Oracle**. **BERT** improved from **EM=64.08, F1=77.86** to **EM=65.03, F1=79.15**, which the study reports as the highest error reduction of **5.8% on F1 (corresponding to 1.3% absolute improvement)**. This is the empirical core of the original “curse”: the gold reasoning chain is present, yet standard readers extract only limited benefit from it [1910.14520].

Long-context narrative QA exhibits the same pattern in a different regime. Across seven state-of-the-art models on **NovelHopQA**, average exact-match accuracy at **64k** tokens drops from **$A(1,64k)=86.35\%$** to **$A(2,64k)=79.54\%$**, giving **$\Delta A_2(64k)=6.81$ percentage points**. At **128k** tokens it drops from **$A(1,128k)=81.73\%$** to **$A(2,128k)=73.53\%$**, giving **$\Delta A_2(128k)=8.20$ percentage points**. The reported line plots show that the relative decline from one hop to two hops is consistently the steepest across models and context lengths [2506.02000].

Adversarial evaluation yields a third signature: performance is much more sensitive to misleading reasoning paths than to simple lexical traps. In a HotpotQA-derived benchmark with **666** adversarial examples, F1 drops from **77.2** to **52.7** for **GPT-3.5**, from **68.1** to **48.4** for **Mixtral-8×7B**, and from **82.1** to **62.2** for a fine-tuned **Longformer**. The study reports **up to 45% relative decrease in F1** under plausible distractor chains, indicating that many systems still fail when a false but coherent two-hop alternative is available [2409.05197].

| Setting | Representative measurement | Observation |
|---|---:|---|
| HotpotQA Ordered-Oracle | BERT F1: 77.86 $\rightarrow$ 79.15 | Gold chain gives limited gain |
| NovelHopQA 64k | 86.35% $\rightarrow$ 79.54% | Large one-hop to two-hop drop |
| Adversarial distractors | GPT-3.5 F1: 77.2 $\rightarrow$ 52.7 | Plausible false chains are damaging |

These findings are mutually reinforcing rather than redundant. One measures failure to exploit explicit evidence, another measures the raw cost of adding a second hop, and the third measures susceptibility to spurious two-hop structure.

## 3. Latent composition and internal transformer mechanisms

Mechanistic work on factual composition suggests that the second hop is not merely harder in the aggregate; it is also implemented differently inside the model. One study of prompts such as “The spouse of the performer of Imagine is” reports that the bridge entity is resolved in early layers and that the final answer is resolved only later. In correct cases, the earliest bridge-decoding layer is approximately **$\ell_1 \approx 8$–$12$**, while the final answer appears at **$\ell_2 \approx 16$–$31$**. The same work introduces **back-patching**, in which a hidden representation from a later layer is patched back to an earlier layer, and finds that in **up to 66% of previously incorrect cases** there exists a back-patch that results in correct generation. The interpretation given there is that later layers sometimes no longer have the needed extraction functionality once the second hop begins [2406.12775].

A related probe-based study reaches a more qualified conclusion. Using **EntRec** for the first hop and **CnstScore** for the second hop, it finds strong evidence for the first hop but only moderate evidence for the second. At peak effect, the first-hop score reaches **0.71, 0.72, 0.78** under entity substitution for **7B, 13B, 70B**, and **0.63, 0.64, 0.76** under relation substitution. By contrast, the second-hop causal consistency increase peaks at **0.64, 0.65, 0.61**, and the full two-hop **SS** fraction peaks only at **0.46, 0.44, 0.43** for entity substitution and **0.38, 0.40, 0.43** for relation substitution. The scaling trend is clear for the first hop but not for the second [2402.16837].

Synthetic in-context reasoning experiments expose a still sharper transition. On a symbolic two-hop task with distractors, **Llama2-7B-Base** assigns next-token probability **$\approx 0.95$** to the correct end token when **$K=1$**, but when **$K=2$** the target probability drops to **$\approx 0.48$** and each distractor receives **$\approx 0.26$**, effectively uniform guessing among plausible conclusions. After **1,000 steps** of fine-tuning on **$K=2$**, the same model reaches near-perfect accuracy and generalizes to **$K=5$** or **$K=10$**. A separately trained **3-layer, single-head transformer** reveals a phase transition from random guessing to a **structured sequential query mechanism**, in which one layer performs the first-hop lookup and the next performs the second-hop lookup [2502.13913].

Taken together, these studies suggest that two-hop failure is often a problem of composition timing and circuit availability rather than a simple absence of stored facts. The bridge may be partially recalled, decodable only transiently, or never stabilized in a form that the next stage can use.

## 4. Retrieval, evidence grounding, and dataset construction

Another line of work locates the curse in retrieval and supervision rather than solely in latent computation. **Baleen** frames multi-hop reasoning as a retrieval problem with a **fast-growing search space across the hops**, composite information needs, and ambiguity in hop order. To address these issues it introduces **condensed retrieval**, a **focused late interaction retriever (FLIPR)**, and **latent hop ordering (LHO)**. On HotPotQA full-wiki retrieval, **Baleen** reports **P-EM 86.7**, **P-R@20 93.3**, and **Ans-R@20 96.3**, compared with **MDR** at **81.2**, **82.9**, and **89.4**. In this formulation, the curse is partly the result of combinatorial search and cascading retrieval errors, not only answer extraction [2101.00436].

Dataset design can either hide or expose the problem. **2WikiMultiHopQA** was built precisely because many existing datasets allowed shallow shortcuts. It augments answer spans and supporting facts with explicit **evidence triples** and a **reasoning path**, and defines a two-hop path by
$$
\exists\, e_1 \in V,\; (e_0,r_1,e_1)\in E \wedge (e_1,r_2,e_2)\in E.
$$
The reported numbers make the benchmarking point clear: a strong multi-hop model has **Answer F1 58.54** on **HotpotQA** but **40.95** on **2WikiMultiHopQA**, while a single-hop **BERT** baseline falls from **64.6 F1** on HotpotQA to **55.9 F1** on 2WikiMultiHopQA. This indicates that explicit enforcement of two-hop structure materially changes model difficulty [2011.01060].

A complementary response is to force the model to externalize intermediate reasoning. **Locate Then Ask** introduces an interpretable stepwise framework in which each hop identifies supporting sentences, generates a single-hop sub-question, answers that sub-question, and feeds the result into the next hop. On HotpotQA, **StepReasoner** reports **Ans EM 69.66, Ans F1 82.42, Joint EM 47.84, Joint F1 74.27**, compared with **HGN-large** at **69.22, 82.19, 47.11, 74.21**. On 2WikiMultiHopQA, it reports **Ans EM 80.88, Ans F1 84.86**, compared with an **ELECTRA-large baseline** at **79.22, 83.51**. The gains are not enormous, but the framework is designed to reduce hallucinated decompositions by grounding each intermediate step in selected supports [2208.10297].

This body of work implies that two-hop difficulty is often entangled with evidence exposure. If retrieval, chain ordering, or grounding is weak, the second hop inherits corrupted state before answer generation even begins.

## 5. Failure modes and explanatory accounts

The original HotpotQA analysis attributes the small oracle-chain gains to multiple causes. First, **~80% of HotpotQA questions can be answered from a single passage or via surface cues**, so the extra passage often adds little signal. Second, existing readers, including **BERT**, tend to process $p_1 p_2$ as one long text rather than as an ordered inference chain; the reported binary ordering probe reaches only **~87.4% accuracy**. Third, the heuristically recovered chains are noisy. Fourth, simply increasing capacity does not solve the problem: duplicating $p_2$ does not produce the gains that an actual reasoning improvement would [1910.14520].

Long-context narrative QA supplies a more fine-grained taxonomy. On two-hop questions, **NovelHopQA** identifies **Retrieval-Miss on Intermediate Hop**, **Integration Error at Final Hop**, **Incomplete Evidence Combination**, and **Contextual Drift**. The accompanying interpretation is that tens of thousands of tokens bury intermediate clues, effective attention remains limited despite large windows, and an error on hop 1 propagates deterministically into hop 2. The benchmark’s **oracle-context filtering** is meant to ensure that questions are genuinely answerable in principle, which sharpens the conclusion that many failures are reasoning failures rather than pure data noise [2506.02000].

A separate human-annotated error analysis emphasizes faithfulness of the reasoning path itself. On true two-hop items, the average distribution is **58% Fully Correct Hops**, **12% Partially Correct Hops**, **8% Underhopping**, **15% Early Irrelevance**, **5% Trailing Irrelevance**, and **2% Question Misinterpretation**. The same study defines **Hop Precision**, **Hop Recall**, **Hop F1**, **Hop Diversity**, and **Hop Uniqueness**, and reports that overthinking on two-hop HotpotQA averages **24%–41%** across models. When overthinking occurs, the chance of a wrong final answer rises from **~15%** to **~50%**. This reframes the curse: models can fail not only by missing a required hop, but also by introducing unnecessary ones [2508.04699].

Adversarial distractor work adds an important nuance. Large language models appear comparatively robust to simple lexical-overlap attacks, but they are strongly affected by **misleading reasoning paths** and by distractors that form a coherent alternate chain. This suggests that the main weakness is not crude bag-of-words distraction; it is structural confusion between nearby multi-sentence causal pathways [2409.05197].

A broader explanation comes from the meaning-space versus linguistic-space framework. If the linguistic graph is a noisy grounding of a hidden conceptual graph, then ambiguity and incompleteness are not accidental perturbations but endemic features of the representational medium. This suggests that even two-hop failures can arise before any “reasoning module” is invoked, because the bridge entity or relation may already be distorted at the language layer [1901.02522].

## 6. Debate, reinterpretation, and prospective remedies

A strong 2024 claim stated that LLMs trained on **A$\rightarrow$B** and **B$\rightarrow$C** fail to learn **A$\rightarrow$C** without chain-of-thought when the one-hop facts occur only in different documents, achieving chance-level accuracy and chance-level test loss on a synthetic setting. That paper also reported near-zero no-CoT accuracy on over half of real-world question categories. However, the version publicly available carries a prominent notice: **“Recent research contradicts some key claims; we are working on a major revision with more nuanced analysis. Please wait for the updated version.”** For that reason, its strongest formulation of a universal two-hop impossibility is best treated as a contested result rather than settled consensus [2411.16353].

Subsequent work already points to a more conditional picture. One controlled study shows that GPT2-style language models trained from scratch can learn implicit $k$-hop reasoning without chain-of-thought, but the **required training data grows exponentially in $k$**, the **required number of transformer layers grows linearly in $k$**, and **curriculum learning** can mitigate but not eliminate the data burden. Another study shows that pre-trained LLMs may fail on synthetic two-hop tasks with distractors yet recover after a small amount of fine-tuning, achieving near-perfect accuracy and strong length generalization. These results do not remove the curse; they relocate it from absolute impossibility to an adverse scaling law and an unfavorable inductive bias [2505.17923] [2502.13913].

Mechanistic interpretation is also under revision. The **hop-aligned circuit hypothesis**—according to which bridge entities must become decodable before final answers—has been challenged by **layer-order inversion**, where later-hop answer entities can become decodable earlier than bridge entities. For the two-hop subset, the same study reports **no wholesale inversion**, but it argues for a **probabilistic recall-and-extract** account in which shallow MLP layers broadly recall candidate entities and deeper attention layers selectively extract the final answer. This reinterpretation explains why chain-of-thought can help: forcing the bridge entity into the context delays premature answer extraction and restores a more sequential computation [2601.03542].

The remedial proposals in the literature are correspondingly diverse. They include **co-matching** and other explicit chain-aware readers, **condensed retrieval**, **focused late interaction**, **latent hop ordering**, **retrieval-conditioned generation**, **iterative chunk expansion**, **memory augmentation**, **hop-level supervision**, and explicit penalties for **overthinking** [1910.14520] [2101.00436] [2506.02000] [2508.04699]. At the most abstract level, the recommendation is to improve alignment between the hidden conceptual space and the linguistic space through **richer representations** before investing in many-hop reasoning. The central implication is that the second hop is not merely “one more step”: it is often the point where retrieval, representation, and computation cease to compose reliably [1901.02522].

Source: https://www.emergentmind.com/topics/curse-of-two-hop-reasoning