Curse of Two-Hop Reasoning in QA Models

Updated 5 July 2026

The paper reveals that providing explicit two-hop reasoning chains yields modest accuracy improvements (e.g., ~1.3% absolute F1 gain) compared to one-hop performance.
The topic is defined by a significant drop in accuracy when linking evidence across two hops, underscoring challenges in integrating separate information pieces.
Empirical studies across benchmarks show that failures in retrieval, latent composition, and sequential processing contribute to the two-hop reasoning curse.

The Curse of Two-Hop Reasoning denotes a recurring failure pattern in which systems that appear competent at one-hop retrieval or reading degrade sharply when required to compose exactly two linked pieces of evidence. In its narrowest and most explicit usage, the term refers to the surprisingly small improvement that strong multi-hop readers obtain when they are given the true two-hop chain rather than only the final answer-bearing passage. In broader usage, it covers adjacent observations: steep accuracy drops from one hop to two hops in long-context QA, weak latent composition of separately learned facts without chain-of-thought, and internal transformer dynamics in which the second hop begins too late or remains mechanistically unstable (Wang et al., 2019, Gupta et al., 20 May 2025, Biran et al., 2024).

1. Origins, scope, and formal definitions

The phrase was articulated most directly in work on HotpotQA readers. There, a two-hop instance consists of a question $q$ , two supporting passages $p_1$ and $p_2$ , and a ground-truth answer span $a$ in $p_2$ . Two-hop reasoning is defined as a reader computing

$\hat a = f_{\text{two-hop}}(H^q, H^{p_1}, H^{p_2}),$

with the requirement that this cannot be reduced to $f_{\text{1-hop}}(H^q, H^{p_2})$ without loss of accuracy. The same study compares Single-Oracle, Ordered-Oracle, and Random settings, thereby operationalizing how much benefit existing readers actually obtain from access to the full reasoning chain (Wang et al., 2019).

A second formalization treats the phenomenon as a drop from one-hop to two-hop performance. In long-context narrative QA, hop depth is written as $h \in \{1,2,3,4\}$ , context length as $L \in \{64k,96k,128k\}$ , and exact-match accuracy as $A(h,L)$ . The two-hop curse is then

$p_1$ 0

which isolates the loss incurred at the second reasoning jump (Gupta et al., 20 May 2025).

Mechanistic studies define two-hop composition in fact-graph terms. One formulation uses a two-hop expression $p_1$ 1 and its one-hop counterpart $p_1$ 2, where $p_1$ 3 and $p_1$ 4 are Wikidata facts and $p_1$ 5 is the bridge entity. The key question is whether a model internally recalls $p_1$ 6 and then uses it to recover $p_1$ 7, as measured through an Internal Entity Recall Score (EntRec) and a Consistency Score (CnstScore) (Yang et al., 2024).

At a more abstract level, multi-hop degradation was anticipated by a formal framework that distinguishes a hidden conceptual meaning space from a noisy linguistic space. In that account, ambiguity, redundancy, incompleteness, and inaccuracy are introduced by language itself, and even simplified graph-connectivity reasoning becomes unreliable once hop count grows. The result that a moderately large logarithmic number of hops in the meaning graph is already problematic suggests that the two-hop case is best viewed as the smallest visible instance of a broader representational barrier (Khashabi et al., 2019).

2. Empirical signatures across QA benchmarks

The canonical empirical signature is that giving a model the correct two-hop chain often helps far less than expected. On HotpotQA, HotpotReader improved from EM=55.07, F1=70.00 in Single-Oracle to EM=55.17, F1=70.75 in Ordered-Oracle. BERT improved from EM=64.08, F1=77.86 to EM=65.03, F1=79.15, which the study reports as the highest error reduction of 5.8% on F1 (corresponding to 1.3% absolute improvement). This is the empirical core of the original “curse”: the gold reasoning chain is present, yet standard readers extract only limited benefit from it (Wang et al., 2019).

Long-context narrative QA exhibits the same pattern in a different regime. Across seven state-of-the-art models on NovelHopQA, average exact-match accuracy at 64k tokens drops from $p_1$ 8 to $p_1$ 9, giving $p_2$ 0 percentage points. At 128k tokens it drops from $p_2$ 1 to $p_2$ 2, giving $p_2$ 3 percentage points. The reported line plots show that the relative decline from one hop to two hops is consistently the steepest across models and context lengths (Gupta et al., 20 May 2025).

Adversarial evaluation yields a third signature: performance is much more sensitive to misleading reasoning paths than to simple lexical traps. In a HotpotQA-derived benchmark with 666 adversarial examples, F1 drops from 77.2 to 52.7 for GPT-3.5, from 68.1 to 48.4 for Mixtral-8×7B, and from 82.1 to 62.2 for a fine-tuned Longformer. The study reports up to 45% relative decrease in F1 under plausible distractor chains, indicating that many systems still fail when a false but coherent two-hop alternative is available (Bhuiya et al., 2024).

Setting	Representative measurement	Observation
HotpotQA Ordered-Oracle	BERT F1: 77.86 $p_2$ 4 79.15	Gold chain gives limited gain
NovelHopQA 64k	86.35% $p_2$ 5 79.54%	Large one-hop to two-hop drop
Adversarial distractors	GPT-3.5 F1: 77.2 $p_2$ 6 52.7	Plausible false chains are damaging

These findings are mutually reinforcing rather than redundant. One measures failure to exploit explicit evidence, another measures the raw cost of adding a second hop, and the third measures susceptibility to spurious two-hop structure.

3. Latent composition and internal transformer mechanisms

Mechanistic work on factual composition suggests that the second hop is not merely harder in the aggregate; it is also implemented differently inside the model. One study of prompts such as “The spouse of the performer of Imagine is” reports that the bridge entity is resolved in early layers and that the final answer is resolved only later. In correct cases, the earliest bridge-decoding layer is approximately $p_2$ 7– $p_2$ 8, while the final answer appears at $p_2$ 9– $a$ 0. The same work introduces back-patching, in which a hidden representation from a later layer is patched back to an earlier layer, and finds that in up to 66% of previously incorrect cases there exists a back-patch that results in correct generation. The interpretation given there is that later layers sometimes no longer have the needed extraction functionality once the second hop begins (Biran et al., 2024).

A related probe-based study reaches a more qualified conclusion. Using EntRec for the first hop and CnstScore for the second hop, it finds strong evidence for the first hop but only moderate evidence for the second. At peak effect, the first-hop score reaches 0.71, 0.72, 0.78 under entity substitution for 7B, 13B, 70B, and 0.63, 0.64, 0.76 under relation substitution. By contrast, the second-hop causal consistency increase peaks at 0.64, 0.65, 0.61, and the full two-hop SS fraction peaks only at 0.46, 0.44, 0.43 for entity substitution and 0.38, 0.40, 0.43 for relation substitution. The scaling trend is clear for the first hop but not for the second (Yang et al., 2024).

Synthetic in-context reasoning experiments expose a still sharper transition. On a symbolic two-hop task with distractors, Llama2-7B-Base assigns next-token probability $a$ 1 to the correct end token when $a$ 2, but when $a$ 3 the target probability drops to $a$ 4 and each distractor receives $a$ 5, effectively uniform guessing among plausible conclusions. After 1,000 steps of fine-tuning on $a$ 6, the same model reaches near-perfect accuracy and generalizes to $a$ 7 or $a$ 8. A separately trained 3-layer, single-head transformer reveals a phase transition from random guessing to a structured sequential query mechanism, in which one layer performs the first-hop lookup and the next performs the second-hop lookup (Guo et al., 19 Feb 2025).

Taken together, these studies suggest that two-hop failure is often a problem of composition timing and circuit availability rather than a simple absence of stored facts. The bridge may be partially recalled, decodable only transiently, or never stabilized in a form that the next stage can use.

4. Retrieval, evidence grounding, and dataset construction

Another line of work locates the curse in retrieval and supervision rather than solely in latent computation. Baleen frames multi-hop reasoning as a retrieval problem with a fast-growing search space across the hops, composite information needs, and ambiguity in hop order. To address these issues it introduces condensed retrieval, a focused late interaction retriever (FLIPR), and latent hop ordering (LHO). On HotPotQA full-wiki retrieval, Baleen reports P-EM 86.7, P-R@20 93.3, and Ans-R@20 96.3, compared with MDR at 81.2, 82.9, and 89.4. In this formulation, the curse is partly the result of combinatorial search and cascading retrieval errors, not only answer extraction (Khattab et al., 2021).

Dataset design can either hide or expose the problem. 2WikiMultiHopQA was built precisely because many existing datasets allowed shallow shortcuts. It augments answer spans and supporting facts with explicit evidence triples and a reasoning path, and defines a two-hop path by

$a$ 9

The reported numbers make the benchmarking point clear: a strong multi-hop model has Answer F1 58.54 on HotpotQA but 40.95 on 2WikiMultiHopQA, while a single-hop BERT baseline falls from 64.6 F1 on HotpotQA to 55.9 F1 on 2WikiMultiHopQA. This indicates that explicit enforcement of two-hop structure materially changes model difficulty (Ho et al., 2020).

A complementary response is to force the model to externalize intermediate reasoning. Locate Then Ask introduces an interpretable stepwise framework in which each hop identifies supporting sentences, generates a single-hop sub-question, answers that sub-question, and feeds the result into the next hop. On HotpotQA, StepReasoner reports Ans EM 69.66, Ans F1 82.42, Joint EM 47.84, Joint F1 74.27, compared with HGN-large at 69.22, 82.19, 47.11, 74.21. On 2WikiMultiHopQA, it reports Ans EM 80.88, Ans F1 84.86, compared with an ELECTRA-large baseline at 79.22, 83.51. The gains are not enormous, but the framework is designed to reduce hallucinated decompositions by grounding each intermediate step in selected supports (Wang et al., 2022).

This body of work implies that two-hop difficulty is often entangled with evidence exposure. If retrieval, chain ordering, or grounding is weak, the second hop inherits corrupted state before answer generation even begins.

5. Failure modes and explanatory accounts

The original HotpotQA analysis attributes the small oracle-chain gains to multiple causes. First, ~80% of HotpotQA questions can be answered from a single passage or via surface cues, so the extra passage often adds little signal. Second, existing readers, including BERT, tend to process $p_2$ 0 as one long text rather than as an ordered inference chain; the reported binary ordering probe reaches only ~87.4% accuracy. Third, the heuristically recovered chains are noisy. Fourth, simply increasing capacity does not solve the problem: duplicating $p_2$ 1 does not produce the gains that an actual reasoning improvement would (Wang et al., 2019).

Long-context narrative QA supplies a more fine-grained taxonomy. On two-hop questions, NovelHopQA identifies Retrieval-Miss on Intermediate Hop, Integration Error at Final Hop, Incomplete Evidence Combination, and Contextual Drift. The accompanying interpretation is that tens of thousands of tokens bury intermediate clues, effective attention remains limited despite large windows, and an error on hop 1 propagates deterministically into hop 2. The benchmark’s oracle-context filtering is meant to ensure that questions are genuinely answerable in principle, which sharpens the conclusion that many failures are reasoning failures rather than pure data noise (Gupta et al., 20 May 2025).

A separate human-annotated error analysis emphasizes faithfulness of the reasoning path itself. On true two-hop items, the average distribution is 58% Fully Correct Hops, 12% Partially Correct Hops, 8% Underhopping, 15% Early Irrelevance, 5% Trailing Irrelevance, and 2% Question Misinterpretation. The same study defines Hop Precision, Hop Recall, Hop F1, Hop Diversity, and Hop Uniqueness, and reports that overthinking on two-hop HotpotQA averages 24%–41% across models. When overthinking occurs, the chance of a wrong final answer rises from ~15% to ~50%. This reframes the curse: models can fail not only by missing a required hop, but also by introducing unnecessary ones (Yadav et al., 6 Aug 2025).

Adversarial distractor work adds an important nuance. LLMs appear comparatively robust to simple lexical-overlap attacks, but they are strongly affected by misleading reasoning paths and by distractors that form a coherent alternate chain. This suggests that the main weakness is not crude bag-of-words distraction; it is structural confusion between nearby multi-sentence causal pathways (Bhuiya et al., 2024).

A broader explanation comes from the meaning-space versus linguistic-space framework. If the linguistic graph is a noisy grounding of a hidden conceptual graph, then ambiguity and incompleteness are not accidental perturbations but endemic features of the representational medium. This suggests that even two-hop failures can arise before any “reasoning module” is invoked, because the bridge entity or relation may already be distorted at the language layer (Khashabi et al., 2019).

6. Debate, reinterpretation, and prospective remedies

A strong 2024 claim stated that LLMs trained on A $p_2$ 2B and B $p_2$ 3C fail to learn A $p_2$ 4C without chain-of-thought when the one-hop facts occur only in different documents, achieving chance-level accuracy and chance-level test loss on a synthetic setting. That paper also reported near-zero no-CoT accuracy on over half of real-world question categories. However, the version publicly available carries a prominent notice: “Recent research contradicts some key claims; we are working on a major revision with more nuanced analysis. Please wait for the updated version.” For that reason, its strongest formulation of a universal two-hop impossibility is best treated as a contested result rather than settled consensus (Balesni et al., 2024).

Subsequent work already points to a more conditional picture. One controlled study shows that GPT2-style LLMs trained from scratch can learn implicit $p_2$ 5-hop reasoning without chain-of-thought, but the required training data grows exponentially in $p_2$ 6, the required number of transformer layers grows linearly in $p_2$ 7, and curriculum learning can mitigate but not eliminate the data burden. Another study shows that pre-trained LLMs may fail on synthetic two-hop tasks with distractors yet recover after a small amount of fine-tuning, achieving near-perfect accuracy and strong length generalization. These results do not remove the curse; they relocate it from absolute impossibility to an adverse scaling law and an unfavorable inductive bias (Yao et al., 23 May 2025, Guo et al., 19 Feb 2025).

Mechanistic interpretation is also under revision. The hop-aligned circuit hypothesis—according to which bridge entities must become decodable before final answers—has been challenged by layer-order inversion, where later-hop answer entities can become decodable earlier than bridge entities. For the two-hop subset, the same study reports no wholesale inversion, but it argues for a probabilistic recall-and-extract account in which shallow MLP layers broadly recall candidate entities and deeper attention layers selectively extract the final answer. This reinterpretation explains why chain-of-thought can help: forcing the bridge entity into the context delays premature answer extraction and restores a more sequential computation (Liu et al., 7 Jan 2026).

The remedial proposals in the literature are correspondingly diverse. They include co-matching and other explicit chain-aware readers, condensed retrieval, focused late interaction, latent hop ordering, retrieval-conditioned generation, iterative chunk expansion, memory augmentation, hop-level supervision, and explicit penalties for overthinking (Wang et al., 2019, Khattab et al., 2021, Gupta et al., 20 May 2025, Yadav et al., 6 Aug 2025). At the most abstract level, the recommendation is to improve alignment between the hidden conceptual space and the linguistic space through richer representations before investing in many-hop reasoning. The central implication is that the second hop is not merely “one more step”: it is often the point where retrieval, representation, and computation cease to compose reliably (Khashabi et al., 2019).