Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval

Abstract: Generative retrieval (GR) ranks documents by autoregressively generating document identifiers. Because many GR methods rely on trie-constrained beam search, they are vulnerable to early pruning of relevant prefixes under finite-beam decoding. Planning Ahead in Generative Retrieval (PAG) mitigates this failure mode by using simultaneous decoding to compute a document-level look-ahead prior that guides subsequent sequential decoding. We reproduce PAG at inference time and stress-test its decoding behavior. Using the authors' released checkpoint and identifier/trie artifacts under the reported decoding setup, we reproduce the main effectiveness results on MS MARCO Dev and TREC-DL 2019/2020, and corroborate the reported beam-size-latency trade-off in our hardware setting. Beyond reproduction, we introduce plan drift diagnostics that quantify how intent-preserving query variations alter the planner's top-n candidate set and highest-weight planner tokens, and how these changes affect guided decoding. We find that PAG's planning signal is brittle under lexical surface-form variation: intent-preserving typos can trigger plan collapse, where the planned candidate pool shifts enough that the look-ahead bonus provides little useful guidance, effectively reverting decoding toward weaker unguided search. We further evaluate fixed-index cross-lingual robustness using non-English mMARCO queries against an English index, and assess query-side mitigation strategies that require no re-indexing; query translation provides the strongest recovery in our setting. Overall, our results confirm PAG's reported effectiveness and the benefit of planning-guided decoding under the released inference setup, while showing that these gains depend on the stability of the planning signal under realistic query variation and query-document mismatch.

• The paper rigorously reproduces headline results for planning-guided decoding (PAG) in generative retrieval, confirming effectiveness and efficiency under ideal conditions.
• Robustness tests show PAG is highly sensitive to surface-level query variations (misspellings, synonyms), suffering frequent plan drift and collapse with up to 12% severe performance drops on TREC-DL benchmarks.
• Cross-lingual evaluation reveals planning guidance fails for non-English queries unless translation is used, underscoring the need for semantically anchored or multilingual planner identifiers for real-world deployment.

Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval

Introduction and Background

This paper presents a rigorous reproducibility and robustness evaluation of the Planning Ahead in Generative Retrieval (PAG) method—a two-stage decoding approach for generative retrieval (GR) models that aims to mitigate early prefix pruning under trie-constrained beam search by leveraging a planning-derived look-ahead prior. The authors replicate claims from the original PAG paper regarding effectiveness and efficiency on standard retrieval benchmarks, but further interrogate the planner's reliability under lexically and linguistically perturbed queries, and under cross-lingual scenarios with a fixed English document index.

The central focus is on the stability and brittleness of the planning stage's guidance signal. The paper introduces a suite of diagnostics—plan drift, candidate and token overlap metrics, and plan collapse rates—that expose the extent to which plausible query variations can disrupt guidance, lead to suboptimal or failed decoding, and hence degrade document retrieval quality.

Planning-Guided Decoding and the PAG Pipeline

PAG addresses the critical failure mode in GR—where relevant docid prefixes are pruned early due to finite-beam constraints—by first ranking documents via a fast set-based "planning" pass and then incorporating the resulting scores as a bonus in trie-constrained beam search over sequential docid tokens. Each document is assigned both an ordered sequential identifier and an unordered set-based identifier. The planning stage produces query-dependent token weights over the planning vocabulary, scores documents by token aggregation, constructs a top-n candidate set, and computes a look-ahead bonus that augments sequential prefix scores only if the prefix is supported by a planned candidate.

Figure 1: The compact PAG pipeline, showing simultaneous decoding for planning and augmented trie-constrained decoding; candidate and token drift diagnostics are introduced at red markers.

The pipeline relies critically on the overlap between planned candidates or planner tokens for clean versus perturbed queries. Instability in this set causes the look-ahead bonus to lose its discriminative efficacy, reverting decoding towards standard beam search as candidate coverage dissipates.

Reproducibility and Effectiveness Evaluation

The authors execute a systematic artifact-based reproducibility study: using the official checkpoint, document identifiers, and trie index, they replicate all headline effectiveness results on MS MARCO Dev and TREC DL 2019/2020, and dissect the impact of planner set size and beam width on both MRR@10 and Recall@10, as well as query latency. Small variations in effectiveness under altered configurations are consistent with the expected trends, confirming PAG's efficacy for strong retrieval at modest computational cost, so long as the planner signal is compatible with the query.

Query Variation Robustness: Drift, Collapse, and Sequence Gain

To interrogate planner robustness, the authors generate five classes of intent-preserving perturbations for each query: misspellings, reordering, synonym substitution, paraphrase, and "naturality" drift. They track:

• CandOverlap@100: Fractional overlap of the top-100 planning candidate sets between clean and perturbed queries.
• TokJaccard@100: Jaccard similarity of the top-100 weighted planner tokens between clean and perturbed queries.

For misspellings and synonymization, both overlap metrics exhibit sharp declines (often $<0.4$), correlating with decreased retrieval performance in both the planning and full PAG stages.

Figure 2: Distribution of candidate set overlap (CandOverlap@100) across splits and variation types, with heavy lower-tail drift for surface-variant queries.

Figure 3: Token-level Jaccard similarity (TokJaccard@100) between planner vocabularies, illustrating planner token drift on perturbed queries.

Decomposing performance drops, the authors introduce plan collapse events—tail queries with both low planner stability and large drops (>0.05) in planning-only retrieval metric. Collapse appears in 10–12% of TREC-DL queries under lexical corruption, corresponding to look-ahead bonuses that are nearly useless and decoding that must proceed unguided.

Crucially, plan swap diagnostics show that swapping the perturbed plan for the clean plan can partially recover effectiveness, but full robustness is unattainable when the planner's signal becomes misaligned with the evidence present in the index.

Comparative Robustness With Dense and GR Baselines

The paper extends robustness analysis to recent dense retrievers and a strong GR baseline (RIPOR). While all systems show some degradation under query perturbation, recent dense retrievers (e.g., Qwen-8B, TAS-B) are notably less sensitive to misspellings and synonym substitutions than both PAG and RIPOR. This suggests that lexically anchored guidance signals, as in current implementation of PAG, disproportionately amplify retrieval failures under surface-form variation.

Figure 4: Mean relative drop in retrieval effectiveness under five query variation types for PAG, dense, and strong GR baselines.

Cross-Lingual Query Shift With a Fixed Index

A further stress-test issues non-English mMARCO queries—Dutch, French, German, Chinese—against the fixed, English-indexed corpus. In this configuration, naive application of the English-trained planner to non-English queries yields severely degraded retrieval. Query translation (using M2M100) restores most performance, significantly increasing both token and candidate set overlaps with the English references, while a light-weight planner-alignment approach (without corpus-side re-indexing) achieves only partial gains.

Figure 5: Cross-lingual query performance for naive, translation-based, and planner-aligned mitigations, highlighting translation as the only effective approach without re-indexing.

Token-level adaptation mildly improves overlap for some European languages but fails for language pairs with minimal vocabulary overlap (e.g., Chinese-English), and does not robustly transfer to candidate set coverage.

Theoretical and Practical Implications

The findings present important caveats for planning-guided generative retrieval. While the look-ahead strategy provides measurable and reproducible benefits in ideal settings, its practical utility hinges on the planner's surface-form robustness. Lexical or linguistic domain shifts—whether from organic user behavior, adversarial perturbations, or cross-lingual deployment—can rapidly erode guidance quality, reintroducing the original failure modes of GR under finite-beam decoding. The tight coupling of the bonus signal to the surface form of queries and the identifier assignment mechanism is exposed as a major bottleneck for real-world robustness, generalization, and multilingual scalability.

The superiority of simple query translation over token-level alignment as a mitigation, under fixed corpus-side artifacts, highlights the inadequacy of current planning-vocabulary mappings across linguistic boundaries and the need for more semantically anchored, cross-lingual or multilingual identifier schemes [zhang2024multilingual].

Future Directions

The results underscore the need for:

• Planner/token identifier strategies less sensitive to surface variation (e.g., semantic, multilingual, or permutation-invariant encodings).
• Diagnostics (e.g., plan drift, collapse rates) as standard components of robustness reporting in generative IR.
• Jointly optimized planners and decoders with explicit objectives for cross-domain and cross-lingual generalization.
• More systematic evaluations of robustness under both query and corpus-side shifts, especially for deployment in dynamic, user-facing systems [kim-etal-2024-exploring-practicality, chen2023continual].

Conclusion

This reproducibility and stress-test study validates that planning-guided decoding in GR (PAG) is both effective and efficient when planner signals are query-compatible. However, the look-ahead mechanism is acutely vulnerable to surface-form shifts, leading to frequent plan drift, collapse, and drastic performance drops. Translation is the only effective query-side mitigation under fixed-index constraints. Future research must directly target planner robustness, develop semantically stable guidance mechanisms, and routinely instrument plan drift diagnostics to ensure that the promise of planning-guided generative retrieval extends to realistic, noisy, or multilingual environments.

---

References:

• "Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval" (2604.23396)
• "Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding" [Zeng2024PlanningAI]
• "Multilingual Generative Retrieval via Cross-lingual Semantic Compression" [zhang2024multilingual]
• "Continual Learning for Generative Retrieval over Dynamic Corpora" [chen2023continual]
• "On the Robustness of Generative Information Retrieval Models: An Out-of-Distribution Perspective" [liu2025robustness]
• "Exploring the Practicality of Generative Retrieval on Dynamic Corpora" [kim-etal-2024-exploring-practicality]