ThinkGR: Unified Framework for Retrieval

• ThinkGR is a unified framework for generative retrieval, interleaving reasoning with document identifier generation to address multi-hop semantic gaps.
• It uses a hybrid decoding strategy that switches between free-form chain-of-thought and constrained docid generation to ensure iterative, valid retrieval steps.
• The two-phase training combining supervised fine-tuning and retrieval-grounded reinforcement learning yields improved recall and efficiency on multi-hop benchmarks.

ThinkGR is a unified framework for generative retrieval that interleaves chain-of-thought with document-identifier generation, enabling iterative thinking and retrieval within a single autoregressive process. It was introduced as a preliminary study on retrieval tasks in which a direct mapping from query to docid is structurally weak, especially multi-hop retrieval with large semantic gaps between the original query and the target supporting documents. The framework combines hybrid decoding, which alternates between free-form reasoning and constrained docid generation, with a two-phase training procedure consisting of supervised fine-tuning and retrieval-grounded reinforcement learning. On four multi-hop retrieval benchmarks, ThinkGR reports an average Recall of 78.44 and an average improvement of +6.86 over the strongest baseline reported in the study (Zhang et al., 21 May 2026).

1. Conceptual basis and problem formulation

ThinkGR is situated within generative retrieval (GR), where retrieval is cast as autoregressive sequence generation rather than vector-space scoring. In the standard GR formulation criticized by the paper, the model learns a direct mapping

$\text{query} \rightarrow \text{docid}.$

This formulation is workable when the target document is semantically close to the query surface form, but the paper argues that it is poorly matched to complex retrieval, particularly multi-hop retrieval, where the final target may only be reachable through intermediate semantic links (Zhang et al., 21 May 2026).

The motivating claim is structural rather than merely empirical. A one-step GR model must jump directly from the original query to the final target docid, even when the supporting path requires several latent subproblems. The paper characterizes this as a semantic gap between query and target. Its illustrative example—“What is the capital of the country where the director of Erta Ale was born?”—requires identifying the director, then the associated country, and only then the capital. ThinkGR treats this missing intermediate deliberation as the central failure mode of standard GR for multi-hop retrieval.

The framework therefore imports chain-of-thought (CoT) into retrieval. In ThinkGR, thought tokens specify what intermediate fact or relation is needed next, and docid tokens execute retrieval actions. This yields a mixed autoregressive trajectory of reasoning segments and retrieval steps, rather than a single-step docid output. In the broader GR literature, this differs from zero-shot instruction-driven indexing frameworks such as ZeroGR, which extend GR across heterogeneous corpora and IR tasks through natural-language instructions but do not introduce explicit intermediate deliberation in the retrieval trajectory (Sun et al., 12 Oct 2025).

2. Sequence structure, triple docids, and hybrid decoding

At training time, ThinkGR represents the target output as an interleaved sequence

$y = (r_1, d_1, r_2, d_2, \ldots),$

where $r_i$ denotes a thought segment and $d_i$ denotes a docid. The operational claim is that retrieval should proceed through iterative thinking and retrieval: generate a thought about what is needed next, generate a valid document identifier for the corresponding retrieval action, continue reasoning, and repeat until the retrieval chain is complete (Zhang et al., 21 May 2026).

The framework is implemented by fine-tuning Llama-3.1-8B-Instruct as the main backbone. Its key representational choice is the use of semantic triples as document identifiers: $$(\text{head entity}, \text{relation}, \text{tail entity}).$$ Documents are transformed into strings of the form:

$r_i$1

The stated rationale is twofold. First, triples expose relational structure that aligns naturally with multi-hop traversal. Second, triples are natural-language-like and therefore easier for a pretrained LLM to generate than opaque symbolic ids. The paper reports that triple collisions are low, with the proportion of triples appearing in at least 2 documents below 3% on all datasets, and that using triples instead of full documents causes negligible retrieval degradation for BGE-large, with slight improvement on some datasets.

The central decoding mechanism is a hybrid decoding strategy. ThinkGR begins in unconstrained decoding mode for free-form thought generation. When the model emits <docid_start>, decoding switches into constrained docid mode. In that mode, valid continuations are enforced using an FM-index following Bevilacqua et al. (2022), which stores all valid triple strings in the corpus and returns the set of valid next tokens consistent with the current prefix in constant time. When the model emits <docid_end>, decoding returns to unconstrained thought generation. This dynamic switching is the paper’s explicit solution to the incompatibility between free-form CoT and validity-constrained retrieval targets.

3. Two-phase training and retrieval-grounded reinforcement learning

ThinkGR uses a two-phase training strategy. The first phase is supervised fine-tuning (SFT), whose purpose is to teach the model the structural pattern of interleaving thought and retrieval. Given an input query $x$, the SFT objective is

$$L_{\text{SFT}} = -\sum_t \log P(y_t \mid y_{<t}, x). \tag{1}$$

Here $y$ is the mixed reasoning-and-docid sequence. The paper argues that this alignment step is necessary because reinforcement learning alone does not teach the model how to emit retrieval delimiters, how to structure interleaved trajectories, or how to reason in a retrieval-oriented way (Zhang et al., 21 May 2026).

The second phase is retrieval-grounded reinforcement learning. Rather than rewarding specific wording of thought segments, ThinkGR treats thought quality as grounded in retrieval outcomes. Retrieval accuracy over generated docids is defined as

$$\text{Acc}_r(y) = \frac{|\text{docids}(y) \cap \text{docids}_{gr}|}{|\text{docids}_{gr}|}. \tag{4}$$

Using this quantity, generated responses are partitioned into desirable and undesirable sets: $$D_{\text{desirable}} = \{(x, y) \mid \text{Acc}_r(y) = 1\}, \tag{2}$$

$y = (r_1, d_1, r_2, d_2, \ldots),$0

with $y = (r_1, d_1, r_2, d_2, \ldots),$1 in the main experiments.

The RL algorithm is Kahneman-Tversky Optimization (KTO) rather than PPO or DPO. The paper’s reason is that KTO can operate directly with binary feedback instead of preference pairs. It uses a prospect-theoretic utility relative to a reference point $y = (r_1, d_1, r_2, d_2, \ldots),$2, with $y = (r_1, d_1, r_2, d_2, \ldots),$3 and $y = (r_1, d_1, r_2, d_2, \ldots),$4 adjusted for data imbalance, $y = (r_1, d_1, r_2, d_2, \ldots),$5 as a risk-aversion hyperparameter, and $y = (r_1, d_1, r_2, d_2, \ldots),$6 as the reference SFT model before RL. The retrieved data after filtering comprise 60K desirable responses and 27K undesirable responses. This design suggests that ThinkGR optimizes thought not as an independently supervised text channel, but as a latent control process whose adequacy is measured by whether the generated retrieval chain is correct.

4. Synthetic supervision, benchmarks, and implementation

A substantial part of ThinkGR concerns the construction of supervision for thought-retrieval trajectories. The paper does not rely on human-written CoT traces. Instead, it first converts each document into one or more knowledge triples using Llama-3.1-8B-Instruct. The prompt instructs the model to extract comprehensive triplets directly from the passage, including main clauses, modifiers, prepositional phrases, subordinate clauses, and attributive phrases (Zhang et al., 21 May 2026).

For SFT data construction, the paper then uses Llama-3.3-70B-Instruct to generate thought-retrieval chains from the question and the correct document triples. The prompting instructions require the model to select only the necessary triples, construct a concise reasoning chain, assume it does not know information until the relevant triple is retrieved, possibly flip triple direction so that reasoning proceeds from known to unknown, and keep the chain minimal. The authors describe a rigorous filtering process that removes formatting errors, incorrect triples, and factual inaccuracies. After filtering, the retained SFT corpus contains 228K examples drawn from HotpotQA, 2WikiMultiHopQA, and MuSiQue.

The evaluation uses four multi-hop retrieval benchmarks: HotpotQA, 2WikiMultiHopQA, MuSiQue, and MoreHopQA. MoreHopQA is treated as an out-of-domain test because it has no training set, and its original corpus is merged with the 2WikiMultiHopQA corpus to create a more realistic retrieval setting. The primary metric is Recall, defined as the ratio of correct documents retrieved to total ground-truth documents. The study also evaluates downstream QA using answer accuracy with a Llama-3.3-70B-Instruct reader and a judging prompt.

The comparison set includes standard retrievers (BM25, Contriever, BGE-large, SEAL), LLM-driven multi-step retrieval systems (Self-Ask, IRCoT, ITER-RETGEN, Auto-RAG, R3-RAG, RT-RAG), and reasoning-augmented dense retrieval methods (MDR, GritHopper). The main ThinkGR configuration uses Llama-Factory, full fine-tuning with DeepSpeed ZeRO-3, SFT learning rate 1e-6, SFT batch size 512, cutoff length 2048, warmup ratio 0.05, RL learning rate 4e-7, RL batch size 128, and risk-aversion hyperparameter $y = (r_1, d_1, r_2, d_2, \ldots),$7. Constrained decoding uses an FM-index, and training runs on NVIDIA A800 GPUs. The framework is also evaluated with Llama-3.2-1B, Llama-3.2-3B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, and Qwen3-14B.

5. Empirical performance, ablations, and practical behavior

The main retrieval results are reported as Recall on the four benchmarks (Zhang et al., 21 May 2026).

Dataset	ThinkGR Recall	Strongest baseline in the study
HotpotQA	76.09	91.03
2WikiMultiHopQA	93.19	59.97
MuSiQue	63.98	60.48
MoreHopQA	80.50	74.82
Average	78.44	71.58

The strongest baseline average in the table is GritHopper at 71.58, yielding the reported +6.86 average improvement for ThinkGR. The gains relative to GritHopper are -14.94 on HotpotQA, +33.22 on 2WikiMultiHopQA, +3.50 on MuSiQue, and +5.68 on MoreHopQA. ThinkGR therefore achieves the best reported Recall on 2WikiMultiHopQA, MuSiQue, and MoreHopQA, but not on HotpotQA. The paper attributes this exception to over-specification in HotpotQA, arguing that questions overlap lexically with supporting documents to a degree that allows simpler retrieval methods to succeed without genuine multi-step reasoning.

The downstream QA results track retrieval improvements. Using retrieved documents as RAG context for Llama-3.3-70B-Instruct, ThinkGR reports QA accuracy of 79.99 on HotpotQA, 79.52 on 2WikiMultiHopQA, 57.94 on MuSiQue, and 53.40 on MoreHopQA, with the paper stating that average QA accuracy exceeds the strongest baseline by 6.37%.

The ablations isolate the contributions of the framework’s three main design elements. Removing SFT yields 35.76 / 57.70 / 35.98 / 55.14 across the four datasets, for an average of 46.15, which the paper treats as evidence that RL cannot discover the thought-retrieval structure from scratch. Removing RL yields 67.66 / 92.03 / 53.40 / 73.84, average 71.73, indicating that SFT establishes the trajectory format while RL materially improves retrieval quality. Removing explicit thought yields 69.86 / 93.01 / 53.23 / 78.00, average 73.53. The drop is especially pronounced on MuSiQue, from 63.98 to 53.23, which the authors interpret as evidence that explicit thought is functionally useful, not merely explanatory.

The RL threshold $y = (r_1, d_1, r_2, d_2, \ldots),$8 is examined at $y = (r_1, d_1, r_2, d_2, \ldots),$9, with average Recall 77.38, 77.39, 78.44, and 77.84, respectively. The paper concludes that $r_i$0 gives the best balance. On 2WikiMultiHopQA, ThinkGR retrieves an average of 2.85 documents, compared with 1.79 for GritHopper and 2.44 ground-truth documents on average, which the authors interpret as evidence that ThinkGR better matches the multi-document support structure.

The paper also reports favorable practical behavior. Average latency per query is 1.04 s, compared with 1.17 s for R3-RAG, 9.19 s for IRCoT, 23.47 s for GritHopper, and 59.38 s for RT-RAG. On storage, the ThinkGR index on HotpotQA is 1.94 GB, versus 80 GB for GritHopper, described as 41x smaller. These results support the paper’s claim that a single-pass generative trajectory can be substantially more efficient than multi-call LLM-retriever pipelines while remaining much more capable than dense implicit methods on the targeted multi-hop tasks.

6. Interpretation, limitations, and position within generative retrieval

ThinkGR is explicitly presented as a preliminary study. Its central methodological claim is that retrieval can be reformulated from one-shot docid generation into an interleaved reasoning-and-retrieval generation problem. The paper’s examples make this concrete. For the question “Who is the mother of the director of film Polish-Russian War?”, ThinkGR first retrieves the triple for the film’s director and then retrieves the triple for that person’s mother. In the RL case study, the full model retrieves that Nicole and Natalie has artist Nina Sky and then retrieves that Nina Sky is composed of Nicole Albino and Natalie Albino, whereas the SFT-only variant begins with an irrelevant release-date triple and the trajectory drifts. This suggests that the model’s thought tokens often expose whether retrieval is on track (Zhang et al., 21 May 2026).

The paper does not claim formal faithfulness guarantees for these generated thoughts. It presents them as interpretable intermediate states and provides qualitative examples where they align with successful retrieval chains, but it does not prove that the thoughts are causally faithful rather than correlated with successful decoding. This matters because the framework’s gains are tied to explicit thought generation, yet the epistemic status of those thoughts remains open.

Several limitations are stated directly. The approach relies on standard SFT + KTO rather than retrieval-specific process supervision; future work is suggested in process reward models, more fine-grained thought-quality rewards, and curriculum learning over reasoning complexity. The triple docid design is specialized to multi-hop entity-relation traversal and may be less suitable for one-hop tasks such as Natural Questions. The authors also note that triples mainly capture factual relational structure and may miss broader or non-factual document content. Evaluation is concentrated on multi-hop retrieval benchmarks rather than general retrieval settings.

Additional limitations are implied by the methodology. ThinkGR depends heavily on synthetic thought supervision, though the paper mitigates this with filtering. It also uses a docid representation tailored to semantic triples rather than arbitrary document forms. In relation to adjacent GR work, this locates ThinkGR in a distinct subproblem. ZeroGR addresses instruction-driven zero-shot retrieval over heterogeneous corpora through semantic docids, instruction-conditioned pseudo-queries, and reverse annealing decoding (Sun et al., 12 Oct 2025). By contrast, ThinkGR focuses on the internal generation process for complex queries that require intermediate reasoning. Together, these directions suggest two different expansions of GR: one toward broader task and corpus generalization, and the other toward explicit deliberation for reasoning-intensive retrieval.