FiD-LightSP: Source-Pointer Re-Ranking

• The paper introduces explicit source-pointer mechanisms to re-rank retrieved passages, enhancing both answer generation and provenance retrieval.
• It employs compressed passage representations with end-to-end sequence-to-sequence training, reducing computation by over 2× while preserving high recall.
• Empirical results on the KILT benchmark show state-of-the-art performance across diverse tasks, affirming both efficiency and effective multi-hop provenance alignment.

Source-Pointer Re-Ranking (FiD-Light$^{SP}$) is an extension of the FiD-Light retrieval-augmented text generation model that introduces explicit source pointer mechanisms for re-ranking retrieved passages, yielding substantial gains in combined answer generation and provenance retrieval, particularly under strict latency and efficiency constraints. The method leverages compressed passage representations and end-to-end sequence-to-sequence training to achieve state-of-the-art results on the KILT benchmark with near-zero added computational cost (Hofstätter et al., 2022).

1. FiD-Light Model Architecture and Data Flow

FiD-Light is based on the Fusion-in-Decoder (FiD) encoder–decoder paradigm, which encodes each retrieved passage independently using a T5-style Transformer encoder, denoted as $T_E$. The encoder input for passage $i$ is $$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$, where $q$ is the query, $p_i$ the passage, and $e_i \in \mathbb{R}^{(|q|+|p|)\times D}$ with $D$ as the hidden size. In the vanilla FiD model, these per-passage encodings are fully concatenated, yielding a decoder input $o = T_D([e_1; e_2; \ldots; e_n])$ for $n$ passages, resulting in input sequences of up to $T_E$0–$T_E$1 vectors.

FiD-Light replaces this full concatenation with a lossy compression function $T_E$2 for each passage, where $T_E$3. The decoder input becomes $T_E$4. The typical instantiation is “take the first $T_E$5 vectors of $T_E$6,” though learned pooling or attention over passage encodings is plausible.

2. Source Pointer Mechanism

FiD-Light$T_E$7 (“FiD-Light with Source Pointers”) incorporates pointer tokens as re-ranking cues, extending the input format as $T_E$8 to mark each passage. At training, the target output concatenates all relevant passage indices (in arbitrary order) with the answer: $T_E$9, with $i$0 the gold-relevant indices and $i$1 the answer string.

At inference, the decoder freely emits index tokens, defining a pointer set $i$2. These indices serve not as explanations but as actionable signals for document re-ranking.

3. Passage Re-Ranking via Decoded Pointer Sets

To robustly select top provenance items, FiD-Light$i$3 leverages the generated pointer set to reorder the original retrieval list $i$4. The re-ranked list is

$i$5

where $i$6 appear in the output. Alternately, each pointer can be viewed as casting a "vote" via $i$7, with $i$8 the pointer token positions. Passages are ranked by $i$9 as an alternate implementation.

Unlike previous FiD-Ex approaches, which directly generate indices, FiD-Light$$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$0 mitigates the problem of raw index under-generation, especially when the gold set size $$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$1 (as in multi-hop questions).

4. End-to-End Joint Pointer and Text Training Objective

End-to-end fine-tuning is performed on the full encoder–decoder stack with a single cross-entropy loss over the concatenated index and answer token output:

$$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$2

In practice, indices are packed as ordinary tokens prior to answer tokens, using a standard sequence-to-sequence cross-entropy formulation without auxiliary weighting. The model learns to emit correct indices before switching to answer generation.

5. Computational Efficiency and Efficacy Trade-Offs

FiD-Light$$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$3 achieves significant efficiency improvements due to passage compression. With $$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$4, decoder compute for 40 passages drops by approximately $$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$5, and the total query latency is reduced by over $$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$6 (under T5-Base), even while all 40 passages are still retrieved, hence maintaining high recall. This design allows leveraging larger T5 backbones (T5-Large, T5-XL) for further quality improvements at the same or reduced latency relative to FiD-Base.

Table: Decoder Input Scale

Model	Passages ($$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$7)	Passage Compression ($$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$8)	Relative Decoder Cost
FiD-Base	40	$$e_i = T_E([\text{“query:”}; q; \text{“context:”}; p_i])$$9 (~100–250)	1.0
FiD-Light$q$0	40	8/32/64	$q$10.17–0.66

All numbers refer to relative decoder input size per query as derived from model configurations.

6. Empirical Results on KILT Benchmark Tasks

FiD-Light$q$2 was evaluated on seven knowledge-intensive tasks (HotpotQA, NQ, TriviaQA, FEVER, T-REx, zsRE, WOW) in the KILT benchmark. Representative combined KILT-score results (text score × R-Precision) demonstrate:

• T5-Base, $q$3: NQ: 45.6 K-EM, HotpotQA: 25.6 K-EM, TriviaQA: 57.6 K-EM, FEVER: 80.6 K-AC, T-REx: 76.0 K-AC, zsRE: 81.1 K-AC
• T5-Large, $q$4: NQ: 49.9, HotpotQA: 28.2, TriviaQA: 61.4, FEVER: 82.1, T-REx: 76.7, zsRE: 84.1
• T5-XL, $q$5: NQ: 51.1, HotpotQA: 29.2, TriviaQA: 63.7, FEVER: 84.5, T-REx: 76.3, zsRE: 84.0

Parentheses in the original data denote gains over the prior state of the art. FiD-Light$q$6 sets new state-of-the-art R-Precision for nearly all tasks, attributed specifically to its pointer-based re-ranking regime.

Ablation studies reveal that direct index decoding in FiD-Ex frequently under-selects relevant indices, especially when $q$7 (multi-hop cases). For instance, T5-Base HotpotQA doc R-Precision increases from 23.7% to 24.1%, TriviaQA from 32.1% to 37.6%, and FEVER from 77.1% to 78.1% when switching from FiD-Ex to FiD-Light$q$8 re-ranking. Pareto-frontier analysis on latency vs. KILT-score shows FiD-Light$q$9 variants strictly dominating the FiD baseline: equal or better quality at 2–3$p_i$0 lower inference cost or much higher effectiveness at iso-cost.

7. Practical and Methodological Significance

The Source-Pointer Re-Ranking mechanism transforms FiD-Light into a dual-purpose generator and re-ranker. This approach introduces virtually no additional computation—merely a minimal increase in target tokens—while delivering robust, multi-passage provenance pointing and integrating seamlessly into the compressed-encoding architecture. Its design addresses several challenges inherent to retrieval-augmented generation, such as efficient handling of long input sequences, joint answer and provenance evaluation, and robust multi-hop provenance under variable $p_i$1. FiD-Light$p_i$2 thus provides an effective, efficient unified solution for high-precision retrieval-augmented generation tasks (Hofstätter et al., 2022).