Temporal Retrieval Preference Optimization (TRPO)

• The study introduces TRPO as a preference learning formulation that trains retrievers to score temporally aligned documents higher than misaligned ones.
• It integrates contrastive learning with a novel pairwise temporal preference loss derived from reference encoder comparisons.
• Empirical results show significant gains in retrieval accuracy on time-sensitive queries across multi-snapshot corpora.

Searching arXiv for the specified papers to ground the article in current research. Temporal Retrieval Preference Optimization (TRPO) is a preference-learning formulation for temporally aware retrieval in which a model is trained to score a temporally aligned document above a temporally misaligned document for the same query. In its explicit retrieval form, introduced within TPOUR, TRPO adapts Direct Preference Optimization (DPO) from generative alignment to dense retrieval by replacing response log-probabilities with similarity scores and by deriving preferences from versioned corpora rather than human annotations (Kim et al., 16 Jun 2026). In a broader sense, closely related 2025 work on long-form video understanding, video-language alignment, and video diffusion demonstrates the same underlying principle: temporal structure itself can define the preference signal, whether through temporal masking, perturbation, or segment-level alignment (Li et al., 23 Jan 2025, Li et al., 21 Mar 2025, Wu et al., 4 Jun 2025).

1. Definition and retrieval setting

TRPO addresses temporal information retrieval over document collections that evolve across time periods, such as yearly or monthly Wikipedia snapshots. In this setting, each document is associated with a time period, and queries may have either explicit temporal intent, such as “Who is the president in 2019?”, or implicit temporal intent, such as “Who is the current president?” Temporal alignment means that relevant documents should come from the snapshot closest to the query’s target time, not merely from documents that are semantically similar but temporally outdated or premature (Kim et al., 16 Jun 2026).

The central motivation is that standard unsupervised dense retrievers optimize semantic similarity without modeling time. The contrastive objective used by systems such as Contriever treats semantically similar documents from different years as largely interchangeable, which leads to retrieval of correct facts from the wrong year when the answer changes over time. Existing temporal IR methods often depend on supervised timestamped labels, structured temporal knowledge, or explicit query-document-time annotations. TRPO is designed for the gap between these paradigms: it injects temporal awareness into unsupervised dense retrieval using unlabeled, multi-snapshot corpora.

Formally, a training example consists of a query $Q_i$, a temporally aligned document $D_i^t$, and a temporally misaligned document $D_i^{t'}$ with $t' \neq t$. The learning target is a preference relation

$S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$

where $S_\theta$ is the retriever’s similarity function. The preferred pair is determined by snapshot alignment, not by human ranking of outputs.

2. Objective and preference formulation

TRPO is implemented in TPOUR as a supplement to standard contrastive learning rather than as a replacement for it. The base semantic objective is an InfoNCE-style contrastive loss,

$$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$

where $q = \pi_q(Q)$, $d^+ = \pi_k(D^+)$, and $d_i^- = \pi_k(D_i^-)$. This objective preserves semantic retrieval behavior but remains time-unaware (Kim et al., 16 Jun 2026).

TRPO introduces a pairwise temporal preference loss directly inspired by DPO. For a preferred aligned pair $D_i^t$0 and a less preferred misaligned pair $D_i^t$1,

$D_i^t$2

A reference encoder $D_i^t$3 defines analogous scores $D_i^t$4 and $D_i^t$5. The TRPO loss is

$D_i^t$6

This objective compares the current model’s preference gap against the reference model’s preference gap. The term inside the sigmoid,

$D_i^t$7

is positive when the current retriever distinguishes aligned from misaligned documents more strongly than the reference retriever. TRPO is therefore a pairwise logistic preference loss over retrieval scores, with a reference-model regularization analogous to the KL-like role played by $D_i^t$8 in DPO.

The overall TPOUR objective is a weighted combination,

$D_i^t$9

which preserves semantic discrimination while explicitly shaping temporal preference. Appendix ablations reported in the paper indicate that moderate $D_i^{t'}$0 values in the range $D_i^{t'}$1–$D_i^{t'}$2 work best; neither pure contrastive learning nor pure TRPO is optimal.

3. TPOUR architecture and corpus construction

TPOUR uses a BERT-base–style bi-encoder retriever, specifically Contriever with 110M parameters, together with a reference encoder updated by MoCo-style momentum. At inference time, document embeddings are precomputed and indexed, queries are encoded by the same encoder, and retrieval maximizes dense similarity $D_i^{t'}$3 (Kim et al., 16 Jun 2026).

The training corpora are derived from multiple English Wikipedia dumps. The paper reports yearly dumps from 2018-12-20, 2021-12-20, and 2023-12-20, and monthly dumps from 2023-01-01, 2023-07-01, and 2023-12-20. The preprocessing pipeline uses WikiExtractor, filters documents with fewer than 50 words, computes the intersection across dumps, retains only documents whose content changed between snapshots as a “Filtered Intersection,” adds snapshot-specific unique documents, and removes any document that appears as a gold document in SituatedQA or RealTimeQA. Fewer than 2.5% of documents explicitly mention the target year, so the model cannot rely primarily on date strings.

Within each training step, the system samples a query $D_i^{t'}$4, an aligned document $D_i^{t'}$5 from the current snapshot, and a misaligned document $D_i^{t'}$6 from another snapshot. The main encoder computes current scores, the reference encoder supplies queue-based negatives and reference scores, and the model optimizes $D_i^{t'}$7. Reported hyperparameters are a learning rate of $D_i^{t'}$8, queue size $D_i^{t'}$9, loss weight $t' \neq t$0, contrastive temperature $t' \neq t$1, momentum $t' \neq t$2, and $t' \neq t$3 training steps. No language-modeling loss is added.

The theoretical conditions identified for effective TRPO are a real temporal preference margin between aligned and misaligned documents, similar semantic coverage across snapshots so that differences reflect time rather than domain shift, and sufficient encoder capacity to represent both semantics and latent temporal signals.

4. Time vectors, interpolation, and continuous temporal control

A distinctive feature of TPOUR is that TRPO itself does not insert explicit time embeddings into the encoder. Instead, time is encoded in the weights of snapshot-specific retrievers. If $t' \neq t$4 denotes the base Contriever parameters and $t' \neq t$5 the parameters fine-tuned for time period $t' \neq t$6, the paper defines a time vector

$t' \neq t$7

This parameter offset captures how the base retriever must shift to specialize for a given time period (Kim et al., 16 Jun 2026).

Intermediate time periods are obtained by linear interpolation in weight space: $t' \neq t$8 where $t' \neq t$9. The paper presents 2019 as approximately $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$0 between 2018 and 2021. The resulting interpolated retrievers require no additional training and shift retrieval timestamp distributions smoothly from one endpoint to the other. The same framework is extended to extrapolation through

$S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$1

with preliminary evidence that an extrapolated model can outperform the latest single-snapshot model on future queries.

This treatment of time is notable because it relocates temporal control from input annotation to model parameter space. A plausible implication is that TRPO’s temporal alignment signal is sufficiently systematic to produce approximately linear temporal structure in the retriever weights, at least over the snapshot ranges studied.

5. Empirical performance and observed behavior

TPOUR is evaluated on SituatedQA, RealTimeQA, and BEIR, with normalized discounted cumulative gain as the main ranking metric. On mixed-timestamp retrieval for temporal QA, TPOUR-Contriever consistently outperforms unsupervised baselines such as Contriever, SimCSE, and REALM; supervised DPR; temporal-aware baselines including a classical temporal language-model method, Temporal Contrastive, and TimeR$S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$2; and large embedding models including Nomic Embed v2 MoE and Qwen3-Embedding-8B variants (Kim et al., 16 Jun 2026).

The paper reports representative results on SituatedQA. For 2018 explicit queries, Contriever achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$3, Qwen3-Embedding-8B achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$4, Qwen3-Embedding-8B + query rewriting achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$5, and TPOUR-Contriever achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$6. For 2021 explicit queries, Contriever achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$7, Qwen3-Embedding-8B + TAI achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$8, and TPOUR-Contriever achieves $S_\theta(Q_i, D_i^t) > S_\theta(Q_i, D_i^{t'}),$9. For 2018 implicit queries, Contriever achieves $S_\theta$0, Qwen3-Embedding-8B achieves $S_\theta$1, and TPOUR-Contriever achieves $S_\theta$2, a gain of $S_\theta$3 absolute over Contriever. For 2021 implicit queries, Contriever achieves $S_\theta$4, Qwen3 achieves $S_\theta$5, and TPOUR-Contriever achieves $S_\theta$6.

The abstract gives the broad comparison most often cited: compared to Qwen-Embedding-8B, despite being about $S_\theta$7 smaller, TPOUR Contriever improves average nDCG@5 by $S_\theta$8 $S_\theta$9 on explicit and $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$0 $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$1 on implicit queries.

Interpolation results show that intermediate years achieve their best performance at intermediate $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$2 values. Reported average nDCG@5 gains from best interpolation are $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$3 and $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$4 on SituatedQA relative to endpoint models, and $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$5 on RealTimeQA. Retrieval timestamp heatmaps further show that as $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$6 varies from $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$7 to $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$8, the distribution of retrieved document years shifts smoothly from 2018 to 2021.

A separate timestamp-prediction analysis treats temporal encoding as a classification problem. A single-encoder Contriever baseline reaches year accuracy $$\mathcal{L}_{\text{CE}} = -\log \frac{\exp\!\big(S(q,d^+)\big)} {\exp\!\big(S(q,d^+)\big)+\sum_{i=1}^{K}\exp\!\big(S(q,d_i^-)\big)} ,$$9 and month accuracy $q = \pi_q(Q)$0; Nomic Embed v2 MoE reaches $q = \pi_q(Q)$1 year accuracy and $q = \pi_q(Q)$2 month accuracy; a Mixture-of-TPOUR with 10 encoders reaches $q = \pi_q(Q)$3 year accuracy and $q = \pi_q(Q)$4 month accuracy. The paper interprets this as evidence that TRPO-trained retrievers encode strong temporal signals rather than merely memorizing isolated temporal keywords.

On BEIR, different datasets prefer different interpolation weights. Older datasets such as MS MARCO favor more 2018-like interpolations, whereas newer datasets such as TREC-COVID and Climate-FEVER favor more 2021-like interpolations. This indicates latent time sensitivity even in benchmarks not usually framed as temporal retrieval tasks.

6. Relation to temporal preference optimization in video

The term TRPO is introduced explicitly in TPOUR, but related work in video establishes a broader family of temporal preference optimization methods. TPO for long-form video understanding constructs preference tuples $q = \pi_q(Q)$5 by manipulating which temporal portions of a video are visible, using localized temporal grounding for segment-specific evidence and comprehensive temporal grounding for long-range dependencies, then optimizing a DPO + SFT mixed objective. On LongVA-7B, the reported 50/50 mixture of localized and comprehensive data improves Video-MME without subtitles from $q = \pi_q(Q)$6 to $q = \pi_q(Q)$7, LongVideoBench from $q = \pi_q(Q)$8 to $q = \pi_q(Q)$9, and MLVU from $d^+ = \pi_k(D^+)$0 to $d^+ = \pi_k(D^+)$1 (Li et al., 23 Jan 2025).

TEMPLE, titled TEMPO in the paper PDF, applies temporal preference learning to Video LLMs by generating preferred responses from clean videos and dispreferred responses from temporally perturbed videos, specifically via random clip dropping, random clip shuffling, and clip reversal. Its distinctive additions are a difficulty schedule $d^+ = \pi_k(D^+)$2 and “Pre-SFT Alignment,” in which temporal DPO is applied before instruction tuning. For Qwen2-VL-7B, the paper reports gains on VideoMME Temporal Perception from $d^+ = \pi_k(D^+)$3 to $d^+ = \pi_k(D^+)$4, Temporal Reasoning from $d^+ = \pi_k(D^+)$5 to $d^+ = \pi_k(D^+)$6, MLVU from $d^+ = \pi_k(D^+)$7 to $d^+ = \pi_k(D^+)$8, and Vinoground average from $d^+ = \pi_k(D^+)$9 to $d_i^- = \pi_k(D_i^-)$0 (Li et al., 21 Mar 2025).

DenseDPO operates in video diffusion rather than retrieval or Video LLM QA, but it provides a fine-grained temporal preference design. It constructs structurally aligned video pairs by denoising corrupted copies of the same ground-truth video, then assigns segment-level preferences instead of clip-level preferences. The DenseDPO objective aggregates per-segment reward differences inside a single log-sigmoid term. With only one-third of the labeled data, the method improves motion generation over vanilla DPO while matching it in text alignment, visual quality, and temporal consistency, and it supports automatic segment-level annotation by off-the-shelf VLMs (Wu et al., 4 Jun 2025).

Taken together, these works suggest a more general interpretation of TRPO as a design pattern rather than a single loss: a system identifies or constructs temporally aligned and temporally degraded evidence, defines a pairwise preference over them, and optimizes the model so that local or global temporal grounding is preferred over temporally incorrect alternatives. In TPOUR this pattern is instantiated for dense retrieval; in TPO and TEMPLE it is instantiated for long-form video reasoning; in DenseDPO it appears as temporally factorized preference learning over aligned video segments.

7. Limitations, misconceptions, and open problems

A common misconception is that TRPO is simply standard contrastive retrieval with timestamp metadata. In TPOUR, temporal alignment is not reduced to appending time tokens or to supervised timestamp prediction. The key mechanism is the DPO-style relative preference gap against a reference encoder, combined with semantic contrastive learning. The authors report that a Temporal Contrastive baseline improves over plain Contriever but remains inferior to TRPO, indicating that direct contrastive separation of aligned and misaligned documents is not equivalent to preference optimization relative to a reference model (Kim et al., 16 Jun 2026).

The most immediate limitation is dependence on temporally distributed corpora. TPOUR requires multiple snapshots such as 2018, 2021, or monthly 2023 dumps to construct temporal preferences. Many domains do not have clean versioned corpora, and the paper identifies coarser time signals or partial temporal metadata as an open direction. Benefits are also more pronounced for explicit temporal queries than for implicit ones, which motivates future work on temporal intent detection and on learned selection of interpolation weights $d_i^- = \pi_k(D_i^-)$1.

The choice of $d_i^- = \pi_k(D_i^-)$2 remains heuristic in the reported experiments, finer-grained temporal resolution may be harder than yearly or monthly alignment, and extrapolation to future time periods is only preliminary. More broadly, temporal preference is only one preference dimension; the paper suggests that similar optimization could be extended to geographic, stylistic, or domain-specific relevance.

The related video literature adds additional cautions. TPO’s gains depend on the quality of self-generated preference pairs, TEMPLE reports that cross-architecture transfer of preference data is possible but less reliable than self-generated data, and DenseDPO notes that segment-level supervision improves local temporal credit assignment but may not fully capture long-term coherence (Li et al., 23 Jan 2025, Li et al., 21 Mar 2025, Wu et al., 4 Jun 2025). This suggests that future TRPO variants may need explicit local-versus-global temporal decomposition, model-specific preference generation, or hierarchical preference objectives spanning both short horizons and extended temporal structure.

In current usage, therefore, TRPO names both a specific retrieval loss and a broader methodological idea. In the strict sense of TPOUR, it is a pairwise DPO-style temporal preference loss for unsupervised dense retrieval on versioned corpora. In the broader research trajectory, it denotes a shift from treating time as metadata to treating temporal alignment itself as the object of preference optimization.