Time-Specifier Model Merging (TSM)

• Time-Specifier Model Merging (TSM) is a technique that merges seven temporal specialist retrievers into one unified dense model to handle explicit time constraints.
• It employs uniform weight averaging to combine individual models, effectively mitigating catastrophic forgetting and preserving non-temporal retrieval quality.
• Evaluated on benchmarks like TimeQA and MS MARCO, TSM demonstrates enhanced temporal recall and ranking performance over traditional fine-tuning methods.

Time-Specifier Model Merging (TSM) is a model-merging method for Temporal Information Retrieval (TIR) that improves dense retrieval on queries with explicit temporal constraints while preserving accuracy on non-temporal queries. The method trains specialized dense retrievers for individual time specifiers and merges them into a single unified retriever by simple weight averaging, thereby avoiding both a temporal/non-temporal router and the catastrophic forgetting that accompanies monolithic temporal fine-tuning. In the reported setting, TSM is instantiated on Contriever and DPR, uses seven time-specifier specialists derived from TimeQA, and is evaluated on temporal and non-temporal retrieval benchmarks including TimeQA, Nobel Prize, Natural Questions, and MS MARCO (Han et al., 9 Jul 2025).

1. Formal setting and motivation

Temporal Information Retrieval extends standard information retrieval by requiring ranked results to satisfy not only topical relevance but also explicit temporal constraints. In a standard dense retriever, a query $q$ and corpus documents $d_i \in \mathcal{C}$ are encoded as

$$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$

with ranking induced by dot-product similarity

$$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$

The retriever returns top-$k$ documents

$D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$

TSM is formulated around the distinction between temporal and non-temporal queries. The temporal subset is defined as

$$Q_T = \{q_T\in Q \mid q_T=(s,t),\, s\in\mathcal{S},\, t\in\mathcal{T}\},$$

where $\mathcal{S}$ is a set of time specifiers and $\mathcal{T}$ is a set of concrete time points or intervals. The remaining queries are

$Q_N = Q \setminus Q_T.$

Within this setup, temporal queries differ from non-temporal ones by containing explicit temporal constraints such as “in 2015,” “between 2010 and 2012,” or “in early 1990s.”

The motivating failure mode is an attention bias in dense retrievers such as Contriever and DPR toward content words rather than explicit temporal expressions. As a result, such models often retrieve passages that match the right entity or event but not the requested time. The paper reports that fine-tuning Contriever on TimeQA improves temporal Recall@20 but significantly degrades performance on Natural Questions, illustrating catastrophic forgetting of non-temporal retrieval capability (Han et al., 9 Jul 2025).

The work situates itself against three broad prior directions. Temporal pre-training and masking methods improve temporal understanding but, when adapted to retrieval, are still typically optimized for temporal tasks. Temporal scoring and specialized temporal models focus on temporal queries and seldom evaluate the effect on non-temporal retrieval. Routing approaches such as TS-Contriever and TempRetriever maintain two retrievers—a vanilla dense retriever and a temporal-fine-tuned retriever—and use a classifier to choose between them, but this requires two dense retrievers in memory and makes retrieval quality sensitive to router errors. TSM addresses the same problem with a single retriever.

2. Time-specifier taxonomy and specialist training

TSM relies on a fixed taxonomy of seven explicit time specifiers drawn from TimeQA. These are: in [time], after [time], before [time], in early [time]s, in late [time]s, between [time_1] and [time_2], and from [time_1] to [time_2]. Each temporal question in TimeQA is labeled by one of these specifiers, and TSM splits the temporal training data accordingly.

The retriever backbone is a standard bi-encoder. For Contriever, the same encoder is used for queries and passages. For DPR, the query and passage encoders share architecture but not necessarily weights. In both cases, retrieval operates over 100-word passages created by sliding-window chunking, following DPR and multi-passage BERT.

For each time specifier $d_i \in \mathcal{C}$0, TSM constructs a dataset $d_i \in \mathcal{C}$1 of query–positive-passage pairs where the query uses $d_i \in \mathcal{C}$2. Because some specifiers are under-represented, the TimeQA training data are augmented using the official scripts and labels, particularly for after, before, in early, and in late. The resulting train/dev counts are: from [time_1] to [time_2] 11,676/2,486; in [time] 5,759/1,233; between [time_1] and [time_2] 4,888/1,054; after [time] 2,741/587; before [time] 2,867/609; in early [time]s 1,885/438; and in late [time]s 2,392/474, for a total of 32,208 train and 6,881 dev examples (Han et al., 9 Jul 2025). Only answerable questions are used, because contrastive training requires at least one positive passage.

Each specifier-specific retriever is fine-tuned from the same base parameters $d_i \in \mathcal{C}$3 using the InfoNCE objective with in-batch negatives: $d_i \in \mathcal{C}$4 where $d_i \in \mathcal{C}$5 and $d_i \in \mathcal{C}$6. Training uses AdamW, learning rate $d_i \in \mathcal{C}$7, batch size 64 per GPU, 5 epochs, 5 in-batch negatives per query, evaluation every 50 steps on the dev set, and best-checkpoint selection by top-1 QA accuracy. The hardware reported is NVIDIA A100 80GB. This produces seven parameter sets $d_i \in \mathcal{C}$8, each specialized to one specifier.

3. Merging mechanism and temporal representation

The defining operation of TSM is uniform weight averaging across the seven specifier-specific retrievers: $d_i \in \mathcal{C}$9 with $$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$0. The merged retriever is a single static model. The paper explicitly characterizes the procedure as one-shot merging, symmetric averaging, and static deployment: all specialists are trained independently from the same base model, merged once after training, weighted equally, and then used as a single encoder for all queries (Han et al., 9 Jul 2025).

TSM does not introduce an explicit temporal embedding or a separate temporal scoring function. Temporal reasoning is instead embedded implicitly in the query and document representations learned through specifier-specific contrastive training. Because each specialist is trained on one specifier semantics, the contrastive signal encourages the encoder to attend to both the specifier tokens and the time-expression tokens. In the merged model, these specifier-specific biases are combined in a single embedding space.

The method is deliberately limited to explicit temporal constraints. It does not address implicit temporal queries such as “Who is the current president?” or relative expressions such as “two years ago.” Within its stated scope, the taxonomy covers point constraints, lower-bounded intervals, upper-bounded intervals, bounded intervals, and fuzzy early/late intervals.

The paper argues that catastrophic forgetting is mitigated because each specialist moves only a moderate distance from the base model along a single temporal dimension, while averaging cancels extreme task-specific deviations. This interpretation is supported by the reported parameter analysis: full fine-tuning and FT+Reg have the largest weight changes, while TSM has the smallest overall weight change among all methods for both Contriever and DPR (Han et al., 9 Jul 2025). A plausible implication is that TSM occupies a low-shift region of the retrieval parameter space in which temporal sensitivity can be added without strongly perturbing non-temporal semantics.

4. Inference procedure and deployment properties

At inference time, TSM uses exactly the same retrieval pipeline as a standard dense retriever. Offline, every passage $$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$1 in the target corpus is encoded with $$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$2 and stored in a dense index. At query time, any query $$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$3, whether temporal or non-temporal, is encoded as $$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$4. Ranking is then performed by the same dot-product similarity

$$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$5

No temporal query classifier is used. No routing is performed. No explicit temporal scoring term is added at runtime. The temporal behavior is a property of the merged representation itself: for temporal queries, the encoder tends to increase similarity to passages whose time expressions satisfy the query constraint; for non-temporal queries, it remains close to the original dense retriever.

This deployment profile is one of the main distinctions between TSM and its baselines. Routing methods require two retrievers and a temporal/non-temporal classifier. Ensembling requires all seven specialist retrievers to be run at inference time, followed by score normalization and fusion. TSM replaces both with a single merged retriever whose inference computation and latency are the same as those of a standard dense retriever (Han et al., 9 Jul 2025).

The additional cost is concentrated in training rather than inference. Seven specialist models must be fine-tuned instead of one, so total GPU time is roughly $$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$6 full fine-tuning, although the training is parallelizable. After merging, the final model has exactly the same parameter count as a single Contriever or DPR encoder, and the intermediate specialist checkpoints can be discarded.

5. Evaluation, metrics, and empirical findings

The evaluation uses Recall@$$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$7 and nDCG@$$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$8. Recall@$$\mathbf{q}=f_\theta(q),\quad \mathbf{d}_i=f_\theta(d_i),$$9 is defined as

$$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$0

and nDCG@$$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$1 is computed from

$$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$2

TSM is evaluated on two temporal QA retrieval tasks—TimeQA and Nobel Prize—and two mostly non-temporal BEIR tasks—Natural Questions and MS MARCO. TimeQA is the in-domain temporal benchmark. Nobel Prize is an out-of-domain temporal benchmark constructed from structured Nobel laureate data. Natural Questions contains 1.54% temporal queries in the test set, and MS MARCO contains 0.05%.

For Contriever-based retrieval, TSM attains the best average performance across the four datasets, with Recall@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$3 and nDCG@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$4. On TimeQA, TSM reaches Recall@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$5 and nDCG@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$6, improving over vanilla Contriever’s Recall@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$7 and nDCG@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$8, and exceeding the strongest baseline LoRA on that dataset. On Nobel Prize, TSM reaches Recall@20 $$sim(\mathbf{q},\mathbf{d}_i) = \mathbf{q}^\top \mathbf{d}_i.$$9 and nDCG@20 $k$0. On Natural Questions, TSM essentially preserves recall and improves ranking quality relative to vanilla Contriever, with Recall@20 $k$1 versus $k$2 and nDCG@20 $k$3 versus $k$4. On MS MARCO, TSM is slightly lower than vanilla Contriever at Recall@20 $k$5 versus $k$6, but remains competitive at Recall@5 $k$7 versus $k$8 (Han et al., 9 Jul 2025).

For DPR-based retrieval, TSM again gives the highest average score across datasets, with Recall@20 $k$9 and nDCG@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$0. On TimeQA, it achieves Recall@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$1 and nDCG@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$2. On Nobel Prize, it reaches Recall@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$3 and nDCG@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$4. On Natural Questions, where vanilla DPR is strongly in-domain, TSM drops from Recall@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$5 and nDCG@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$6 to $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$7 and $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$8, but still performs much better than FT, FT+Reg, LoRA, and Ensembling. On MS MARCO, TSM improves over vanilla DPR, reaching Recall@20 $D=\{d_1,\ldots,d_k\} = Retriever(q,\mathcal{C}).$9 and nDCG@20 $$Q_T = \{q_T\in Q \mid q_T=(s,t),\, s\in\mathcal{S},\, t\in\mathcal{T}\},$$0 compared with $$Q_T = \{q_T\in Q \mid q_T=(s,t),\, s\in\mathcal{S},\, t\in\mathcal{T}\},$$1 and $$Q_T = \{q_T\in Q \mid q_T=(s,t),\, s\in\mathcal{S},\, t\in\mathcal{T}\},$$2.

Several analyses characterize how the merged model behaves. Progressive merging from 1 to 7 specialists increases Recall@20 on TimeQA, Nobel Prize, Natural Questions, and MS MARCO, indicating that additional specifiers are synergistic rather than destabilizing. Per-specifier evaluation shows that TSM has the highest Recall@20 for every time-specifier subset of TimeQA. The paper’s similarity heatmaps show that TSM places higher weight on temporal tokens such as years while retaining salience for entity tokens, whereas vanilla Contriever underweights dates and FT+Reg overweights temporal tokens without integrating them as effectively. In the Charles Clarke case study, TSM retrieves the passage stating that he was “Member of Parliament (MP) for Norwich South from 1997 until 2010,” while the compared baselines either match the entity without the interval or over-focus on unrelated dates.

6. Position in the model-merging literature, limitations, and disambiguation

Within model merging, TSM occupies a particularly simple design point. It uses static, symmetric, one-shot averaging of specialist retrievers. This contrasts with gradient-based supervised merging as in "SuperMerge: An Approach For Gradient-Based Model Merging" (Yang et al., 2024), input-conditioned shared/exclusive decomposition as in "Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging" (Lu et al., 2024), test-time continual merging with null-space constrained gating as in "MINGLE: Mixtures of Null-Space Gated Low-Rank Experts for Test-Time Continual Model Merging" (Qiu et al., 17 May 2025), and prompt-specific LoRA merging with sparse centroid-based routing as in "Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging" (Bertolissi et al., 20 May 2025). This suggests that TSM’s contribution is not a new general-purpose merging operator, but a retrieval-specific use of specifier-specialized training coupled to an unusually lightweight merge rule (Han et al., 9 Jul 2025).

The authors identify four main limitations. First, TSM depends on labeled temporal data organized by specifier, and under-represented specifiers are less well captured. Second, the current formulation focuses only on explicit temporal constraints rather than relative, implicit, or vague temporal expressions. Third, it considers only seven specifiers. Fourth, it uses only uniform weight averaging and does not explore layer-wise averaging, spherical interpolation, or Fisher-weighted merging.

The acronym “TSM” is overloaded in adjacent literature. In "Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents" (Su et al., 12 Jan 2026), TSM denotes Temporal Semantic Memory, a memory and retrieval framework rather than model merging. In "TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Vision" (Zhuang et al., 10 Mar 2025), TSM denotes a timestep-dependent LoRA-expert mixture for diffusion models. In the information-retrieval context considered here, TSM refers specifically to Time-Specifier Model Merging.