ADQAB: Analytical Diachronic QA Benchmark

• ADQAB is a benchmark focused on evaluating longitudinal analytical QA by requiring retrieval of temporally coherent evidence across extended time periods.
• It uses a hybrid corpus of real and synthetic financial news to ensure dense temporal coverage and accurate trend analysis.
• The benchmark stresses temporally aware retrieval systems by addressing challenges such as endpoint bias reduction and contiguous evidence synthesis.

Analytical Diachronic Question Answering Benchmark (ADQAB) is a benchmark for evaluating systems that answer questions requiring synthesis and analysis across extended time periods rather than retrieval of a single time-local fact. It was introduced in “Reading Between the Timelines: RAG for Answering Diachronic Questions” as an evaluation suite for Retrieval-Augmented Generation (RAG) over a hybrid corpus of real and synthetic financial news, with questions grounded in stock-price trajectories for 25 publicly traded stocks over 2012–2022 (Lau et al., 21 Jul 2025). ADQAB centers longitudinal reasoning: a system must retrieve evidence that is simultaneously semantically relevant and temporally congruent with the query, cover the queried window rather than isolated endpoints, and generate an answer that is temporally coherent across that window.

1. Definition and formal task structure

ADQAB is built around “analytical diachronic questions” (ADQ), defined as queries that “require synthesizing and analyzing information across extended time periods” (Lau et al., 21 Jul 2025). In the formulation given for the benchmark, an ADQ combines a specific subject or entity, a broad time range rather than a single time point, and an analytical task such as trend analysis, summarization of evolution, change-point characterization, or cross-period comparison. Temporal constraints may appear as an interval $[t_{start}, t_{end}]$, as a set of discrete points $\{t_1, t_2, \ldots\}$, or as relative conditions such as “before $t_x$” or “after $t_y$.” A corpus $D=\{d_1,\ldots,d_n\}$ associates documents with time metadata, and the answering system must retrieve a subset $D' \subset D$ that is semantically relevant to the semantic content $Q_s$ and temporally congruent with the temporal constraints $Q_t$.

The benchmark is motivated by a specific failure mode of conventional RAG. Standard pipelines are described as being optimized for semantic similarity rather than temporal logic, which produces three characteristic errors: neglect of temporal constraints, poor temporal coverage across the requested window, and endpoint bias, meaning disproportionate attention to documents mentioning only the beginning or end of a period (Lau et al., 21 Jul 2025). ADQAB therefore differs from point-fact temporal QA benchmarks by requiring extended, contiguous temporal coverage and coherent cross-time synthesis. A common misconception is to treat diachronic QA as date lookup with light re-ranking; ADQAB instead tests whether a system can reconstruct and summarize evolution across time.

2. Corpus design and benchmark construction

The benchmark corpus is financial and stock-centric. Its real-news component is drawn from FNSPID, described as containing Nasdaq-supplied articles from 2012 to 2022. Real articles were curated in two stages: first, spaCy NER filtered for articles containing explicit date mentions including a year; second, LLaMA-3.1-7B-Instruct verified whether the article narrative contained sufficient internal temporal context for approximate timeline reconstruction (Lau et al., 21 Jul 2025).

To guarantee temporal density, the benchmark augments the curated real corpus with synthetic news. LLaMA-3.3-70B-Instruct generated one synthetic article for each stock and each month from January 2012 to December 2022, conditioned on the historical stock price for that stock-month. The paper states the final augmented corpus as

$C_{final} = C_{real} \cup C_{synth}$

with month-level coverage ensured by the synthetic component (Lau et al., 21 Jul 2025).

The resulting corpus statistics are explicit. The real corpus $C_{real}$ contains 23,737 articles with average length 840 words. The synthetic corpus $\{t_1, t_2, \ldots\}$0 contains 3,300 articles with average length 192 words. The total corpus $\{t_1, t_2, \ldots\}$1 contains 27,037 articles with average length 761 words. Documents were split into non-overlapping chunks up to 2,048 tokens, yielding 77,965 chunks. This construction is integral to ADQAB’s design: the benchmark is not merely a question set layered over an arbitrary archive, but a temporally densified corpus meant to support month-by-month diachronic retrieval (Lau et al., 21 Jul 2025).

The construction process also includes quality control on the evaluation items. Questions were produced from historical stock-price graphs using InternVL3-78B, which generated an open-ended question-answer pair from a selected trend window; a separate Vision-LLM verified correctness. Distractors were then generated by LLaMA-3.3-70B-Instruct using wrong-trend, correct-trend-but-wrong-period, and fabricated-content strategies. A VLM-based correctness framework inspired by VMCBench retained only items with confidence at least 4 on a 5-point scale. A 20% random sample was manually checked, and 94% met the criteria for correctness and clarity after VLM pre-filtering (Lau et al., 21 Jul 2025).

3. Question families and evaluation protocol

ADQAB is formulated as multiple-choice question answering. The evaluation set contains 525 questions, partitioned evenly across seven temporal query families: 75 Specific Time Period (year-only trend), 75 Before Year Anchor, 75 Before Month Anchor, 75 After Year Anchor, 75 After Month Anchor, 75 Interval (Years), and 75 Interval (Months) (Lau et al., 21 Jul 2025).

These question families are designed to force retrieval over windows rather than isolated dates. Specific Time Period questions ask for trend characterization within a single year. Before/After questions require aggregated evidence on one side of a temporal anchor, either at year or month resolution. Time Range questions require analysis within a bounded interval such as a multi-year span or a month-bounded subperiod. The exemplars given in the benchmark description emphasize “general stock price trend,” “predominant stock price trend,” and “overall stock price trend,” indicating that the target operation is analytical summarization over a period rather than extraction of one timestamped fact (Lau et al., 21 Jul 2025).

The evaluation protocol sharply separates retrieval from answer selection. For each multiple-choice item, the retrieval module receives only the question text and not the answer options. The generator then receives the retrieved context, the question stem, and the options, and selects the final answer. Performance is reported as Accuracy, with each experiment run five times and the mean and standard deviation reported. Top-$\{t_1, t_2, \ldots\}$2 analysis uses $\{t_1, t_2, \ldots\}$3 to test the effect of retrieval depth on end-to-end answering (Lau et al., 21 Jul 2025).

The paper does not define separate temporal coherence or temporal coverage metrics. Instead, temporal coherence is enforced by the task design itself: the questions require longitudinal reasoning, and the retrieval methods are constrained by explicit temporal handling. This means that ADQAB’s principal score is end-to-end correctness under a task whose solution presupposes temporally aligned retrieval.

4. Temporally aware retrieval and the TA-RAG framework

ADQAB was introduced together with TA-RAG, a temporally aware RAG framework, but the benchmark is explicitly intended for any system that claims to support longitudinal analytical question answering (Lau et al., 21 Jul 2025). The framework is important because it makes explicit the retrieval assumptions that ADQAB is designed to stress-test.

TA-RAG begins with time information extraction from documents. A two-stage LLM annotation pipeline estimates publication time $\{t_1, t_2, \ldots\}$4 and event intervals $\{t_1, t_2, \ldots\}$5. The first stage infers $\{t_1, t_2, \ldots\}$6 and a brief abstract from document heads and tails; the second stage annotates chunk-level events and intervals using document-level context. Question processing then disentangles the semantic core $\{t_1, t_2, \ldots\}$7 from temporal constraints $\{t_1, t_2, \ldots\}$8, isolating what the system must retrieve from when it must retrieve it (Lau et al., 21 Jul 2025).

The central retrieval device is the hypothetical temporal query embedding

$\{t_1, t_2, \ldots\}$9

where anchor points $t_x$0 sample the temporal landscape of the query at a granularity one level finer than the query’s primary temporal unit. Candidate chunks are then filtered by interval overlap, expressed as $t_x$1, and ranked by semantic similarity to $t_x$2. The implementation uses an interval-tree for temporal filtering and a flat embedding index for semantic ranking. Retrieved chunks are finally sorted chronologically by $t_x$3 and concatenated with explicit time metadata so that the generator can follow event evolution (Lau et al., 21 Jul 2025).

The implementation details underscore the benchmark’s technical orientation. Time extraction uses LLaMA-3.1-7B-Instruct; question processing and answer generation use LLaMA-3.3-70B-Instruct; embeddings use nomic-ai/nomic-embed-text-v1.5; reranking, when applied, uses bge-reranker-v2-m3; chunking uses spaCy; semantic retrieval uses Faiss; and interval operations use PyRange (Lau et al., 21 Jul 2025). ADQAB therefore functions both as an evaluation suite and as an operational specification of what temporally congruent retrieval must solve.

5. Empirical results and diagnostic value

The reported benchmark results show that ADQAB is difficult for conventional retrieval pipelines and that temporal handling materially changes performance (Lau et al., 21 Jul 2025).

Method	Acc@5 / @10 / @20 / @50
BM25	36.72 / 55.09 / 65.79 / 79.39
Naive RAG	44.27 / 56.72 / 67.50 / 74.82
Naive RAG + Reranker	48.30 / 59.39 / 72.42 / 82.10
TS-Retriever	32.04 / 42.78 / 52.15 / 58.51
TA-RAG + Reranker	59.89 / 71.50 / 80.53 / 87.09
TA-RAG (Full)	71.73 / 84.84 / 88.23 / 88.00

TA-RAG (Full) is reported as strongly outperforming all baselines. At $t_x$4, it exceeds Naive RAG by 27.46 percentage points and Naive RAG + Reranker by 23.43 points. At $t_x$5, it exceeds Naive RAG by 20.73 points and Naive RAG + Reranker by 15.81 points. These gains are described as consistent with the paper’s abstract claim of “13% to 27%” improvements over standard RAG, depending on the baseline and $t_x$6 (Lau et al., 21 Jul 2025).

The benchmark also exposes negative results that are diagnostically important. TS-Retriever performs poorly on ADQAB, which the paper interprets as evidence that retrievers optimized for time-sensitive queries can still fail on the broader demands of analytical diachronic QA, especially temporal coverage and endpoint-bias reduction. General-purpose reranking degrades TA-RAG: TA-RAG + Reranker is consistently below TA-RAG (Full), with the paper attributing this to semantic rerankers undoing temporal prioritization and demoting temporally crucial evidence (Lau et al., 21 Jul 2025).

Ablation experiments identify which components matter most. Removing $t_x$7 reduces Acc@5 from 71.73% to 67.05%. Removing time filtering reduces Acc@5 to 67.16%. Removing temporal context structuring reduces Acc@5 to 65.68%. At $t_x$8, all three ablations very slightly exceed the full system by at most 0.65 points, which the authors attribute to long-context effects characterized as “Lost in the Middle” (Lau et al., 21 Jul 2025). The diagnostic lesson is that ADQAB is not only a benchmark of answer correctness but also a probe for whether a system can maintain temporal fidelity under retrieval-depth and context-length stress.

6. Position within the temporal QA landscape

ADQAB occupies a specific niche within temporal QA: analytical synthesis over contiguous time windows in a retrieval-augmented setting. Its closest direct precursor is ChronoQA, a Chinese benchmark for temporal-sensitive RAG that contains 5,176 question-answer pairs derived from approximately 300,000 news articles published from January 1, 2019 to August 30, 2024, with 100% of questions requiring temporal reasoning and 1,915 multi-document instances, or 37%, requiring cross-document temporal reconciliation (Chen et al., 17 Aug 2025). ChronoQA emphasizes temporal alignment, logical consistency, explicit and implicit time expressions, and structural fields such as question_date, temporal_granularity, temporal_scope, and golden_chunks. This suggests a natural extension of ADQAB toward implicit-time normalization and evidence-level retrieval auditing.

HistoriQA-ThirdRepublic shifts the setting from finance to historical research. It is a French-language corpus of 1,782 questions, comprising 897 single-hop questions from parliamentary debates and 885 multi-hop questions linking debates and newspapers. Its multi-hop construction uses semantic similarity thresholds of at least 0.7 and temporal constraints such as within-one-week publication for cross-newspaper pairs, and its retrieval error analysis introduces “day-tolerance recall” to measure temporal near-misses (Pellet et al., 30 Jun 2026). A plausible implication is that ADQAB can be generalized from financial trend analysis to cross-source historical reasoning while preserving explicit temporal edge constraints.

ComplexTempQA contributes scale and taxonomic breadth. It contains 100,228,457 question-answer pairs spanning 1987–2023, organized into Attribute, Comparison, and Counting questions over Event, Entity, and Time, with explicit time scopes, difficulty labels, multi-hop questions, and unnamed-event questions (Gruber et al., 2024). Whereas ADQAB is compact and task-specific, ComplexTempQA demonstrates how diachronic evaluation can be stratified by time span, hop count, and question type at very large scale.

TempQA-WD contributes a different form of rigor: explicit temporal semantics over a knowledge base. The dataset contains 839 Wikidata questions with gold SPARQL and answers, with a 175-question dev subset carrying fine-grained intermediate annotations including AMR, $t_x$9-expressions, and gold entities and relations. Its SPARQL encodes before/after, overlap, earliest/latest, and derived intervals such as teenage years through Wikidata qualifiers and xsd:duration operations (Neelam et al., 2022). In relation to ADQAB, TempQA-WD shows how interpretability can be embedded directly into the benchmark through executable temporal semantics rather than only through end-to-end answer accuracy.

Across these resources, ADQAB is distinguished by its focus on contiguous temporal coverage within a dense corpus and by its explicit coupling to longitudinal analytical tasks. ChronoQA prioritizes evolving factuality in news RAG; HistoriQA-ThirdRepublic prioritizes historical cross-source synthesis under sparse evidence; ComplexTempQA prioritizes large-scale temporal taxonomy; TempQA-WD prioritizes interpretable temporal semantics over knowledge bases. Together they define the broader design space into which ADQAB fits.

7. Limitations and likely directions of expansion

The benchmark’s limitations are explicit. ADQAB is domain-specific, focusing on finance and only 25 stocks. Its synthetic news improves temporal continuity but can introduce stylistic or distributional biases and may not capture the full richness of real-world narratives. Its temporal granularity is concentrated at year and month levels, leaving finer-grained day- or week-level reasoning untested. Its principal evaluation metric is multiple-choice Accuracy, which does not separately quantify temporal coverage, temporal coherence, or evidence correctness. The underlying pipeline is also LLM-intensive: offline time extraction is described as computationally intensive, and question processing adds about 1.4× runtime compared with naive RAG (Lau et al., 21 Jul 2025).

Related benchmarks make the expansion path relatively clear. ChronoQA indicates how implicit temporal expressions can be normalized through a question_date anchor and how evaluation can be broken down by temporal type, expression type, answer modality, and single- versus multi-document reasoning (Chen et al., 17 Aug 2025). HistoriQA-ThirdRepublic suggests explicit graph-based evidence paths with semantic and temporal edges, source-aware retrieval, and temporal labels such as same-day or within-week (Pellet et al., 30 Jun 2026). ComplexTempQA suggests large-scale stratification by question taxonomy, difficulty, and explicit $t_y$0 intervals (Gruber et al., 2024). TempQA-WD suggests adding gold executable representations so that temporal reasoning steps, not only final answers, become auditable (Neelam et al., 2022).

A plausible synthesis is that ADQAB serves as the benchmark archetype for analytical diachronic RAG, while adjacent datasets supply complementary mechanisms for its generalization: implicit-time handling from ChronoQA, cross-source historical reasoning from HistoriQA-ThirdRepublic, large-scale taxonomic breadth from ComplexTempQA, and interpretable temporal semantics from TempQA-WD. Under that reading, ADQAB is less a closed benchmark than a precise formulation of what diachronic question answering must evaluate: temporally congruent retrieval, contiguous coverage of the requested window, and coherent synthesis of change over time.