RTQA: Domain-Specific QA Paradigms

• RTQA is a multifaceted set of QA paradigms that address dynamic retrieval for open-domain questions, clinically grounded report evaluation, structured table reasoning, and recursive temporal KG decomposition.
• It employs specialized benchmarks and methodologies such as live news retrieval, LLM-based report mapping, and executable reasoning chains to overcome the limitations of static evaluation.
• Common challenges include handling uncertainty, ensuring retrieval freshness, and mitigating error propagation through robust verification and temporal decomposition.

Searching arXiv for RTQA-related papers to ground the article and disambiguate the acronym across domains. RTQA is an overloaded acronym in recent arXiv literature, denoting several distinct question-answering formulations and evaluation frameworks rather than a single canonical method. The term has been used for dynamic open-domain question answering over current events, radiology textual question answering and report evaluation, industrial table question answering, and recursive temporal knowledge-graph question answering. Across these usages, a common theme is the inadequacy of static benchmarks or purely surface-level evaluation when the underlying task requires temporally current information, clinically grounded fact transfer, structurally complex table reasoning, or explicit temporal decomposition (Kasai et al., 2022, Shi et al., 13 Jun 2026, Pan et al., 12 Jan 2026, Gong et al., 4 Sep 2025). This plurality of meanings makes “RTQA” best understood as a family of domain-specific QA paradigms whose shared concern is operational realism.

1. RTQA as a dynamic benchmark for instantaneous open-domain QA

In "RealTime QA: A Dynamic Benchmark for Instantaneous Open-Domain Question Answering" (Kasai et al., 2022), RTQA refers to a benchmark that departs from conventional open-domain QA benchmarks such as Natural Questions, TriviaQA, and ELI5 by requiring answers “right now” about novel or evolving events. Traditional benchmarks assume a fixed, time-invariant knowledge source and a one-time batch of questions, whereas this RTQA operates in a fully dynamic, weekly cycle.

Its question stream consists of multiple-choice quizzes scraped from news websites—CNN, The WEEK, and USA Today—every Saturday at 3 am GMT. The benchmark is explicitly time-sensitive: answers may change daily, as in “How many home runs has Shohei Ohtani hit so far this season?”, or weekly, as in “Which head of state announced his resignation this week?” Continuous evaluation requires systems to integrate newly published documents in order to remain current, reflecting information needs such as emergency management, financial updates, and sports scores (Kasai et al., 2022).

This formulation shifts the emphasis of open-domain QA from recall over a frozen corpus to adaptation under retrieval drift. A plausible implication is that RTQA redefines knowledge access as a temporally indexed systems problem rather than only a parametric memorization problem. The benchmark therefore targets instantaneous applications rather than archival question answering.

2. Weekly cycle, retrieval design, and baseline architecture

The weekly workflow in the dynamic RTQA benchmark comprises five components integrated into an evaluation loop: question announcement, document pool generation, optional retrieval, answer generation, and online evaluation (Kasai et al., 2022). Approximately 30 multiple-choice questions are announced at 3 am GMT; Google Custom Search is queried with each question; top-10 news articles are scraped and parsed; participants may use either the provided document pool or their own retriever; and submissions are scored automatically until the next batch is announced.

Two retrieval options are central. Google Custom Search indexes all articles published on CNN, USA Today, and The WEEK to date, uses exact question text as the query, returns top-10 URLs sorted by relevance, and reflects article additions or removals immediately through live crawling every Saturday. Dense Passage Retrieval, by contrast, operates over December 2018 English Wikipedia split into 100-word passages, with a BERT-based bi-encoder producing 768-dim vectors and inner-product similarity selecting top-5 passages (Kasai et al., 2022). The design explicitly juxtaposes live news retrieval against a static Wikipedia retriever.

The benchmark provides six real-time baselines. The open-book variants retrieve top-5 documents and then generate an answer using either RAG-Sequence with BART-large finetuned on Natural Questions or GPT-3 prompting with text-davinci-002. For GPT-3, the prompt template prepends both question date and article dates to signal temporal context. The closed-book lower-bound baselines are T5-11B finetuned on Natural Questions and GPT-3 prompting without retrieved passages (Kasai et al., 2022).

The retrieval and answer-generation setup demonstrates that RTQA is not merely a benchmark of language modeling. It is equally a benchmark of retrieval freshness, timestamp-sensitive prompting, and robustness to partial evidence. This suggests that, in real-time open-domain QA, retrieval policy can dominate generator architecture.

3. Evaluation protocol and empirical behavior in dynamic RTQA

The dynamic RTQA benchmark supports multiple-choice accuracy, a NOTA setting in which one choice is randomly replaced by “None of the above,” and free-generation evaluation via Exact Match and token-level F1 (Kasai et al., 2022). Because Google Custom Search results evolve hour by hour, the benchmark also includes a Temporal Degradation Protocol in which submissions carry timestamps and performance is plotted against submission delay.

The year-long averages reported for June 2022 to June 2023 establish the central empirical pattern:

Retrieval	Generator	Acc (orig)	Acc (NOTA)	EM (gen)	F1 (gen)
DPR	RAG	27.4 %	24.8 %	2.4 %	4.1 %
DPR	GPT-3	43.9 %	35.8 %	13.3 %	19.7 %
GCS	RAG	46.9 %	37.9 %	17.5 %	22.1 %
GCS	GPT-3	66.5 %	58.4 %	34.6 %	45.3 %
—	T5	39.1 %	35.3 %	9.7 %	14.7 %
—	GPT-3	44.9 %	34.1 %	15.3 %	22.3 %

The strongest baseline is GPT-3 with Google Custom Search, which reaches 66.5% in the original multiple-choice setting, 58.4% in the NOTA setting, 34.6% EM, and 45.3% F1 (Kasai et al., 2022). The paper states that retrieval matters most: GPT-3 with GCS and temporal prompts outperforms both closed-book and Wikipedia-only retrieval by about 20 points in multiple-choice accuracy and about 20 points in EM. All systems drop 6–10 points under the NOTA perturbation, underscoring uncertainty-estimation difficulty. Performance for GCS+GPT-3 remains stable for the first 24 hours after announcement but degrades sharply thereafter. In the first six weeks, the reported error breakdown is 52% correct, 34% retrieval errors, and 13% reading-comprehension errors (Kasai et al., 2022).

These results identify stale or insufficient evidence as the main failure mode. The paper further reports that GPT-3 can often properly update its generation results based on newly retrieved documents, but tends to return outdated answers when retrieved documents do not provide sufficient information (Kasai et al., 2022). The benchmark therefore exposes a boundary condition of retrieval-augmented generation: temporal adaptation is possible, but only when retrieval surfaces enough up-to-date evidence.

4. RTQA in radiology: report-grounded evaluation rather than news retrieval

In "ReportQA: QA-Based Radiology Report Evaluation" (Shi et al., 13 Jun 2026), RTQA denotes Radiology Textual Question Answering, a setting that demands evaluation metrics and protocols beyond surface-level text overlap in order to verify whether an automatically generated report conveys the clinically relevant information a radiologist would need. Here RTQA does not refer to temporal news QA. Instead, it is a clinically grounded evaluation regime in which QA is used as a proxy for report utility.

The ReportQA framework consists of four stages: construction of clinical knowledge trees, extraction of structured information from free-form reports, QA pair generation and quality control, and QA-based evaluation with reports as context (Shi et al., 13 Jun 2026). Radiologist-guided knowledge trees enumerate clinical entities and fine-grained attributes such as presence, location, shape, size, margin, chronicity, enhancement, severity, and certainty. Entities form a hierarchy, while attributes form a taxonomy of 15 dimensions.

Structured information extraction is performed with a two-stage LLM-based mapping process. Stage 1 extracts findings, diagnoses, and raw attributes; the output is compared against the radiologist-defined trees; and missing or ambiguous nodes are used to refine the ontology. Stage 2 remaps entity-attribute pairs to canonical nodes, filtering any item that cannot be aligned to the trees (Shi et al., 13 Jun 2026). The result is a controlled structured report.

QA generation includes leaf-level questions, ancestor-level questions for partial correctness, and negative questions over absent entities with “Insufficient information” as an answer option. Two-stage filtering removes trivial or unreliable questions: self-filtering discards questions answerable without the report, and report-based filtering discards items answered inaccurately even when the ground-truth report is given as context. The final output is approximately 100 QAs per report (Shi et al., 13 Jun 2026).

Evaluation uses an LLM judge model that answers each QA given the report as context. Per-question accuracies are aggregated into QAScore, which combines positive-question accuracy with an exponential penalty on the false-positive rate for negative questions via a harmonic mean (Shi et al., 13 Jun 2026). On the RadEvalX chest X-ray benchmark, QAScore shows Pearson $=0.4507$, Spearman $=0.4612$, and Kendall $=0.3910$ with radiologists’ total error counts, exceeding BLEU-4, BERTScore, CheXbert, RadGraph F1, RaTEScore, and GREEN (Shi et al., 13 Jun 2026).

A notable empirical claim is that report-based inference in vision-LLMs exhibits a strong negative prior, defaulting to “no findings” templates even with no image, whereas question-driven inference produces a milder positive prior bias and greater reliance on visual evidence (Shi et al., 13 Jun 2026). This suggests that, in radiology RTQA, decomposition into entity- and attribute-specific questions is not only an evaluation method but also a diagnostic lens on inference bias.

5. RTQA in industrial TableQA: structural complexity and verifiable reasoning

In "ReasonTabQA: A Comprehensive Benchmark for Table Question Answering from Real World Industrial Scenarios" (Pan et al., 12 Jan 2026), RTQA is used as shorthand for a large-scale bilingual benchmark for industrial table question answering. This usage differs sharply from both the real-time news benchmark and radiology RTQA. Its defining challenges are multi-sheet configurations, nested headers, large-scale tables, and multi-table setups.

The dataset contains 1,932 unique tables—1,101 Chinese and 831 English—with 5,523 questions spanning 30 sub-domains across seven major sectors (Pan et al., 12 Jan 2026). Structural complexity is quantified directly: 28.3% of tables are multi-sheet or multi-table directories, 34.4% have nested or complex headers, and 8.3% exceed 50 K cells. The dataset provides both final answers and explicit reasoning chains in two modes: thinking mode with explicit multi-step logic and intermediate computations, and no-thinking mode with concise direct reasoning steps (Pan et al., 12 Jan 2026).

Annotation uses six strong LLMs to generate candidate Python reasoning code, followed by regex extraction, code execution, human verification, and adjudication by expert annotators when needed. Reasoning chains are represented as Python code snippets plus human-written commentary, with explicit DataFrame operations and a uniform final print statement (Pan et al., 12 Jan 2026). This design makes the reasoning process executable and verifiable.

The associated TabCodeRL method uses a two-stage training pipeline consisting of cold-start supervised fine-tuning and DAPO-based reinforcement learning with three reward components: piecewise correctness reward, table-path selection reward, and inner-group code similarity reward (Pan et al., 12 Jan 2026). On ReasonTabQA, Gemini-3-Pro-Preview achieves 67.58%, the best open-source reasoning model before RL reaches 58.76%, and TabCodeRL raises Qwen3-8B from 49.87% to 61.89% in thinking mode and from 40.97% to 58.01% in no-thinking mode (Pan et al., 12 Jan 2026).

The benchmark also reports that table-specific models such as TableLLM and TableLlama fall below 15%, and that the best open-source model with TabCodeRL still trails the best closed-source model by about 5–6 percentage points (Pan et al., 12 Jan 2026). Performance drops by 9.95 points on hard queries and 10.06 points on complex tables, emphasizing unresolved difficulties in multi-table joins, nested headers, and large-scale noise handling.

A related but distinct line appears in "Enhancing TableQA through Verifiable Reasoning Trace Reward" (Kwok et al., 30 Jan 2026), where RTQA is glossed as "Reasoning-Trace Question Answering" under the RE-Tab paradigm. There the task is formalized as a POMDP over table states, atomic table operations, and rewards combining state-transition sufficiency and confidence (Kwok et al., 30 Jan 2026). This suggests a broader convergence in industrial TableQA toward reward-guided execution traces rather than answer-only supervision.

6. RTQA in temporal knowledge graphs and other adjacent usages

In "RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with LLMs" (Gong et al., 4 Sep 2025), RTQA denotes a training-free framework for Temporal Knowledge Graph Question Answering. The system consists of three plug-and-play components: Temporal Question Decomposer, Recursive Solver, and Answer Aggregator. Its objective is to address limited reasoning on compound temporal constraints and error propagation in decomposition frameworks.

The decomposer uses few-shot prompting to identify temporal keywords, generate explicit sub-questions in parent-to-children form, and annotate placeholders. The solver proceeds bottom-up over a decomposition tree, retrieving temporal KG facts, transforming quadruples $(s,p,o,t)$ into natural-language statements, and reasoning over them with the LLM. At each non-leaf node, the Answer Aggregator compares the answer obtained directly from retrieval plus reasoning with the answer summarized from child nodes, preferring the child-derived answer when valid and otherwise falling back (Gong et al., 4 Sep 2025).

The reported benchmark results show improved performance on temporally complex categories. On MultiTQ, RTQA reaches 0.424 Hits@1 on the “Multiple” category versus 0.335 for TimeR4, and overall Hits@1 increases from 0.728 to 0.765 (Gong et al., 4 Sep 2025). On TimelineKGQA, RTQA improves Hits@1 on the “Complex” split from 0.009 for a RAG baseline to 0.135 (Gong et al., 4 Sep 2025). Ablations indicate that removing the decomposer reduces overall Hits@1 to 0.709, disabling multi-answer aggregation reduces it to 0.752, and removing fact retrieval collapses performance to 0.070 (Gong et al., 4 Sep 2025).

Beyond the papers explicitly titled RTQA, adjacent works broaden the acronym’s semantic neighborhood. "Accelerating Real-Time Question Answering via Question Generation" (Fang et al., 2020) uses RTQA to mean real-time question answering in the systems sense of low-latency deployment. Its Ocean-Q framework eliminates online neural encoding by generating a large offline QA-pair pool and performing IR-style matching at test time, achieving 0.28 s per query versus 1.24 s per query for DENSPI, a 4.4× speedup, with about a 3–5 EM-point loss on SQuAD-open and HotpotQA (Fang et al., 2020). "RTime-QA: A Benchmark for Atomic Temporal Event Understanding in Large Multi-modal Models" (Liu et al., 25 May 2025) is not named RTQA, but it shares the concern with temporal correctness and strict bidirectional evaluation; its Strict-ACC requires both a video and its temporal negative counterpart to be answered correctly, and Qwen2-VL-7B improves from 34.6 Strict-ACC zero-shot to 65.9 after instruction tuning on RTime-IT (Liu et al., 25 May 2025).

These neighboring usages show that RTQA-related research recurrently treats brittleness in time, structure, or deployment as the core source of difficulty.

7. Conceptual synthesis, recurring challenges, and terminological ambiguity

Taken together, the literature indicates that RTQA is not a single benchmark family but an acronymic intersection of several research programs. In dynamic open-domain QA, RTQA emphasizes weekly question streams, live retrieval, timestamp-sensitive prompting, and degradation under retrieval drift (Kasai et al., 2022). In radiology, RTQA emphasizes clinically grounded report transfer, ontology-constrained QA generation, and evaluation aligned with expert judgment rather than lexical overlap (Shi et al., 13 Jun 2026). In industrial tables, RTQA emphasizes executable reasoning traces, multi-table operations, and verifiable reward design (Pan et al., 12 Jan 2026, Kwok et al., 30 Jan 2026). In temporal knowledge graphs, RTQA emphasizes recursive decomposition and multi-path answer aggregation under explicit temporal constraints (Gong et al., 4 Sep 2025).

Several recurring technical problems span these otherwise disjoint domains. One is uncertainty handling. The dynamic benchmark shows substantial degradation in the NOTA setting and identifies outdated answering under insufficient retrieval evidence as a key failure mode (Kasai et al., 2022). ReportQA introduces negative questions and a false-positive penalty within QAScore, directly penalizing unsupported presence predictions (Shi et al., 13 Jun 2026). Industrial table methods introduce verifiable reward components that punish extraction failure, execution errors, or incorrect table-path selection (Pan et al., 12 Jan 2026). A plausible implication is that abstention, controllable confidence, and verifiable intermediate states are emerging as cross-domain design requirements for RTQA-labeled systems.

A second shared issue is decomposition. In temporal KGQA, recursive decomposition is the central strategy for converting vague temporal constraints into explicit sub-problems, but it risks error propagation (Gong et al., 4 Sep 2025). In radiology, decomposition into entity-, attribute-, and hierarchy-specific questions is used to reveal negative prior bias and to score partial correctness (Shi et al., 13 Jun 2026). In TableQA, explicit reasoning chains and code-level supervision provide decomposed, auditable trajectories rather than opaque answer strings (Pan et al., 12 Jan 2026). This suggests that many RTQA settings are best seen as structured inference problems in which decomposition is beneficial only when accompanied by strong verification or aggregation mechanisms.

Finally, terminological ambiguity itself is part of the subject. Because “RTQA” has been used for RealTime QA, Radiology Textual Question Answering, ReasonTabQA, and Recursive Thinking for Temporal KGQA, the acronym does not uniquely identify a research area. For precise scholarly communication, the full expansion and citation are therefore essential. In practice, the open-domain benchmark of current-events QA (Kasai et al., 2022) is the earliest explicit “RealTime QA” usage in the supplied literature, but later work repurposes the same four letters for tasks with very different data structures, evaluation objectives, and deployment assumptions (Shi et al., 13 Jun 2026, Pan et al., 12 Jan 2026, Gong et al., 4 Sep 2025).