LeAdQA: Temporal Grounding in VideoQA

• LeAdQA is an LLM-driven, context-aware temporal grounding framework designed for VideoQA, emphasizing causal and multi-step reasoning in long videos.
• It refines question-option pairs using GPT-4 rewriting and applies multi-threshold interval fusion with query-focused sampling to improve evidence selection.
• Empirical evaluations on NExT-QA, IntentQA, and NExT-GQA demonstrate increased answer accuracy and enhanced temporal evidence localization.

Searching arXiv for LeAdQA and closely related recent work. arxiv_search(query="LeAdQA Video Question Answering temporal grounding", max_results=5, sort_by="relevance") LeAdQA is an LLM-driven context-aware temporal grounding framework for Video Question Answering (VideoQA) designed for semantically complex questions over long videos. It targets a failure mode common in contemporary multimodal systems: critical evidence is typically sparse in time, yet task-agnostic frame sampling and heuristic retrieval expose the model to large amounts of irrelevant visual content, weakening temporal and causal reasoning. LeAdQA addresses this by coupling LLM-based causal-aware query refinement with fine-grained temporal grounding, multi-threshold interval fusion, and query-focused sampling into a video MLLM, yielding improved answer accuracy and grounded evidence localization on NExT-QA, IntentQA, and NExT-GQA (Dong et al., 20 Jul 2025).

1. Problem setting and conceptual scope

LeAdQA is formulated for VideoQA regimes in which videos are long, important events are sparse, and the questions require temporal reasoning, causal reasoning, or multi-step event understanding. The motivating observation is that processing all frames or uniformly sampled frames often dilutes the decisive evidence among redundant background content, producing what the authors call an “attention burnout effect.” In such settings, the model’s cross-modal capacity is spent on irrelevant frames, and the final answer can drift toward language priors rather than grounded video evidence (Dong et al., 20 Jul 2025).

The framework is specifically positioned against two prevalent strategies. The first is task-agnostic frame sampling, such as uniform or random sampling, which ignores the semantics of the question and its candidate options. The second is heuristic retrieval, including caption-then-retrieve pipelines or coarse segment localization methods that rely on superficial semantic overlap while missing causal-temporal structure. LeAdQA’s central premise is that temporal localization should be conditioned not only on the question but also on the meaning of each answer option, because complex multiple-choice VideoQA often depends on disambiguating events that are temporally adjacent yet causally distinct (Dong et al., 20 Jul 2025).

This places LeAdQA within a class of systems that treat retrieval and evidence selection as first-class reasoning operations rather than as preprocessing. A plausible implication is that the framework is less a new answer-generation model than a mechanism for restructuring the evidence presented to an existing MLLM.

2. Pipeline and causal-aware query refinement

LeAdQA is organized as a three-stage pipeline in the paper’s overall description, though its operational flow can be decomposed into four components: LLM-based causal-aware query refinement, temporal grounding, multi-threshold interval fusion, and MLLM-based answer generation. The input is a video $V$, a question $Q$, and a set of multiple-choice options $O = \{o_i\}_{i=1}^N$; the output is a predicted answer and, for NExT-GQA, temporal grounding as well (Dong et al., 20 Jul 2025).

The first stage rewrites each question-option pair into a refined textual description:

$$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$

where $\mathcal{F}$ is the LLM and $\mathcal{P}_{\text{rewrite}}$ is the rewriting prompt. The resulting set is $D = \{d_i\}_{i=1}^N$. In the reported implementation, GPT-4o performs this rewriting. The purpose is not merely paraphrastic. The rewritten description is intended to make temporal order explicit, surface causal structure, and encode option-specific constraints in a form that is more suitable for text-to-video grounding. A question such as “What does the man do after closing the door?” is therefore transformed from a short ambiguous query into a richer description of an event chain (Dong et al., 20 Jul 2025).

This option-conditioned rewriting is one of LeAdQA’s distinctive design choices. Rather than treating answer options as late-stage labels, the framework uses them as dynamic constraints during evidence retrieval. This changes the semantics of grounding itself: the grounding model is not locating a generic answer span in the video but retrieving temporally precise segments relevant to a particular hypothesis embodied by each option.

3. Temporal grounding formulation and model

After query refinement, LeAdQA applies a text-to-video temporal grounding model to each description $d_i$. The video is uniformly divided into $L_v$ clips,

$V \to \{v_i\}_{i=1}^{L_v},$

and for each clip the model predicts a foreground flag $Q$0, a boundary offset $Q$1, and a saliency score $Q$2. The boundary prediction defines a grounded temporal segment

$Q$3

where $Q$4 is the clip center. The grounded segment set is

$Q$5

Foreground clips are required to have positive saliency, while background clips have $Q$6 (Dong et al., 20 Jul 2025).

Architecturally, the grounding module is a dual-stream, transformer-based text-video alignment network influenced by MomentDETR and UniVTG. Text queries and video clips are encoded into $Q$7-dimensional features, query tokens are aggregated into a sentence representation via attentive pooling, positional and modality embeddings are added, and the joint token sequence is processed through $Q$8 transformer layers. The final video-token representations feed three prediction heads: a foreground head for binary relevance classification, a boundary regression head for start-end offsets, and a saliency head based on contrastive alignment (Dong et al., 20 Jul 2025).

The loss is the sum of foreground classification, boundary regression, and saliency terms:

$Q$9

Here $O = \{o_i\}_{i=1}^N$0 is binary cross-entropy for the foreground flag, $O = \{o_i\}_{i=1}^N$1 combines Smooth L1 and IoU losses for boundary refinement when $O = \{o_i\}_{i=1}^N$2, and $O = \{o_i\}_{i=1}^N$3 combines inter-video and intra-video contrastive saliency objectives. During inference, the model generates candidate segments for each refined description and applies non-maximum suppression to reduce redundant overlaps (Dong et al., 20 Jul 2025).

A notable aspect of this formulation is that grounding quality is not ancillary to QA quality. The paper treats temporal localization accuracy as a direct determinant of downstream answer accuracy, and the ablations support that interpretation.

4. Multi-threshold interval fusion and MLLM inference

For each option, the grounding model returns top-$O = \{o_i\}_{i=1}^N$4 intervals, giving a total candidate set

$O = \{o_i\}_{i=1}^N$5

LeAdQA then applies a two-stage fusion mechanism. Intra-option fusion merges overlapping intervals produced for the same option; inter-option fusion merges strongly overlapping intervals across different options. Overlap and union are defined in the usual temporal form,

$O = \{o_i\}_{i=1}^N$6

and intervals with $O = \{o_i\}_{i=1}^N$7 are merged by

$O = \{o_i\}_{i=1}^N$8

The threshold $O = \{o_i\}_{i=1}^N$9 governs the granularity of fusion: high values preserve fragmentation, while low values risk including irrelevant content (Dong et al., 20 Jul 2025).

The final fused interval set $$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$0 is used for query-focused keyframe sampling. Only frames within these grounded windows are sampled and passed to the video MLLM. The visual stream is encoded with CLIP-ViT,

$$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$1

projected into the LLM token space by an MLP,

$$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$2

and concatenated with a structured QA prompt encoding the question and candidate options. The answer is then generated by Tarsier-7B or Tarsier-34B. In the reported configuration, Tarsier functions as a pretrained backbone; LeAdQA improves QA primarily by improving the relevance and compactness of the visual evidence it receives rather than by retraining the MLLM itself (Dong et al., 20 Jul 2025).

This division of labor is methodologically important. LeAdQA trains the temporal grounding module, but causal rewriting and MLLM answer generation are used in a training-free manner. A plausible implication is that the framework can be interpreted as a plug-in retrieval-and-grounding layer for existing video MLLMs.

5. Datasets, metrics, and empirical performance

LeAdQA is evaluated on three long-video multiple-choice benchmarks with temporal and causal emphasis: NExT-QA, IntentQA, and NExT-GQA. NExT-QA contains 5,440 videos with average duration 44 seconds and 47,692 questions; IntentQA contains 4,303 videos and 16,297 questions; NExT-GQA extends NExT-QA with visual evidence annotations, using 5,417 videos and 10,531 annotated temporal segments for 8,911 QA pairs over 1,557 videos. VideoQA is measured by accuracy, while temporal grounding on NExT-GQA is measured by mIoP, [email protected], [email protected], mIoU, [email protected], [email protected], and Acc@GQA, where Acc@GQA requires both a correct answer and IoP $$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$3 (Dong et al., 20 Jul 2025).

Benchmark	Focus	Reported LeAdQA result
NExT-GQA	Grounded QA + temporal localization	LeAdQA-7B Acc@GQA 19.2; LeAdQA-34B VideoQA 75.7%
NExT-QA	Temporal, causal, descriptive VideoQA	LeAdQA-34B 80.6 average accuracy
IntentQA	Intent and causal reasoning	LeAdQA-34B 78.5 average accuracy

On NExT-GQA test, LeAdQA-7B achieves Acc@GQA 19.2, exceeding prior values reported for QGAC-TR at 18.3 and SeViLA at 16.6. It also reports mIoP 30.8, [email protected] 35.8, [email protected] 31.8, mIoU 14.3, [email protected] 21.1, and [email protected] 12.5. For VideoQA accuracy on the same benchmark, LeAdQA-7B reaches 66.9% and LeAdQA-34B 75.7%, compared with 52.9% for FrozenBiLM, 68.1% for SeViLA, and 63.6% for QGAC-TR (Dong et al., 20 Jul 2025).

On NExT-QA, Tarsier-7B improves from 71.6 to 72.1 with LeAdQA, and Tarsier-34B improves from 79.3 to 80.6. The per-type gains for LeAdQA-34B over Tarsier-34B are 74.4 to 75.7 on temporal questions, 80.5 to 81.9 on causal questions, and 85.3 to 86.6 on descriptive questions. On IntentQA, Tarsier-7B improves from 67.4 to 68.2 and Tarsier-34B from 76.9 to 78.5, with the largest reported subtype gain on Causal How, from 78.8 to 83.0 (Dong et al., 20 Jul 2025).

The ablation results provide the clearest interpretation of the method’s behavior. GPT-4 rewriting improves Tarsier-34B with uniform sampling from 79.3 to 80.6 on NExT-QA. Ground-truth segments yield 82.1, while random sampling gives 78.0, indicating a strong correlation between localization quality and QA quality. In interval fusion, top-5 segments with $$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$4 give the best reported NExT-QA accuracy of 80.2 for Tarsier-34B with 16 frames, whereas $$d_i = \mathcal{F}\big(\mathcal{P}_{\text{rewrite}}(Q, o_i)\big),$$5 drops to 73.3. Query-focused sampling also improves efficiency: 32 LeAdQA-selected frames achieve similar accuracy to uniform 48-frame sampling, reported as 81.2 versus 81.2 (Dong et al., 20 Jul 2025).

6. Interpretation, limitations, and relation to adjacent QA research

LeAdQA’s main conceptual claim is that precision in evidence selection can be as important as model scale for VideoQA. The framework does not propose a new multimodal backbone; instead, it sharpens the causal-temporal interface between the question-answer semantics and the video evidence. This suggests a shift in emphasis from “how large is the MLLM?” to “how precisely is the MLLM grounded?” (Dong et al., 20 Jul 2025).

The paper also identifies several limitations. Causal rewriting depends on GPT-4o, which is proprietary and may inject errors if its commonsense inference is wrong or biased. Temporal grounding remains clip- or segment-level rather than pixel- or region-level. The MLLM remains a black box whose internal reasoning is not explicitly structured by causal representations; the gains come primarily from better input selection. In addition, the grounding model requires segment-level annotations such as those provided by NExT-GQA, and such supervision is not always available (Dong et al., 20 Jul 2025).

Within the broader QA literature, LeAdQA belongs to a family of systems that improve answering by augmenting the model with auxiliary capabilities beyond direct response generation. LOVA3 argues for a multimodal triad of Answer, Ask, and Assess, showing that adding question generation and QA assessment improves VQA and multimodal benchmark performance (Zhao et al., 2024). DataFactory addresses hard TableQA by decomposing reasoning across a Database Team, a Knowledge Graph Team, and a ReAct-style Data Leader, explicitly separating structured retrieval from relational reasoning (Wang et al., 10 Mar 2026). AQAG-style work based on fine-tuned Meta-Llama-2-7B likewise treats controllable question generation and evaluation as reusable components for educational QA pipelines (Ehsan et al., 26 Aug 2025). This suggests a broader methodological pattern: contemporary QA systems increasingly externalize subproblems such as grounding, asking, assessment, and tool-mediated retrieval rather than expecting a monolithic LLM or MLLM to solve them entirely in-context.

In that sense, LeAdQA is best understood not simply as a VideoQA model but as an evidence orchestration framework for long-video reasoning. Its significance lies in making option semantics operational at retrieval time, grounding causal-temporal hypotheses before answer generation, and demonstrating that improved temporal grounding can yield simultaneous gains in accuracy, interpretability, and computational efficiency (Dong et al., 20 Jul 2025).