FlashMMR: Multi-Moment Video Retrieval

• FlashMMR is a framework for multi-moment retrieval in video temporal grounding, enabling the detection of multiple semantically valid segments per query.
• It builds on a strong FlashVTG backbone by adding a post-verification module that refines segment boundaries, recalibrates confidence, and prunes low-quality proposals.
• The framework introduces the QV-M² benchmark and MMR-aware metrics, bridging the gap between single-moment and multi-moment evaluation in video retrieval.

Searching arXiv for FlashMMR and closely related video moment retrieval work to ground the article. FlashMMR is a framework for Multi-Moment Retrieval (MMR) in video temporal grounding, introduced in "When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions" (Cao et al., 20 Oct 2025). It addresses a limitation of classical Single-Moment Retrieval (SMR), in which a video–query pair is assumed to have exactly one correct temporal interval. FlashMMR is introduced together with the QV-M$^2$ benchmark and MMR-aware evaluation metrics, and is best understood as a strong MR backbone—implemented in the paper as an extension of FlashVTG—augmented with a Multi-moment Post-Verification (PV) module for temporal adjustment, semantic re-evaluation, and low-confidence proposal pruning.

1. Task Formulation and Motivation

The paper argues that standard video moment retrieval is too narrow for realistic use (Cao et al., 20 Oct 2025). In the SMR formulation, given a video $V$ and a language query $Q$, a model predicts one segment $[t^s, t^e]$ or a small ranked list optimized around a single target. By contrast, MMR assumes that one query can correspond to multiple non-overlapping moments in one video. Representative examples given in the paper include repeated instances of “cutting vegetables,” “a woman speaks to the camera,” or recurring sports events.

This shift changes the objective. The core challenge is no longer merely to find the best single segment, but to recover all semantically valid moments while avoiding redundant and false-positive segments. The paper situates this as a mismatch between current MR benchmarks and real deployment settings. Existing datasets typically annotate one dominant segment per query or are not designed and evaluated for one-to-many grounding. As a result, methods trained on them tend to optimize toward the highest-scoring instance and suppress or ignore additional valid moments.

The paper further distinguishes between proposal-free and proposal-based MR paradigms. Proposal-free set prediction methods may emit multiple spans, but are still supervised and evaluated under single-target assumptions. Proposal-based methods can return many candidates, but generally lack explicit mechanisms to reason about consistency across multiple moments. FlashMMR is presented as a response to this deficiency: it retains a strong retrieval backbone, but adds a second-stage mechanism intended to refine boundaries, recalibrate confidence, and preserve multiple valid alignments.

2. QV-M$^2$ Benchmark and MMR-Aware Metrics

To support MMR, the paper introduces QV-M$^2$, an MMR extension of QVHighlights (Cao et al., 20 Oct 2025). It keeps the original QVHighlights videos and adds new human annotations tailored to one-query-to-many-moments retrieval.

Item	Value
New queries	2,212
Videos	1,341
Annotated temporal moments	6,384
Average moments per query	2.9

The videos are from vlogs and news footage, and the annotations are manually curated with quality control. The paper characterizes the benchmark as substantially more multi-target than prior SMR-oriented datasets.

The evaluation protocol adapts standard MR metrics so that each prediction can match any unmatched ground-truth moment above an IoU threshold. The principal metric is Generalized mAP:

$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$

where $T$ is a set of IoU thresholds, for example $\{0.5, 0.55, \ldots, 0.9\}$, and $AP(\tau)$ is average precision at threshold $V$0. The paper also reports mAP stratified by the number of target moments: $V$1, $V$2, and $V$3. This stratification is important because difficulty rises sharply as the number of valid moments increases.

Two additional MMR metrics are defined. Mean IoU@k is

$V$4

computed only for queries with at least $V$5 ground-truth moments. Mean Recall@k is

$V$6

again computed only on queries with at least $V$7 ground-truth moments.

A central design property of these metrics is backward compatibility with SMR. When there is one target moment, G-mAP, mIoU@1, and mR@1 reduce to standard SMR-style evaluation. This makes QV-M$V$8 not only a new benchmark but also a bridge between single-target and multi-target temporal grounding.

3. Backbone Structure and Multi-Scale Temporal Processing

FlashMMR has both model-agnostic and backbone-tied components (Cao et al., 20 Oct 2025). The model-agnostic part is the post-verification philosophy: predicted spans are post-processed, candidate features are extracted, proposals are re-scored with a verification network, and low-confidence proposals are pruned using supervision derived from IoU and representation consistency. The backbone-tied part is inherited from FlashVTG. In the experiments, FlashMMR shares the common components and parameter settings of FlashVTG and augments them with PV.

Given a video $V$9, represented as clip-level features $Q$0, and a query $Q$1, represented as word features $Q$2, the model predicts a set of temporal spans

$Q$3

where $Q$4 are start and end times for the $Q$5-th candidate moment and $Q$6 is its confidence. The video is sampled at a fixed frame rate $Q$7, with the paper mentioning for example 0.5 FPS, and encoded using frozen visual backbones: SlowFast for video motion and content, and CLIP for visual and text semantics. Text is encoded by CLIP. Both modalities are projected into a shared $Q$8-dimensional space through MLPs. The architecture also uses dummy tokens / dummy encoder, following prior work, to absorb semantics irrelevant to the query.

Video–text fusion is performed with Adaptive Cross Attention (ACA) in a CG-DETR/FlashVTG-style design. ACA uses learnable dummy tokens to encode contextual information beyond explicit query content. After ACA, the fused temporal representation is

$Q$9

A Transformer encoder is then applied to capture long-range dependencies. This fused representation is the substrate for both proposal generation and later verification.

To support moments of varying duration, FlashMMR uses a temporal pyramid:

$[t^s, t^e]$0

Thus $[t^s, t^e]$1 is the original fused sequence, and higher levels are downsampled versions with coarser temporal granularity. Boundary predictions are computed at each scale using a shared convolutional head, and confidence estimation combines intra-scale and inter-scale scores with a learnable weighting factor $[t^s, t^e]$2. Up to this point, FlashMMR remains close to a strong SMR-style retriever that emits multiple proposals. The MMR-specific contribution appears in the post-verification stage.

4. Multi-moment Post-Verification Module

The Multi-moment Post-Verification (PV) module is the central contribution of FlashMMR (Cao et al., 20 Oct 2025). It is designed to refine initial predictions and improve alignment with ground truth by enforcing more robust temporal consistency and semantic verification. The module has two parts: post-processing with feature refinement and semantic consistency control / verification.

Let the initial predictions be

$[t^s, t^e]$3

Before verification, FlashMMR applies a constrained post-processing function

$[t^s, t^e]$4

where $[t^s, t^e]$5 controls boundary clipping and $[t^s, t^e]$6 controls temporal discretization or rounding granularity. The paper states that this function enforces structured constraints including minimum window length, maximum window length, temporal clipping, and rounding heuristics. This is the mechanism referred to as constrained temporal adjustment. Its practical role is to clip segment boundaries to the valid temporal range, prevent implausibly short or long windows, align boundaries to the video’s frame-sampling or clip grid, and produce interpretable candidate intervals for later verification.

For each refined interval $[t^s, t^e]$7, the model re-reads the multimodal content inside the proposed window by extracting

$[t^s, t^e]$8

This is a key design choice: verification does not rely only on the scalar confidence $[t^s, t^e]$9, but on the fused temporal representation inside each adjusted candidate span.

Each candidate feature sequence $^2$0 is then passed through a recurrent verification network:

$^2$1

where $^2$2 is a GRU-based module and $^2$3 is the refined confidence score. The paper states that this network captures contextual dependencies across extracted moment representations. A careful reading of the method indicates that the title’s “cross-moment interactions” are reflected more in this shared re-evaluation stage and in semantic consistency supervision than in an explicitly formalized pairwise cross-moment attention or graph interaction block.

The verifier is supervised by the best temporal IoU between a prediction and the set of ground-truth moments $^2$4:

$^2$5

In addition, the module uses a representation-consistency loss

$^2$6

where $^2$7 is a cosine-similarity matrix computed over fusion features and $^2$8 is the pairwise segment agreement from ground-truth labels. The full post-verification loss is

$^2$9

This gives PV two supervision signals: IoU regression / calibration, which trains refined confidence to match temporal overlap quality, and representation consistency, which encourages semantically related temporal regions to remain aligned. In MMR terms, the latter is intended to prevent repeated valid moments from being treated as unrelated or noisy detections.

5. Training Objective and Inference Procedure

FlashMMR follows FlashVTG’s base training losses and adds the PV loss (Cao et al., 20 Oct 2025). The inherited losses are Focal Loss for classification labels, L1 loss for temporal boundaries, and Clip-Aware Score Loss for clip-level confidence scores. The new verification term is the post-verification loss

$^2$0

The paper states that $^2$1 and $^2$2 are weighted by 9 and 7, respectively. Optimization uses AdamW, inference uses NMS threshold 0.7, and the reported hardware is a single RTX 4090.

The inference workflow has eight described stages. First, video clip features are extracted with SlowFast + CLIP, and text features with CLIP, after which both are projected into a shared space. Second, ACA with dummy tokens and Transformer refinement produce the fused sequence $^2$3. Third, multi-scale temporal features $^2$4 are built, temporal boundaries are predicted at each scale, and confidence scores are computed through intra-scale and inter-scale score heads, yielding initial proposals $^2$5. Fourth, constrained temporal adjustment applies $^2$6. Fifth, each adjusted proposal is cropped from the fused sequence. Sixth, the cropped feature is passed to the PGRU verifier to obtain refined score $^2$7. Seventh, the refined confidence is used to suppress low-quality proposals and is combined with NMS to remove redundant overlaps. Eighth, the system retains multiple semantically consistent, temporally valid segments and discards low-confidence or semantically inconsistent ones.

The paper does not provide a hard threshold formula for proposal pruning. It explicitly states, however, that the verification module re-evaluates candidate segments, and that through the filtering pipeline low-confidence proposals are pruned. This suggests that the effective pruning signal is the refined verifier confidence $^2$8, likely coupled with standard NMS.

6. Empirical Results and Ablation Evidence

The paper retrains and evaluates six existing MR methods—Moment-DETR, EATR, QD-DETR, TR-DETR, CG-DETR, and FlashVTG—on QV-M$^2$9 and QVHighlights under both SMR and MMR settings (Cao et al., 20 Oct 2025). It reports that evaluating on QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$0 causes a performance drop for all methods because MMR is harder, that training with QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$1 improves performance for all methods on both SMR and MMR metrics, and that FlashMMR is best overall.

On the QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$2 test set, FlashMMR achieves:

• G-mAP 35.14
• mAP@1_tgt 52.59
• mAP@2_tgt 42.52
• mAP@3+_tgt 22.89
• mIoU@1 56.29
• mIoU@2 49.64
• mIoU@3 42.92
• mR@1 48.81
• mR@2 44.33
• mR@3 38.50

The best prior baseline in that comparison, FlashVTG, achieves 32.14 G-mAP, 20.19 mAP@3+tgt, and 35.94 mR@3. The headline gains are therefore +3.00 G-mAP, +2.70 mAP@3+tgt, and +2.56 mR@3.

On QVHighlights validation, FlashMMR reports 48.07 G-mAP, 56.95 mAP@1_tgt, 35.78 mAP@2_tgt, 15.15 mAP@3+_tgt, 62.09 mIoU@1, 45.32 mIoU@2, 40.32 mIoU@3, 55.02 mR@1, 40.63 mR@2, and 36.68 mR@3. Compared with FlashVTG, the gains are smaller—48.07 vs 48.02 on G-mAP, 35.78 vs 35.08 on mAP@2_tgt, 15.15 vs 13.85 on mAP@3+_tgt, 45.32 vs 43.80 on mIoU@2, and 36.68 vs 35.17 on mR@3—consistent with QVHighlights being less MMR-focused.

The ablation study isolates the contribution of PV. On QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$3, removing PV gives 32.14 G-mAP, 39.48 mAP@2_tgt, 20.19 mAP@3+_tgt, 47.85 mIoU@2, 40.92 mIoU@3, 41.30 mR@2, and 35.94 mR@3. Adding PV raises these to 35.14, 42.52, 22.89, 49.64, 42.92, 44.33, and 38.50, respectively. On QVHighlights, PV also yields consistent gains, though smaller. The paper notes that it does not provide separate ablations for internal PV substeps such as constrained temporal adjustment alone versus verification alone; the evidence therefore supports the effectiveness of the PV module as a whole rather than a finer-grained attribution.

7. Relation to Prior Work, Limitations, and Naming Ambiguity

Relative to standard SMR methods, FlashMMR differs by adding a post-hoc multi-moment verification stage that recalibrates candidate quality and filters predictions for broader and cleaner multi-instance coverage (Cao et al., 20 Oct 2025). Relative to earlier MMR efforts such as SFABD and NEXT-VMR, its novelty lies less in a radically new base detector than in a practical refinement module built from constrained temporal adjustment, semantic re-evaluation with a verification network, and explicit post-verification supervision via best-IoU and representation consistency.

The paper also identifies several limitations. First, verification is described as still early-stage, and the authors suggest stronger strategies such as reinforcement learning or contrastive learning for better moment discrimination. Second, although QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$4 is high-quality, dataset scale remains limited relative to what larger future models may require. Third, the title’s emphasis on “cross-moment interactions” should be interpreted cautiously: the paper does not define a rich explicit inter-proposal interaction layer such as graph matching or pairwise attention over candidate moments. Fourth, PV introduces extra post-processing, feature cropping, verification, and NMS on top of the base retriever, so the method is heavier than pure one-pass SMR inference.

A common source of confusion is nomenclature. FlashMMR should not be conflated with MMR-Mamba, which concerns multi-modal MRI reconstruction and does not mention a method named FlashMMR (Zou et al., 2024). Nor should it be conflated with FLASH, a speculative decoding framework for large multimodal models that likewise does not use the exact string “FlashMMR” (Wang et al., 19 May 2025). In current arXiv usage, FlashMMR refers specifically to the video temporal grounding framework introduced with QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$5.

Taken together, the benchmark and method formalize a shift from one-query-to-one-moment grounding toward one-query-to-many-moments grounding. The paper’s broader contribution is therefore twofold: QV-M$$\text{G-mAP} = \frac{1}{|T|}\sum_{\tau \in T} AP(\tau),$$6 supplies the evaluation regime, and FlashMMR supplies a verification-centered baseline whose gains are concentrated on the metrics most sensitive to multi-target coverage.