Decoupling Semantics and Logic: A Training-Free Coarse-to-Fine Pipeline for Video Retrieval-Augmented Generation

Abstract: This paper presents our system description for the 2nd Workshop on Multimodal Augmented Generation via MultimodAl Retrieval (MAGMaR). Addressing the critical challenges of cross-lingual long-video comprehension, strict persona adherence, and zero-hallucination temporal grounding, we propose a fully training-free, two-stage cascaded Video RAG pipeline. Our architecture strategically decouples semantic retrieval from cognitive logical reasoning through a modality-aware division of labor. In the first stage, a high-recall semantic pre-fetching module employs dense retrieval using only high-fidelity visual summaries and global text descriptions, explicitly isolating noisy modalities (e.g., OCR and ASR) to maintain a pristine vector space. In the second stage, an Adaptive, Iterative, and Reasoning-based (A.I.R.) filtering agent, powered by a commercial LLM, performs fine-grained cognitive reranking. The agent re-incorporates full multimodal contexts to enforce strict logical alignment with user personas, effectively pruning semantically similar but logically irrelevant candidates. Finally, a Prompt Sculpting mechanism constrains the generator to synthesize the distilled subset into strictly formatted JSON responses with exact chunk-level citations. Evaluated on the RAG track, our resource-aware approach shows exceptional precision in both information retrieval and persona-conditioned generation.

• The paper introduces a training-free, coarse-to-fine pipeline (C2F-RAG) that decouples semantic retrieval from logical filtering in video RAG.
• It employs a two-stage approach using dense semantic pre-fetching and cognitive reranking via an A.I.R. agent to drastically reduce modality noise.
• Empirical results show significant improvements in retrieval metrics and persona-aligned synthesis compared to state-of-the-art methods.

Decoupling Semantics and Logic in Video Retrieval-Augmented Generation: The C2F-RAG Pipeline

Introduction and Motivation

The exponential growth of multimodal corpora, particularly large-scale video archives, presents severe challenges for retrieval-augmented generation (RAG) systems requiring fine-grained evidence synthesis, persona adherence, and zero-hallucination grounding. Existing approaches for video RAG, typically rooted in text-based paradigms, are fundamentally constrained by the high-dimensional modality noise of long-form videos and an entrenched overreliance on surface-level semantic similarity. This results in an inability to distinguish genuine logical relevance from distractors—a deficit especially acute when strict persona-conditioned responses and precise temporal mapping are mandated. The paper introduces C2F-RAG, a fully training-free, coarse-to-fine pipeline that decouples semantic retrieval from cognitive logical reasoning, operationalizing modality-aware division of labor for scalable, high-precision video RAG (2606.07924).

C2F-RAG Architecture and Workflow

C2F-RAG is centered around a two-stage cascaded pipeline that systematically separates the dense retrieval of semantically relevant candidates from the logical reranking required for true evidence isolation and persona alignment.

The architecture comprises the following stages:

1. Coarse Stage: High-Recall Semantic Pre-fetching
   • Utilizes BGE-M3 as a dense retriever on lightweight global visual summaries and textual descriptions.
   • Excludes noisy modalities (OCR, ASR) in indexing, constructing a vector space with high SNR.
   • Retrieves a Top-1000 candidate pool from a 110k+ video corpus.
2. Fine Stage: Deep Cognitive Reranking
   • Serializes multimodal data (global summaries, frames, OCR/ASR) into a Serial Multimodal Context (SMC).
   • Employs an adapted A.I.R. (Adaptive, Iterative, Reasoning-based) agent—implemented as a commercial LLM—to perform logic-gated, persona-constrained reranking and pruning.
3. Generation Stage: Persona-Constrained Synthesis
   • Leverages prompt-based constrained decoding for strict persona fidelity and deterministic JSON formatting.
   • Each response is temporally grounded at the chunk level using the SMC’s fine-grained metadata.

     Figure 1: Overview of the C2F-RAG pipeline, delineating the decoupled coarse-to-fine retrieval and generation workflow.

This decoupling ensures that the computationally scalable dense retrieval is not compromised by the multimodal noise, while downstream logical filtering enforces strict evidence and persona alignment.

Technical Contributions

Modality Decoupling and Coarse Retrieval

The pipeline harnesses modality-aware indexing, removing OCR and ASR during vector space construction. BGE-M3 is exploited for its multilingual and long-context representation capacity, avoiding truncation of high-information summary inputs. By structuring video content as a fusion of multimodal LLM-generated global summaries and visual keyframe captions—termed the "Sandwich" text structure—the retriever dramatically reduces the inclusion of irrelevant background noise in the Top-1000 candidate set.

Cognitive Reranking with A.I.R.

The fine stage’s core is an adaptation of the A.I.R. (Adaptive, Iterative, Reasoning-based) framework originally developed for intra-video evidence selection to inter-video cognitive filtering. The SMC linearizes heterogeneous modalities with explicit markers, maintaining timeline integrity for cross-modal reasoning. The A.I.R. agent applies a 5-tier Relevance Scoring Matrix and an iterative batch-wise refinement, isolating a "golden subset" (∼10–15 videos) through explicit logical justification and aggressive exclusion of "hard negatives."

Scoring is governed by a Logic-Gated Exponential Attenuation (LGEA) mechanism:

$$S_{final}(v) = s_{coarse}(v) \cdot \exp\left(-\gamma \cdot (1 - L(v))\right)$$

where the final score exponentially penalizes candidates with logical misalignment, directly controlling the margin between semantically similar distractors and truly relevant evidence.

Zero-Hallucination Persona-Constrained Generation

Generation relies on prompt sculpting with enforced persona adherence, cross-lingual adaptability, and deterministic schema validation. The response generation process incorporates strict zero-hallucination directives, sentence-level timestamped citation, and post-processing for JSON conformance. This ensures not only that narrative content matches user persona but also every factual statement is auditable to a support chunk in the source videos.

Empirical Results

Retrieval Precision and Logical Filtering

Compared to SOTA multimodal retrievers (OmniEmbed, OmniEmbed+RankVideo, Mixedbread), C2F-RAG yields a pronounced performance advantage on the MultiVENT 2.0 corpus. It achieves an nDCG@10 of 0.848—surpassing all baselines by 13+ absolute points—and Recall@10 of 0.773. The ability to penalize hard negatives is directly attributed to the deep logical filtering of the adapted A.I.R. agent, as single-stage or standard two-stage retrievers fail to traverse the semantic-logical gap.

Generation Performance and Grounding

The pipeline’s chunk-level synthesis in the Oracle setting (using ground-truth retrieval) achieves an Info F1 of 0.463 and Cite F1 of 0.337, an improvement of 15–66% over the official CAG baseline in recall and F1 metrics. The system demonstrates not just higher overall average score but also a much-improved balance between information recall and precision due to its rigorous persona alignment and comprehensive evidence extraction.

Qualitative Persona Adaptation

A case study reveals that under differing persona constraints (e.g., "Cynical Journalist" vs. "Research Analyst" for the same event), generated outputs diverge in both lexical texture and evidence selection, confirming effective persona bifurcation and evidence filtration by the A.I.R. agent.

Efficiency and Practicality

Despite the computational overhead of evaluating 1,000 candidates per query, the pipeline, aided by parallelization and early stopping on null outputs for negatives, completes reranking for 19,000 video-query pairs in under five minutes using 15 threads. The synthesis overhead (mean: 63 seconds per query) is considered acceptable given the complexity of multimodal, temporally-grounded generation. Unlike systems requiring costly full corpus LLM inference or fine-tuning, C2F-RAG scales with high throughput and minimal physical retraining.

Theoretical and Practical Implications

The work affirms several principles for large-scale retrieval-augmented reasoning:

• Necessity of Decoupling: Pure dense semantic retrieval is inadequate for persona-constrained, zero-hallucination video RAG at scale, necessitating explicit separation of semantic fetching and logical reranking.
• Multimodal Serialization: Unified SMC linearization is critical for enabling LLM-based agents to perform deep evidence reasoning, suggesting further generalizations for multi-document and cross-modal tasks.
• Training-Free Adaptability: Prompt-based cascaded architectures offer strong adaptability without fine-tuning, which is significant for industrial deployment under shifting persona, modality, or language regimes.

Future work is likely to extend toward improved fine-grained open-set temporal grounding and real-time efficiency tuning, possibly leveraging further advances in hybrid retriever-generator system design and deeper cognitive agent integration.

Conclusion

C2F-RAG demonstrates that a fully training-free, cascaded architecture—explicitly decoupling semantic retrieval (via advanced dense embeddings) from logical reasoning (via persona-constrained LLM filtering)—is both necessary and sufficient for achieving state-of-the-art performance in large-scale video RAG. The system’s robust outperformance on both retrieval and synthesis tracks, alongside scalable practicality, lays groundwork for future research in plug-and-play, multimodal cognitive agents for evidence-driven generation tasks in diverse, dynamic environments.