Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval

Abstract: Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.

• The paper introduces DreamPRVR, which employs text-supervised truncated diffusion to generate global registers enhancing holistic video-text alignment.
• It integrates coarse-to-fine representation learning with Query Similarity Preservation and diversity losses, achieving state-of-the-art performance.
• Empirical evaluations on ActivityNet, Charades, and TVR datasets confirm improved retrieval accuracy and efficiency by reducing query ambiguity and local noise.

Diffusion-Guided Registers for Hierarchical Contextual Imagination in Partially Relevant Video Retrieval

Problem Motivation and Limitations in PRVR

Partially Relevant Video Retrieval (PRVR) addresses retrieval on untrimmed videos where textual queries often correspond exclusively to partial sub-events. The primary obstacles in PRVR are query ambiguity and local noise, manifesting as spurious local activations in globally irrelevant contexts—an issue exacerbated by sparse clip-level supervision frameworks such as Multiple Instance Learning (MIL). Prior approaches only partially mitigate these deficiencies by treating global context as regularization during training without explicitly enhancing holistic representations at inference.

Figure 1: (a) Local spikes emerge in globally irrelevant "people boating" videos, outscoring the true "demonstrating the accordion" reference. (b) MIL leads to sparse supervision. (c) DreamPRVR imagines registers via text-diffusion, concentrating on fine-grained learning for joint cross-modal embedding optimization.

Method: Coarse-to-Fine Diffusion-Guided Contextual Registers

DreamPRVR introduces a coarse-to-fine paradigm. The model first leverages a truncated text-supervised diffusion mechanism to generate global register tokens that encode holistic semantic context, then utilizes these registers for fine-grained cross-modal alignment, facilitating discriminative and robust retrieval.

Textual Semantic Structure Learning

Text queries, encoded by RoBERTa and Transformer layers, form a structured latent space via Query Similarity Preservation (QSP) and diversity losses. QSP aligns queries from the same video as positive pairs to reinforce intra-video compactness, while diversity loss encourages inter-video separation. The Textual Perturbation Sampler (TPS) injects controllable perturbations to model uncertainty and supervise register generation, ensuring the latent space retains both semantic variability and alignment.

Register Generation via Truncated Diffusion

The register generation pipeline initializes embeddings from a video-centric distribution via Probabilistic Variational Sampler (PVS), supporting structured initialization rather than random noise. Subsequently, a lightweight Diffusion Register Estimator (DRE) iteratively denoises embeddings, guided by TPS-sampled textual targets. The truncated diffusion model, with few steps and registers, is empirically validated for high efficiency and efficacy.

Figure 2: Method overview: query branch produces embeddings and supervision for register generation; PVS and DRE refine video features into optimal registers for hierarchical representation learning.

Register-Augmented Representation and Cross-Modal Similarity

Registers are integrated with frame- and clip-level embeddings via Register-Augmented Gaussian Attention Blocks with asymmetric masks. Video tokens interact with both registers and other tokens, while registers attend exclusively to video tokens, maximizing global contextual fusion during fine-grained representation learning.

Similarity between query and video is computed as a weighted combination of frame-level and clip-level maximum cosine similarities, yielding robust retrieval.

Empirical Evaluation and Ablation

DreamPRVR achieves state-of-the-art performance on ActivityNet Captions, Charades-STA, and TVR datasets, consistently surpassing baseline PRVR architectures and outperforming T2VR and VCMR models on SumR and recall metrics. Ablations validate that register-guided architectures—especially those using structured initialization and iterative diffusion refinement—exhibit clear improvements over pooling or non-generative mappings.

Figure 3: Retrieval accuracy as a function of register number and diffusion timesteps; optimal efficiency achieved with few registers and diffusion steps.

Model efficiency is demonstrated by competitive training and inference times, with minimal overhead relative to performance gains. Registers constructed with diffusion guidance deliver high final semantic purity, with t-SNE visualizations confirming their ability to form discriminative clusters for robust video boundaries.

Figure 4: t-SNE of learned textual space; full QSP and diversity losses produce a well-structured latent manifold with strong intra-video compactness.

Qualitative studies show that DreamPRVR suppresses spurious local spikes in globally irrelevant videos and concentrates responses on relevant moments. Iterative refinement, as illustrated in register space visualizations, progressively purifies global semantics through diffusion steps.

Figure 5: t-SNE of register generation at T=0, T=5, and T=10. Iterative diffusion yields convergence to discriminative clusters.

Implications and Future Directions

DreamPRVR provides a unified generative-discriminative retrieval pipeline, marking a shift from conventional instance-level supervision toward hierarchical imagination that bridges partial query relevance and full video context. The proposed diffusion-guided register construction demonstrates that lightweight generative models can improve multimodal embedding spaces while maintaining efficiency, suggesting broader applicability for cross-modal retrieval in open-world and ambiguous settings.

Practically, the integration of context-aware registers within PRVR frameworks reduces ambiguity and local noise, facilitating robust retrieval in real-world archives with untrimmed content. Theoretically, this work opens further investigation into hybrid paradigms combining structured textual supervision, generative modeling, and hierarchical attention for multimodal retrieval.

Conclusion

DreamPRVR employs text-supervised diffusion-guided registers for hierarchical cross-modal alignment in PRVR, combining coarse global imagination and fine-grained concentration to address query ambiguity and local noise. It outperforms prior models both in retrieval performance and efficiency, substantiating the efficacy of generative register architectures for robust video-text matching. This approach establishes new perspectives for generative-discriminative multimodal retrieval architectures (2604.03653).