Through the PRISM: Preference Representation in Intermediate States of Video Diffusion Models

Abstract: Evaluating video generation with clean, pixel-based reward models disconnects evaluation from the noisy diffusion process and incurs massive VAE decoding costs. In this paper, we challenge this paradigm by asking a fundamental question: Can a powerful video generator inherently discriminate preferences directly from noisy latents? To answer this, we introduce \textbf{PRISM} (\textbf{P}reference \textbf{R}epresentation in \textbf{I}ntermediate \textbf{S}tates of Diffusion \textbf{M}odels). PRISM employs a lightweight Query-based Aggregation head with a frozen video diffusion backbone to decode preference signals from noisy latents. Surprisingly, PRISM not only achieves SOTA preference accuracy but also unlocks strong noise-robustness, which enables early-stage Best-of-$N$ sampling. This allows for filtering suboptimal candidates at the very beginning of denoising, drastically reducing computation while boosting video quality. We also reveal a strong positive correlation between a backbone's generative performance and its inherent evaluative power, enabling self-improving video backbones.

• The paper introduces PRISM, a decoding-free latent video reward model that evaluates preferences in intermediate states, enhancing alignment efficiency.
• Using a Query-based Aggregation head and pairwise preference data, PRISM bypasses full denoising and reduces inference time up to 7.6× while preserving fidelity.
• Empirical evaluations demonstrate that PRISM outperforms pixel-level models under high noise, achieving state-of-the-art performance on key benchmarks.

Preference Representation in Intermediate States of Video Diffusion Models (PRISM): Technical Analysis

Motivation and Background

The proliferation of advanced Video Diffusion Transformers has greatly enhanced text-to-video generation, but preference alignment remains a major constraint in practical deployment. Conventional Video Reward Models (VRMs) operate externally and exclusively in the pixel space, introducing architectural mismatches and prohibitive computational costs—specifically, full denoising and VAE decoding for reward evaluation. This renders alignment methods such as Reinforcement Learning and inference-time scaling (Best-of-$N$ sampling) inefficient and noise-sensitive. The PRISM framework is motivated by the hypothesis that generative backbones themselves, as likelihood-based models, encode robust discriminative and evaluative priors in their latent space throughout the denoising trajectory.

Figure 1: PRISM evaluates preferences directly in the latent space, avoiding full denoising and VAE decoding and enabling efficient, noise-resilient reward modeling.

Methodology

PRISM formulates latent video reward modeling by directly exploiting the internal representations of a frozen Video Diffusion Transformer. Given noisy latent $z_t$, prompt $c$, and timestep $t$, the backbone’s intermediate features—before denoising is complete—are aggregated via a Query-based Aggregation head employing learnable queries and cross-attention. The features are unified across visual and textual modalities, projected, and dynamically probed by the queries, producing a global preference embedding. This approach circumvents global pooling, which typically degrades discriminative signals, and provides robust, noise-aware preference modeling.

Training employs pairwise preference data and adopts the Bradley-Terry model with ties, allowing for ambiguous cases and robust learning across the diffusion trajectory. The reward function is fully noise-conditioned and enables step-level evaluative guidance necessary for efficient alignment and early rejection strategies.

Empirical Evaluation

Benchmarking Preference Accuracy

PRISM demonstrates state-of-the-art alignment accuracy across VideoGen-RewardBench and VLRM-Bench, significantly outperforming pixel-based baselines (VideoReward, UnifiedReward, VideoScore2), especially at higher noise levels ($t \to 1000$). Pixel-level models exhibit performance collapse under noisy conditions due to unreliable decoding, while PRISM, operating directly in the latent space, maintains consistent accuracy across the entire denoising process.

Figure 2: PRISM preserves high preference accuracy across all noise levels, outperforming pixel-level models whose performance collapses as $t$ increases.

A notable finding is that backbone representational quality is more impactful than parameter count; for example, Wan2.1-1.3B achieves higher reward modeling accuracy than CogVideoX-2B despite its smaller scale, underscoring the importance of architectural and pretraining quality.

Inference-Time Scaling and Efficiency

PRISM enables efficient Best-of-$N$ sampling via early-stage latent evaluation. Instead of fully decoding $N$ candidates, PRISM prunes suboptimal trajectories immediately after scoring noisy latents, continuing only the optimal candidate to final pixel space, thus reducing redundant computation.

Figure 3: PRISM allows for early-stage scoring and pruning during Best-of-$N$ sampling, improving efficiency compared to full denoising required by conventional baselines.

Qualitative results indicate PRISM selects samples with superior semantic and physical fidelity, particularly in scenarios where baselines are susceptible to subject counting artifacts or motion inconsistencies.

Figure 4: BoN sampling with PRISM achieves higher semantic fidelity and motion consistency, often surpassing pixel-level reward models.

Efficiency evaluations indicate PRISM achieves performance plateaus at early intervention steps, allowing for up to $7.6\times$ reduction in inference time without loss in generative quality. This is enabled by modern schedulers like Flow Matching, which solidify semantic structures early in the denoising process.

Figure 5: PRISM enables a favorable efficiency-quality trade-off in BoN sampling, reaching generative quality plateaus at early intervention steps.

Ablation and Interpretability

Ablation studies confirm that the Query-based Aggregation mechanism substantially outperforms global pooling, particularly at high noise levels, by dynamically attending to preference-relevant regions and suppressing background or artifacts.

Visualization of the cross-attention scores highlights PRISM’s discriminative sensitivity: regions associated with high-fidelity structure receive stronger attention, while malformed or distorted regions elicit suppressed responses. This behavior offers interpretable preference assessment and validates the attention-driven aggregation methodology.

Figure 6: Visualization of cross-attention maps demonstrates PRISM’s ability to focus on high-fidelity semantic regions and suppress structurally distorted areas.

Implications and Future Directions

PRISM’s decoding-free, noise-aware latent reward modeling establishes a new paradigm for aligning generative models in video synthesis. The approach implies that generative backbones encode transferable evaluative priors, fostering joint scaling and self-evolution. Its architectural coupling to specific VAEs remains a constraint, limiting plug-and-play flexibility across heterogeneous models, but ongoing design of latent-space adapters or alignment techniques could mitigate this.

Practically, PRISM transforms inference-time scaling from theoretical optimization into a deployable solution, significantly improving both quality and efficiency. Theoretically, the strong correlation between generative performance and evaluative capability suggests generative models can serve as intrinsic reward sources, opening the door to self-improving architectures and latent-based alignment.

Conclusion

PRISM establishes an efficient, interpretable, and robust latent video reward modeling framework, leveraging generative priors directly in the noisy latent space. Its Query-based Aggregation head maintains strong discriminative power, enabling state-of-the-art preference alignment, decoding-free operation, and efficient scaling strategies. PRISM paves the way for practical, high-fidelity, preference-aligned video generation and advances the theoretical understanding of latent evaluative capacity in generative transformers.