Detector Reward in Video Generation

• Detector Reward is a reward function that uses outputs from high-performance object detectors to quantify the quality of visual content in generated videos.
• It combines an appearance reward based on per-frame human detection confidence with a temporal prompt-alignment reward that matches video frames to ordered action phases.
• Incorporated into GRPO for diffusion models, this approach has been shown to enhance video realism and human action fidelity, achieving high accuracy in human-preference benchmarks.

A detector reward is a reward function for generative models that leverages outputs from high-performance object detectors, particularly human detectors, to quantify the quality of visual content in generated data. In the context of video generation, detector reward models have been demonstrated to outperform specialized and trained alternatives for evaluating and optimizing human motion fidelity without the need for additional domain-specific finetuning. The HuDA (Human Detector-based Appearance) reward model exemplifies this approach, integrating human detection confidence with a temporal prompt-alignment criterion for evaluating and improving the realism of generated videos, especially those featuring human actions (Ashutosh et al., 15 Jan 2026).

1. Detector-Based Appearance Reward

A detector reward is constructed from the outputs of a pretrained object detector. In HuDA, the detector $\mathcal{D}$ (specifically, ViTDet from Detectron2) processes each frame $I_t\in\mathbb{R}^{H\times W\times3}$ and produces candidate bounding boxes $B$ with associated classification probabilities: $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$ For each frame, the maximum detection confidence is taken: $s_t = \max_B p_t(B).$ To provide a robust per-video appearance metric, HuDA aggregates over a sliding window of $W$ contiguous frames ($W=6$), evaluating: $$R_{\rm app}(\bm V) = \min_{1 \leq t \leq T - W + 1} \sum_{\tau=t}^{t+W-1} s_\tau,$$ where $\bm V = (I_1, \ldots, I_T)$ and $T$ is the video length. This “worst-case” formulation emphasizes identification of frames where the human detector is most likely to fail, thus penalizing videos with temporally local errors such as missing limbs or gross artifacts. An average-confidence variant normalizes by $I_t\in\mathbb{R}^{H\times W\times3}$0: $I_t\in\mathbb{R}^{H\times W\times3}$1

2. Temporal Prompt-Alignment Reward

HuDA’s second component, $I_t\in\mathbb{R}^{H\times W\times3}$2, addresses temporal grounding of generated content to a user-supplied textual prompt $I_t\in\mathbb{R}^{H\times W\times3}$3. The prompt $I_t\in\mathbb{R}^{H\times W\times3}$4 (e.g., “A person doing a backflip”) is decomposed into $I_t\in\mathbb{R}^{H\times W\times3}$5 ordered “phases” $I_t\in\mathbb{R}^{H\times W\times3}$6 using a LLM (Llama-3), with each phase intended to describe approximately one second of action. Each $I_t\in\mathbb{R}^{H\times W\times3}$7 is embedded via a CLIP-style model (BLIP) into a $I_t\in\mathbb{R}^{H\times W\times3}$8-dimensional space: $I_t\in\mathbb{R}^{H\times W\times3}$9. Similarly, visual embeddings $B$0 are computed per frame.

Representative frame indices $B$1 are selected for alignment with each $B$2. For each phase-frame pair, the cosine similarity is computed: $B$3 The total temporal alignment reward aggregates these scores: $B$4 with an average variant $B$5.

3. Composite HuDA Detector Reward

The principal HuDA reward is a weighted sum of the appearance and alignment components: $B$6 where $B$7, $B$8; this assigns double the influence to the appearance score relative to phase alignment. The design enforces frame-wise detection fidelity and “phase coverage” of dynamic motion, effectively penalizing videos that lack clear human detections or fail to follow the intended prompt sequence.

4. Group Reward Policy Optimization (GRPO) in Diffusion Models

HuDA is incorporated into the Group Reward Policy Optimization (GRPO) algorithm for fine-tuning diffusion-based video generators treated as policies $B$9. GRPO generalizes the Proximal Policy Optimization (PPO) framework by averaging over a group of $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$0 ($$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$1) samples per policy update. The workflow is summarized:

1. Sample $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$2 videos $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$3 from $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$4.
2. Compute rewards $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$5, then standardize to advantages:

$$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$6

1. Calculate likelihood ratios:

$$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$7

1. Compute unclipped $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$8 and clipped $$p_t(B) = \Pr\left(\mathcal{D}(I_t[B]) = \text{human}\right) \in [0,1].$$9 with $s_t = \max_B p_t(B).$0.
2. Minimize the surrogate loss:

$s_t = \max_B p_t(B).$1

1. Apply gradient step and update $s_t = \max_B p_t(B).$2.

This method enables large-batch, on-policy optimization with stability properties similar to PPO.

5. Empirical Performance and Ablation Analysis

Extensive benchmarking demonstrates the efficacy of the HuDA reward model. On 600 human vs. human video pairs from Wan 2.1, HuDA predicts human-preferred clips with 77.4% accuracy, surpassing both trained reward models such as VBench-2.0 (72.7%) and zero-shot vision-LLMs (55.0%). The “DanceGRPO” visual/motion baseline achieves 51.5%. Ablation indicates the critical role of each reward component: $s_t = \max_B p_t(B).$3 alone yields 74.2%, $s_t = \max_B p_t(B).$4 alone 59.1%, and single global prompt-video similarity 75.5%. An optical-flow-based variant gives 74.8%. In direct video generation comparisons (human preference over 300 prompts, 24 videos each), HuDA+GRPO achieves win-rates of 63% (easy), 65% (medium), and 73% (hard) over Wan 2.1, with prompt faithfulness preserved within ±4%. HuDA outperforms both prompt–video similarity (57% win-rate) and optical flow (90% win-rate head-to-head).

Model / Experiment	Reward-Model Accuracy	Generation Win-Rate vs. Wan 2.1
HuDA (full)	77.4%	63–73% (by prompt difficulty)
$s_t = \max_B p_t(B).$5 only	74.2%	75% (GRPO vs. $s_t = \max_B p_t(B).$6)
$s_t = \max_B p_t(B).$7 only	59.1%	-
Prompt–video similarity	75.5%	57% (vs. HuDA)
Optical flow	74.8%	90% (vs. HuDA)
VBench-2.0	72.7%	-
Zero-shot VLM	55.0%	-

6. Implementation Details and Extensibility

HuDA utilizes ViTDet as the detector $s_t = \max_B p_t(B).$8, BLIP for all text and video embeddings, and Llama-3 for generating prompt phase boundaries. The backbone video model is Wan 2.1 (1.3B and 14B parameter variants), with hyperparameters window $s_t = \max_B p_t(B).$9, phases $W$0, phase-weight $W$1, clip-ratio $W$2, and group size $W$3. No task-specific finetuning of the detector or embedding models is required, demonstrating the potential of off-the-shelf vision models as high-utility reward sources. Empirical evidence supports applicability of the detector reward paradigm to visually-complex non-human cases, including animal videos and human-object interactions (Ashutosh et al., 15 Jan 2026).

7. Context and Significance

This detector reward approach—integrating per-frame detection confidence and temporally-localized prompt grounding—offers a practical and effective method for improving and evaluating high-fidelity video generation, especially for human-centric actions. The findings suggest that “simple” reward models built atop robust pretrained detectors can exceed the performance of trained alternatives on tasks typically believed to require heavy manual annotation or domain adaptation. The framework provides both a template for further applications of detector-based rewards in reinforcement learning for generative modeling, and benchmarks for future work in controllable and realistic video synthesis.