TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL

Abstract: Autoregressive (AR) video diffusion models enable low-latency streaming generation by synthesizing videos chunk by chunk with cached visual context, but this chunk-wise formulation makes temporal instruction following ambiguous. A single global prompt does not specify which sub-event should be realized in each chunk, while naively switching to step-wise prompts often leads to delayed reactions, blended step semantics, and error propagation across prompt transitions. These failures are difficult to address with supervised fine-tuning or distillation alone: SFT suffers from exposure bias, while rollout-based distillation still optimizes low-level denoising or teacher-distribution matching rather than directly enforcing action ordering and prompt-transition correctness. We address these challenges with TempAct, a planner--executor reinforcement learning framework that jointly optimizes temporal decomposition and step-conditioned execution for temporally plausible AR video generation. TempAct uses an LLM planner to explore span-aware step prompts that are executable by the video model, and trains an AR diffusion executor to follow these prompts under its own generated histories. Its key mechanism is hierarchical group exploration: candidate plans form planning groups, and each plan induces an execution group of multiple continuations from a shared visual context, enabling plan-level credit assignment for long-horizon temporal outcomes and executor-level credit assignment for prompt-switch behavior. We further design hierarchical rewards that combine plan-quality and full-video temporal feedback for the planner with local transition-level step-following rewards, aesthetic regularization, and KL constraints for the executor. Experiments on Self-Forcing and LongLive show that TempAct improves temporal consistency while preserving overall visual quality.

• The paper introduces a hierarchical planner–executor RL approach that decomposes complex instructions into temporally coherent, step-wise video generation.
• It employs an LLM planner and an AR diffusion executor optimized with localized, temporally structured rewards to mitigate issues like prompt-switch errors.
• Experimental results reveal a 15.5% increase in Temporal-Following Score while preserving overall visual quality, validating the framework's effectiveness.

TempAct: Reinforcement Learning for Temporally Faithful Autoregressive Video Generation

Motivation and Background

Instruction-following video generation with AR diffusion backbones has progressed, but temporal semantic fidelity under complex instructions remains unsolved in deployed systems. Streaming AR video generation typically uses a chunk-wise factorization, where each segment is conditioned on a fixed prompt or a step-wise instruction sequence. However, both paradigms are fundamentally limited: global-prompt conditioning results in event blending and ambiguous step realization (temporal confusion), while naive step prompts exacerbate drift, blending errors, and error propagation at prompt-switch transitions. Existing methods relying on supervised fine-tuning or distillation (e.g., DMD, Self-Forcing, LongLive) largely optimize for denoising or teacher-distribution matching, not explicit temporal action ordering or robust transition handling.

TempAct proposes to address these pathologies with a hierarchical planner--executor RL scheme (Figure 1), combining an LLM-based temporal decomposition mechanism with an autoregressive video diffusion executor. Crucially, both modules are directly optimized via RL against temporally structured rewards that explicitly quantify plan executability and step-following under model rollouts.

Figure 1: The TempAct framework contrasts single-prompt, step-prompt, and planner–executor RL paradigms and illustrates qualitative improvements and reward dynamics arising from hierarchical optimization.

Methodology

Hierarchical Planner--Executor Pipeline

TempAct formalizes temporally plausible AR video generation as two intertwined subproblems: (1) optimal segmentation of a complex instruction into span-aware executable steps (planning), and (2) accurate realization of these steps under AR context with robust prompt-switch execution (execution). The method instantiates these with an LLM planner and an AR diffusion executor (Figure 2):

• LLM Planner: Given an instruction, it samples $M$ candidate temporal plans, each decomposing the task into $J$ steps with explicit temporal assignments. These plans undergo prompt enhancement—each step is clarified to include action, object, and state constraints, and then smoothed across boundaries to mitigate abrupt semantic shifts.
• Hierarchical Execution: For each plan, the executor produces a shared context up to the step boundary, then generates $N$ future continuations conditioned on the new step prompt. This nested group sampling supports localized and plan-level credit assignment.

  Figure 2: Schematic of TempAct’s nested planner–executor rollout and group-relativity structure enabling hierarchical, localized RL updates.

RL Objective and Credit Assignment

Plans and executions are scored at multiple levels:

• Planner-level reward ($R_\mathrm{plan}$): Combines structured LLM evaluation of plan faithfulness, coverage, and hallucination with VLM-based video-level temporal following, physical consistency, and prompt alignment scores.
• Executor-level reward ($R_\mathrm{exec}$): Aggregates local step-following (VLM) and PickScore-based aesthetics, computed exclusively on the first chunk after each prompt switch to maximize relevance and assignment precision.

Update rules leverage sequence-level GSPO for the planner and Flow-GRPO (with regulated ratio normalization and KL penalties) for the executor, ensuring both sample-efficient and robust policy improvement at each level.

Experimental Results

TempAct is instantiated on Self-Forcing and LongLive backbones, with RL training over 5k temporally structured instructions. Evaluation focuses on both general generation metrics (VBench, PickScore) and fine-grained temporal instruction following (Temporal-Following Score) using multiple VLM judges, including Qwen3-VL-8B and Gemini-3-Flash.

Key Results:

• Temporal-Following Score: On Self-Forcing, TempAct improves the average score from 0.400 (step-prompt) to 0.462 (TempAct), a +15.5% relative gain, with consistent improvements across both in-domain and out-of-domain VLMs and on both simple and hard temporal instructions.
• Preservation of Visual Quality: Visual quality (VBench, PickScore) is fully preserved, demonstrating that temporal RL can improve sequencing without regressing overall aesthetics or fidelity.

Qualitative analysis shows that TempAct robustly disentangles sub-events, eliminates blending errors, and produces more faithful event progression not achievable by either single-prompt or naive step-prompt settings.

Figure 3: Qualitative comparison on temporally ordered prompts—Self-Forcing backbone.

Figure 4: Qualitative comparison on temporally ordered prompts—LongLive backbone.

Ablation and Analysis

Detailed ablations confirm the complementarity of hierarchical RL:

• Component-wise analysis: Joint planner–executor optimization yields the highest temporal consistency; either in isolation yields limited gains.
• Prompt smoothing: Critical for AR stability, as naive prompt switches result in large performance degradation.
• Reward design: Localized (step-level) rewards for the executor significantly outperform global (full-video) rewards regarding both temporal plausibility and sample efficiency.

Further, user studies and VLM–human agreement analyses confirm that Temporal-Following Score improvements map to real human judgments (81% agreement with Gemini-3-Flash evaluations).

Theoretical and Practical Implications

TempAct's contribution lies in demonstrating that:

• Hierarchical RL frameworks with explicit separation of high-level temporal planning and low-level AR execution provide improved credit assignment and decompositional robustness for complex temporal instructions in generative video models.
• VLM feedback can serve as a reliable reward signal, and group-relative policy optimization can be adapted to complex video RL settings.

Practically, TempAct enables instruction-following AR video models to faithfully realize multi-stage event sequences required in interactive and streaming applications, without sacrificing general output quality. The hierarchical composition also facilitates modular improvement and interpretability in policy structure.

Limitations and Future Directions

While TempAct removes major sources of temporal confusion, some pathologies—such as visually implausible transitions and accumulated AR artifacts—persist due to executor limitations. The fixed allocation of temporal spans by the planner is also suboptimal for variable-duration actions. Extension to duration-adaptive or closed-loop (VLM-observant) planning is a natural next step, potentially increasing both adaptability and long-horizon consistency.

Conclusion

TempAct establishes a hierarchical planner–executor RL framework as a superior approach for temporally plausible autoregressive video generation. Empirical improvements in temporal following, validated by VLMs and humans, confirm the effectiveness of hierarchical decomposition and localized reward design. These findings indicate that coupling explicit temporal planning and robust AR execution with multi-level RL is a fruitful direction for scalable, instruction-following video synthesis.