Improving Action Controllability in Generative World Models

Determine principled techniques to increase action controllability in high-capacity generative world models for real-world robotic control, ensuring that predicted future outcomes reliably and accurately follow diverse conditioning action sequences across manipulation tasks.

Background

The paper discusses the need for world models that can serve as interactive environments for reinforcement learning in robotics. While recent generative models have substantially improved visual realism, a key control requirement is that predicted futures must adhere to the conditioning actions, which is crucial for policy improvement and reliable imagined rollouts.

The authors highlight that, despite advances, achieving strong action controllability in such models is still unresolved. Their proposed framework (RISE) addresses aspects of control through task-centric training and a compositional design, but the broader challenge of robust, general action controllability across varied tasks is explicitly identified as open.

References

Despite the improved visual realism by integrating high-capacity generative models, how to improve controllability over various actions remains an open problem.

— RISE: Self-Improving Robot Policy with Compositional World Model  (2602.11075 - Yang et al., 11 Feb 2026) in Section 1 Introduction

Future work can test whether using ACPC during planning improves task success.

— Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency  (2608.12939 - An et al., 13 Aug 2026) in Section 5, paragraph “Future work”

When does action conditioning support counterfactual reasoning? Action-conditioned generation is not equivalent to reliable intervention reasoning. Observational data record outcomes only for executed actions, while responses to unchosen actions remain unobserved. Counterfactual claims therefore require explicit causal assumptions, uncertainty bounds, and distributional limits, supported where possible by interactive simulation, controlled experiments, natural experiments, or learned response models.

— Rethinking World Models for Safety-Critical Embodied Systems  (2609.03774 - Ma et al., 3 Sep 2026) in Section “Open challenges and outlook,” subsection “Open challenges”