GrndCtrl: Self-Supervised World Grounding Framework

• GrndCtrl is a self-supervised framework that grounds generative video world models by aligning them with verifiable geometric and temporal invariants.
• It utilizes offline rewards—pose cycle-consistency, depth reprojection, and temporal coherence—and optimizes via Group Relative Policy Optimization (GRPO) to ensure spatial and temporal accuracy.
• Empirical evaluations demonstrate significant reductions in translation error and drift, affirming its efficacy for robust, closed-loop navigation tasks under varied conditions.

GrndCtrl is a self-supervised, reward-aligned post-training framework for grounding generative video world models in physically verifiable geometric and temporal invariants. It addresses the gap between high-fidelity generative world models—which can simulate plausible environmental dynamics visually—and the need for spatially coherent, actionable predictions required by embodied navigation and control. The approach is instantiated within the Reinforcement Learning with World Grounding (RLWG) paradigm, treating the generative world model as a stochastic policy and aligning it via offline geometric and perceptual “verifiable” rewards, analogous to reinforcement learning from verifiable feedback (RLVR) in LLMs. GrndCtrl employs Group Relative Policy Optimization (GRPO) to directly optimize a pretrained world model for improved pose cycle-consistency, depth reprojection, and temporal coherence, thereby yielding world models that maintain stable trajectories, consistent geometry, and reliable long-horizon rollouts in physically grounded navigation tasks (He et al., 1 Dec 2025).

1. The RLWG Paradigm and GrndCtrl Motivation

Reinforcement Learning with World Grounding (RLWG) is formulated to bridge the disconnect between plausible video synthesis and actionable, spatially grounded dynamics. While large-scale generative world models, e.g., diffusion-based video predictors, can generate visually realistic scenes, they often suffer from geometric drift, lack of temporal stability, and unreliable pose estimation—making them unsuitable for policy rollout in navigation tasks.

Drawing direct analogy to RLVR—where a LLM is post-trained by verifiers scoring logical consistency—RLWG treats the world model as the “policy,” with fixed black-box geometric and perceptual evaluators providing continuous, verifiable rewards for the rollouts, completely bypassing human labeling or online environment access. GrndCtrl is an instantiation of RLWG, specifying both the suite of rewards and the optimization method (GRPO) tailored for aligning world model rollouts with physical invariants (He et al., 1 Dec 2025).

2. Self-Supervised Geometric and Perceptual Rewards

GrndCtrl employs three classes of rewards, each computed without ground-truth labels or external simulators, but rather from network-predicted outputs and frozen evaluators:

• Pose Cycle-Consistency Reward ($R_{\rm pose}$): Measures the agreement between the model-commanded SE(3) action $(R_t,\mathbf t_t)$ and the pose deltas $(\Delta R_t, \Delta \mathbf t_t)$ inferred by a frozen 3D evaluator. The translational and rotational deviations are combined:

$$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$

This reward penalizes misalignment in both translation and orientation over a rollout.

• Depth Reprojection Reward ($R_{\rm depth}$): Assesses geometric consistency by back-projecting each frame’s depth $D_t(p)$, transforming with estimated pose, and checking reprojection inliers within a tolerance $\gamma$, summarized as the Depth Temporal Reprojection Inlier ratio (DTRI):

$$R_{\rm depth} = \frac{1}{T}\sum_{t=1}^T \frac{1}{|\Omega|} \sum_{p\in \Omega} \mathbf{1}_{t}(p)$$

where the indicator $\mathbf{1}_{t}(p)$ measures consistency for each pixel.

• Temporal Coherence and Visual Fidelity Reward ($R_{\rm temp}$): Derived from a frozen video evaluator $(R_t,\mathbf t_t)$0, combining motion smoothness and frame-level appearance scores:

$(R_t,\mathbf t_t)$1

This incentivizes rollouts with both plausible movement and visual quality.

The total reward for each rollout is formed by normalizing and summing these components across a group of stochastic samples.

3. Group Relative Policy Optimization (GRPO) for Post-Training

GrndCtrl applies the Group Relative Policy Optimization (GRPO) algorithm to optimize the world model parameters. The process is as follows:

• For each context $(R_t,\mathbf t_t)$2, sample a group of $(R_t,\mathbf t_t)$3 independent rollouts $(R_t,\mathbf t_t)$4 using the (frozen) base model.
• Compute the vector of rewards $(R_t,\mathbf t_t)$5 for each rollout. Normalize each reward across the group to zero-mean and unit-variance, producing an aggregated normalized score $(R_t,\mathbf t_t)$6.
• Compute group-relative advantages $(R_t,\mathbf t_t)$7:

$(R_t,\mathbf t_t)$8

• The clipped GRPO surrogate loss, analogous to PPO, is optimized:

$(R_t,\mathbf t_t)$9

where $(\Delta R_t, \Delta \mathbf t_t)$0 is the per-step likelihood ratio under current and reference parameters, and a KL-penalty towards a periodically frozen reference model is included to stabilize training and maintain sample diversity.

This post-training protocol is conducted entirely offline, without access to ground-truth environment dynamics, relying exclusively on the frozen evaluators and the generative world model.

4. Empirical Evaluation and Comparative Results

The GrndCtrl framework was evaluated on navigation datasets (CODa, SCAND, CityWalk), across both in-distribution (“seen”), counterfactual (action sequence mirrored/inverted), and out-of-distribution (“unseen”) splits. Evaluation metrics include:

• Translation Error ($(\Delta R_t, \Delta \mathbf t_t)$1) and Rotation Error ($(\Delta R_t, \Delta \mathbf t_t)$2): Derived from $(\Delta R_t, \Delta \mathbf t_t)$3, reporting averaged spatial misalignments.
• Depth Reprojection Inlier Ratio (DTRI): Quantifies frame-to-frame geometric consistency.
• Video Quality ($(\Delta R_t, \Delta \mathbf t_t)$4): Averaged $(\Delta R_t, \Delta \mathbf t_t)$5 score.
• All metrics are reported as means over 200 evaluation sequences.

Key findings include:

• Baseline (pretrained) world models exhibit severe trajectory drift and loss of spatial coherence, especially under counterfactual interventions (e.g., $(\Delta R_t, \Delta \mathbf t_t)$6 translation error).
• Adding the pose reward alone reduces translation error by up to $(\Delta R_t, \Delta \mathbf t_t)$7 on counterfactual tests.
• Full composite rewards (pose, depth, video) yield the most robust rollouts: in CODa, up to $(\Delta R_t, \Delta \mathbf t_t)$8 reduction in translation error even on unseen scenes with minor visual fidelity degradation.
• Ablation confirms that each reward term adds incremental spatial or temporal coherence.
• Qualitative results show that GrndCtrl-aligned rollouts maintain stable paths and avoid divergence under mirrored actions, whereas baseline models consistently exhibit drift and lose geometric alignment.

5. Workflow and Implementation Protocol

The GrndCtrl post-training implementation can be summarized as:

1. Start from a pretrained world model $(\Delta R_t, \Delta \mathbf t_t)$9 (e.g., diffusion-based video predictor).
2. Freeze the geometric ($$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$0) and video ($$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$1) evaluators.
3. For each training minibatch:
   • Sample conditioning context $$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$2.
   • Draw $$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$3 stochastic rollouts via nonzero diffusion noise.
   • Compute all rewards and normalize within the group.
   • Backpropagate the GRPO loss on the first $$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$460% of diffusion steps for stability.
4. Typical hyperparameters: group size $$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$5, 100–200 GRPO iterations, batch size 1 per GPU, 8 GPUs, diffusion steps 35, clip parameter $$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$6, learning rate $$R_{\rm pose}(\hat x_{1:T}) = -\left[\frac{1}{T}\sum_{t=1}^T \| \Delta \mathbf t_t - \mathbf t_t \|_2^2 + \sum_{t=1}^T \arccos\left(\frac{\operatorname{tr}(\Delta R_t R_t^\top) - 1}{2}\right)\right]$$7.

No pixel-level reconstruction loss is retained after the initial pretraining: GrndCtrl strictly aligns latent and perceptual structure via verifiable rewards.

6. Limitations and Future Directions

• Variance Sensitivity: GRPO requires a spectrum of rollout qualities within each group. Homogenously poor (or perfect) rollouts can destabilize the advantage computation, though checkpointing and KL-regularization offer partial mitigation.
• Accumulation of Pixel Noise: Absence of pixel reconstruction results in noisy outputs during long rollouts; further regularization or hybrid pixel/geometric losses may be necessary for extreme horizons.
• Reward Hacking: Black-box evaluators can be exploited by pathological outputs that score well under geometric/perceptual metrics despite semantic errors. Robustness to such cases is an open problem.
• Reward Weighting: Fixed weights may not provide optimal trade-offs; adaptive or learned weighting (e.g., Lagrangian multipliers) could better balance global and local structure.
• Task Generality: Extending RLWG to non-navigation tasks (e.g., manipulation, drone flight) will require design of new verifiable rewards reflecting task-specific constraints such as contact force consistency, object permanence, or collision avoidance.

GrndCtrl demonstrates that post-training large video world models via self-supervised geometric and perceptual alignment can produce models suitable for long-horizon, closed-loop planning and control in real-world and counterfactual settings—all without supervised labels or explicit environment simulation (He et al., 1 Dec 2025). A plausible implication is that as world models scale in pretraining capacity, complementary grounding via RLWG/GrndCtrl-style reward alignment will become essential for embodied utility.