FFDC-WAM: Adaptive Robot Action Execution

• FFDC-WAM is a framework for adaptive action execution that verifies future predictions through causal attention to align imagined and real observations.
• It integrates World Action Models with a dynamic verifier to adjust rollout lengths in robotic manipulation, optimizing efficiency and robustness.
• Experimental results in both simulation and real-world settings show improved success rates and reduced computational load compared to fixed-horizon methods.

The Future Forward Dynamics Causal Attention for World Action Models (FFDC-WAM) framework is a method for adaptive action execution within robotic manipulation tasks, specifically designed to address the challenge of maintaining alignment between predicted and actual sequences of future states and actions. By introducing a principled mechanism for future–reality verification, FFDC-WAM enables robots to dynamically adjust the length of action rollouts, improving the efficiency and robustness of long-horizon execution under uncertainty. The framework combines World Action Models (WAMs), which jointly model future visual and action trajectories, with a causal-attention-based verifier that adaptively determines when to trust or abort a model-predicted plan segment based on observation consistency.

1. World Action Models: Joint Future Prediction and Fixed-Horizon Limitations

World Action Models (WAMs) are designed to model the conditional distribution

$p(O_{t+1:t+H}, A_{t+1:t+H} \mid o_t, \ell)$

where $O_{t+1:t+H}$ are future visual tokens, $A_{t+1:t+H}$ are future actions, $o_t$ is the current observation, and $\ell$ represents instruction semantics. WAMs are trained on video–action trajectories using the sum of an action flow-matching loss and a video flow-matching loss:

$$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$

At inference, the WAM predicts a fixed chunk of length $H$:

$$(\hat A_{t+1:t+H}, \hat O_{t+1:t+H}) = \pi_\theta(o_t, \ell)$$

with all $H$ actions then executed in open-loop before inferring again. This approach introduces a trade-off: smaller $H$ increases robustness but incurs computational overhead, while larger $O_{t+1:t+H}$0 improves efficiency but is brittle in distributional shift or contact-rich phases where open-loop predictions quickly diverge from reality. This undermines both reliability and computational efficiency.

2. Future–Reality Verification: Formulation and Decision Process

FFDC-WAM recasts adaptive execution as a future–reality verification problem. Each WAM rollout generates not only future actions but also the corresponding latent visual tokens (“imagined video”). After executing $O_{t+1:t+H}$1 actions, the robot compares the actual observation $O_{t+1:t+H}$2 against the aligned WAM-predicted latent frame with the pending predicted actions.

A transformer-based verifier $O_{t+1:t+H}$3 estimates a scalar trust score $O_{t+1:t+H}$4:

$O_{t+1:t+H}$5

A threshold $O_{t+1:t+H}$6 partitions the decision: $O_{t+1:t+H}$7 continues execution; $O_{t+1:t+H}$8 triggers model replanning. The emergent executed chunk size is the number of steps before the verifier’s score drops below $O_{t+1:t+H}$9. This mechanism allows the robot to adaptively lengthen or shorten WAM execution segments in response to the ongoing consistency between prediction and reality.

3. FFDC Module Architecture: Inputs, Causal Attention, and Integration

Inputs to the Verifier

• Real observation token at current step: $A_{t+1:t+H}$0
• Predicted future action segment: $A_{t+1:t+H}$1
• Latent video tokens:
  • Past window: $A_{t+1:t+H}$2
  • Future window: $A_{t+1:t+H}$3, where $A_{t+1:t+H}$4 is action-to-video stride
• Instruction semantic tokens: $A_{t+1:t+H}$5
• A learnable [CLS] token for global aggregation

The verifier input sequence is

$A_{t+1:t+H}$6

Causal-Attention Transformer Architecture

$A_{t+1:t+H}$7 is realized as an $A_{t+1:t+H}$8-layer transformer over $A_{t+1:t+H}$9, employing standard multi-head attention. Temporal-causal constraints are enforced by a Boolean mask $o_t$0:

• $o_t$1 and $o_t$2 attend to all tokens.
• Each future-video token attends only to $o_t$3.
• Each future-action token attends to $o_t$4. Sliding window constraints further restrict token access to local neighborhoods for computational efficiency. After $o_t$5 layers, the hidden state of [CLS] is used:

$o_t$6

$o_t$7

with $o_t$8 denoting the sigmoid.

Integration into the Inference Loop

The FFDC module is integrated as follows:

1. Obtain $o_t$9.
2. Run WAM inference: $\ell$0.
3. Cache predicted and semantic tokens.
4. Execute predicted actions one by one, periodically computing $\ell$1.
5. If $\ell$2, halt and replan. This adaptive chunking emerges directly from the interplay between model trust and observation.

4. Mixture-of-Horizon Training and Losses

Sampling Procedure

During training, to ensure coverage across all horizon lengths and episode stages:

• Condition on a uniformly-random index $\ell$3.
• Randomize horizon $\ell$4 from $\ell$5 (e.g., $\ell$6).
• Action and video target indices built as:
  • $\ell$7, $\ell$8
  • $\ell$9, $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$0
  • Yielding training targets $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$1, $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$2. This “mixture-of-horizon” strategy exposes the model to diverse rollout lengths, increasing stability and generalization.

Optimization

• WAM: $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$3.
• Verifier: Binary executability classification on segments:

$$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$4

Valid segments from demonstrations or successful rollouts are labeled $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$5, corrupted or failure-inducing ones as $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$6.

5. Empirical Evaluation

Simulated RoboTwin Benchmark

On a suite of 50 multi-task manipulation scenarios (including domain-randomized variations), FFDC-WAM was evaluated against fixed-chunk and chunked baselines:

Method	Success Rate (SR)	Task Time (T, s)	WAM Calls
Base-Motus	85.66%	24.4	5.47
FFDC-WAM	88.20%	16.1	1.69

FFDC-WAM reduced forward passes by 69.1%, execution time by 34.0%, and improved average SR by 2.54 percentage points relative to the short-chunk baseline. On the subset of “hard” tasks ($$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$7), SR increased from $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$854% to 76%; on “easy” tasks, peak SR was retained while halving execution time.

Real-World Testing

Two pick-and-place tasks on the Astribot S1 manipulator demonstrated transfer: baseline LC-16 vs. FFDC-WAM (check interval $$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$9):

Method	Success Rate (SR)	Task Time (T, s)	WAM Calls
LC-16	45%	25.6	14
FFDC-WAM	80%	28.1	16

FFDC-WAM improved robustness ($H$0SR = +35 pp), executing more successful trials under perceptual and system noise, with only a modest increase in average inference calls.

Ablation Findings

Ablation experiments established that removal of any among predicted visual tokens, predicted actions, real observations, or semantic instruction tokens degraded success rate. The largest drop ($H$15pp) occurred when omitting predicted visuals. This underscores the importance of joint reasoning over all four input modalities for verification fidelity.

6. Summary and Implications

FFDC-WAM advances robotic manipulation by converting fixed-horizon action execution into an adaptive, observation-aware process. The causal-attention verifier enables real-time trust assessment between imagined and real world trajectories, adaptively resizing action chunks—a capability that preserves computational efficiency during reliable phases while enforcing rapid replanning when inconsistencies arise. Mixture-of-horizon training promotes generalization across task durations and transition points.

Empirical benchmarks demonstrate notable gains in the robustness-efficiency Pareto frontier: lower computational burden and improved task reliability in both simulation and real robotic settings. The architecture’s dependency on all four input streams for predictive fidelity suggests pathways for further exploration of cross-modal verification in sequential decision making. FFDC-WAM’s principled future–reality verification loop represents a substantive step for adaptive robot autonomy under model imperfection and environmental uncertainty (Wang et al., 7 May 2026).