EBT-Policy: Energy Unlocks Emergent Physical Reasoning Capabilities (2510.27545v1)

Abstract: Implicit policies parameterized by generative models, such as Diffusion Policy, have become the standard for policy learning and Vision-Language-Action (VLA) models in robotics. However, these approaches often suffer from high computational cost, exposure bias, and unstable inference dynamics, which lead to divergence under distribution shifts. Energy-Based Models (EBMs) address these issues by learning energy landscapes end-to-end and modeling equilibrium dynamics, offering improved robustness and reduced exposure bias. Yet, policies parameterized by EBMs have historically struggled to scale effectively. Recent work on Energy-Based Transformers (EBTs) demonstrates the scalability of EBMs to high-dimensional spaces, but their potential for solving core challenges in physically embodied models remains underexplored. We introduce a new energy-based architecture, EBT-Policy, that solves core issues in robotic and real-world settings. Across simulated and real-world tasks, EBT-Policy consistently outperforms diffusion-based policies, while requiring less training and inference computation. Remarkably, on some tasks it converges within just two inference steps, a 50x reduction compared to Diffusion Policy's 100. Moreover, EBT-Policy exhibits emergent capabilities not seen in prior models, such as zero-shot recovery from failed action sequences using only behavior cloning and without explicit retry training. By leveraging its scalar energy for uncertainty-aware inference and dynamic compute allocation, EBT-Policy offers a promising path toward robust, generalizable robot behavior under distribution shifts.

• The paper introduces EBT-Policy as an energy-based alternative to diffusion policies, significantly reducing inference steps for robotic manipulation.
• It employs Energy-Based Transformers and adaptive Langevin Dynamics to model an explicit energy landscape, enabling robust and uncertainty-aware control.
• Experimental results demonstrate superior performance in both simulated and real-world tasks, with emergent retry behavior improving recovery from failures.

EBT-Policy: Energy-Based Transformers for Emergent Physical Reasoning in Robotic Control

Introduction and Motivation

EBT-Policy introduces a scalable, energy-based approach to policy learning for robotic manipulation, leveraging Energy-Based Transformers (EBTs) to address the limitations of diffusion-based generative policies. The method is motivated by the need for robust, efficient, and interpretable policies capable of handling multimodal sensory inputs and distributional shifts in real-world environments. Unlike diffusion models, which rely on externally defined noise schedules and suffer from exposure bias and compounding errors, EBT-Policy directly models an explicit energy landscape over action trajectories, enabling equilibrium dynamics and uncertainty-aware inference.

Figure 1: EBT-Policy architecture searches for low-energy action trajectories via energy minimization in joint or Cartesian space.

EBT-Policy Formulation and Architecture

EBT-Policy frames visuomotor control as an energy minimization problem. The policy receives multimodal inputs—RGB frames, proprioceptive states, and optional language instructions—and outputs a sequence of actions by descending the energy landscape defined by $E_\theta(\ell, o_t, a)$. The energy function is parameterized by a Transformer, which enables scalability and expressivity in high-dimensional spaces.

Inference is performed via adaptive Langevin Dynamics, where the action trajectory is iteratively updated:

$$\hat{\mathbf{y}}_{i+1} = \hat{\mathbf{y}}_{i} - \alpha_i \nabla_{\hat{\mathbf{y}}} E_\theta(\mathbf{x}, \hat{\mathbf{y}}_i) + \eta_i$$

Here, $\alpha_i$ is an energy-scaled step size, and $\eta_i$ is annealed Gaussian noise. The number of inference steps is dynamically determined by the energy gradient norm, allowing the model to allocate compute adaptively based on uncertainty.

Figure 2: EBT-Policy interprets per-frame energy as uncertainty, triggering retries after failures and converging to successful plans via energy minimization.

Key architectural and training components include:

• Energy-scaled MCMC step sizes for stable optimization.
• Pre-sample normalization (RMSNorm) to prevent gradient explosion.
• Nesterov-accelerated gradients for efficient traversal of complex energy surfaces.
• Scaled Langevin Dynamics with cosine annealing for robust multimodal sampling.
• Gradient clipping to ensure training stability.

Comparison with Diffusion Policies

Diffusion Policy (DP) models the score (gradient of log-likelihood) via a denoising process, requiring a large number of inference steps and suffering from exposure bias. In contrast, EBT-Policy learns the energy landscape directly, enabling equilibrium dynamics that correct OOD samples and reduce compounding errors.

Empirically, EBT-Policy achieves comparable or superior success rates with orders of magnitude fewer inference steps and training epochs. For example, EBT-Policy converges in as few as 2 inference steps, while DP requires up to 100 steps for similar performance.

Figure 3: EBT-Policy reaches 100% success by epoch 30 with only 2 inference steps, while DP requires 90 epochs and 100 steps.

Emergent Retry and Physical Reasoning

A salient property of EBT-Policy is its emergent retry behavior. When encountering OOD states or failures (e.g., missing a hook during Tool Hang), EBT-Policy leverages its energy landscape to recognize high-uncertainty states and adaptively retry, despite not being explicitly trained for recovery. This contrasts with DP, which fails catastrophically under similar conditions due to compounding error and lack of uncertainty modeling.

Figure 4: EBT-Policy demonstrates emergent retry behavior, recovering from OOD failures, while DP diverges.

This retry capability is attributed to the equilibrium dynamics of the energy function, which naturally guide the policy toward plausible, low-energy solutions even in novel states. The scalar energy serves as both a compatibility measure and an uncertainty signal, enabling interpretable decision-making and dynamic compute allocation.

Experimental Results

EBT-Policy was evaluated on both simulated (robomimic suite) and real-world dual-arm manipulation tasks. Two variants were tested: EBT-Policy-S (simulation, ~30M params) and EBT-Policy-R (real-world, ~100M params, multimodal).

• Simulation (robomimic tasks): EBT-Policy-S outperformed DP on Lift, Can, Square, and Tool Hang, achieving higher success rates with fewer inference steps.
• Real-world tasks: EBT-Policy-R exceeded DP in FoldTowel, PlacePan, and PickAndPlace, demonstrating robustness to sensor noise and environmental variability.

Notably, reducing DP's inference steps from 100 to 10 resulted in a collapse in performance (0% success), while EBT-Policy maintained high success even at 2 steps, underscoring its computational efficiency and reliability.

Implementation Considerations

• Computational Requirements: EBT-Policy is significantly more efficient at inference, requiring up to 50× fewer steps than DP. Training stability is enhanced via gradient clipping and energy-scaled updates.
• Scalability: The Transformer backbone enables scaling to high-dimensional, multimodal inputs, including language and vision.
• Deployment: Dynamic inference allows for real-time adaptation to task complexity and uncertainty, making EBT-Policy suitable for deployment in safety-critical and resource-constrained robotic systems.
• Limitations: Training under highly multimodal action distributions remains challenging; further work is needed to improve mode coverage and optimization stability.

Implications and Future Directions

EBT-Policy demonstrates that energy-based implicit policies can achieve robust, efficient, and interpretable control in both simulated and real-world robotic settings. The emergent retry behavior and uncertainty modeling suggest a path toward embodied reasoning and self-verification in robot policies, moving beyond brittle generative approaches.

Future research should explore:

• Scaling EBT-Policy to larger, more diverse datasets and tasks.
• Integrating richer multimodal inputs (e.g., tactile, audio).
• Advancing optimization schemes for improved stability and multimodal coverage.
• Investigating theoretical properties of energy-based reasoning and its connection to System 2 cognition in embodied agents.

Conclusion

EBT-Policy establishes energy-based minimization as a viable and superior alternative to diffusion-based generative policies for robotic manipulation. By leveraging equilibrium dynamics, uncertainty-aware inference, and dynamic compute allocation, EBT-Policy achieves state-of-the-art performance with substantially lower computational cost. Its emergent retry and physical reasoning capabilities mark a significant step toward unified, interpretable, and robust robot world models under a single energy function.