AME-2: Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding

Abstract: Achieving agile and generalized legged locomotion across terrains requires tight integration of perception and control, especially under occlusions and sparse footholds. Existing methods have demonstrated agility on parkour courses but often rely on end-to-end sensorimotor models with limited generalization and interpretability. By contrast, methods targeting generalized locomotion typically exhibit limited agility and struggle with visual occlusions. We introduce AME-2, a unified reinforcement learning (RL) framework for agile and generalized locomotion that incorporates a novel attention-based map encoder in the control policy. This encoder extracts local and global mapping features and uses attention mechanisms to focus on salient regions, producing an interpretable and generalized embedding for RL-based control. We further propose a learning-based mapping pipeline that provides fast, uncertainty-aware terrain representations robust to noise and occlusions, serving as policy inputs. It uses neural networks to convert depth observations into local elevations with uncertainties, and fuses them with odometry. The pipeline also integrates with parallel simulation so that we can train controllers with online mapping, aiding sim-to-real transfer. We validate AME-2 with the proposed mapping pipeline on a quadruped and a biped robot, and the resulting controllers demonstrate strong agility and generalization to unseen terrains in simulation and in real-world experiments.

• The paper presents the integration of attention-based neural map encoding within a reinforcement learning framework, achieving agile and generalized control for legged robots.
• The system fuses local and global terrain features using multi-head attention to handle sparse footholds, occlusions, and uncertain sensor inputs in real time.
• Extensive experiments on quadrupeds and bipeds validate real-time performance and robust zero-shot generalization on complex terrains.

Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding: Expert Overview

Introduction and Motivation

AME-2 presents a reinforcement learning (RL) framework for perceptive legged locomotion, targeting the dual challenge of achieving high agility and strong generalization across diverse and previously unseen terrains. Previous methods typically compromise between agility, generalization, interpretability, and computational efficiency. Classical approaches relying on explicit mapping and model-based planning provide generalization but lack the real-time agility required for complex terrains due to their computational burden and reliance on brittle pipeline elements. Conversely, end-to-end RL-based methods often demonstrate remarkable agility in structured environments but fail to generalize and provide little interpretability due to the implicit encoding of terrain information.

AME-2’s central contribution is the integration of a novel attention-based neural map encoder (AME-2) within the policy architecture, combined with a fast, uncertainty-aware, learning-based mapping pipeline. This combination enables onboard real-time execution (Figure 1) and supports sim-to-real transfer by using identical mapping in both domains.

Figure 1: Our method enables agile and generalized legged locomotion across diverse terrains with onboard sensing and computation.

System Architecture

The framework consists of two principal modules: the mapping pipeline and the RL-based controller with the AME-2 encoder. The system operates as a structured modular pipeline—learning-based, yet departing from monolithic sensorimotor approaches by clearly separating mapping and control stages while allowing full end-to-end policy optimization (Figure 2).

Figure 2: Overview: RL trains a teacher policy with ground-truth maps in simulation, and a student under the neural mapping; both output joint-level actions for goal-reaching.

Attention-Based Map Encoder (AME-2)

The AME-2 encoder replaces generic CNN/MLP-based encoders with a design that explicitly extracts both global and local map features, using attention mechanisms conditioned on proprioception and the global terrain context. This attention enables the policy to prioritize terrain regions salient for the current locomotion task, thus facilitating adaptation to unseen terrains. The policy accepts proprioceptive inputs and the egocentric elevation map (with uncertainty) and produces joint PD targets at 50 Hz (Figure 3).

Figure 3: AME-2 policy architecture: Proprioceptive and mapping features are fused with attention to produce context-sensitive embeddings for control.

AME-2’s hierarchical representation—local features via CNN/MLP, global features via max pooling—serves as the foundation for multi-head attention, enhancing the model’s sensitivity to sparse footholds and occlusions, critical in challenging terrains.

Learning-Based Uncertainty-Aware Mapping

The mapping module predicts elevation and associated uncertainty from raw depth input in real time using a lightweight CNN (Figure 4). Predictions are fused into a persistent global map using odometry, with explicit modeling of uncertainty and probabilistic winner-take-all fusion, preventing spurious overconfidence due to repeated observations in occluded zones.

Figure 4: Mapping pipeline projects each depth frame into local grids, predicts elevations and uncertainties, fuses these into a global map, and queries local regions for control.

Training the mapping model leverages procedurally synthesized terrains and heavy noise/data augmentation for robustness and generalization, decoupling deployment-time performance from the terrain distribution observed during controller training and ensuring the pipeline can support thousands of parallel RL environments in simulation.

Training Protocol and Deployment

The system employs an asymmetric actor-critic RL formulation with a teacher-student paradigm. The teacher is trained with ground-truth maps for sample efficiency, while the deployable student is trained with the real mapping module, supervised both via RL and distillation from the teacher. Robust sim-to-real transfer is achieved through domain randomization and noise augmentation at both the dynamics and observation levels.

The reward structure is task-driven and terrain-agnostic, prioritizing goal-reaching with head orientation and stability regularization. Emergent behaviors (whole-body contacts, impact mitigation, and active perception) arise in the absence of explicit foot contact shaping, facilitating adaptive strategies on challenging topologies.

Experimental Results

Agility and Generalization

AME-2 demonstrates state-of-the-art capability: the ANYmal-D quadruped and TRON1 biped, trained only on primitive terrains, exhibit zero-shot traversal of previously unencountered, highly-challenging parkour and rubble terrains. Both show peak forward velocities above 1.5 m/s and demonstrate complex behaviors such as climbing, leaping, and balancing on sparse footholds under significant perception degradation.

Figure 5: ANYmal-D zero-shots the hardest parkour/rubble terrains reported in prior work.

Robustness and Emergence

On real hardware, the system achieves robust performance across wide terrain diversity—narrow beams, stepping stones, floating elements—and maintains stability on mutable (e.g., moving or tilting) surfaces (Figure 6).

Figure 6: Controllers are robust to moving terrains; recovery and stabilization are achieved via emergent whole-body contact strategies.

The explicit modeling of mapping uncertainty results in conservative, robust decision-making in occluded regions; active perception emerges as the policy learns to gather information through purposeful interactions (Figure 7). Whole-body behaviors—e.g., knee use during climbing/descending and impact reduction—arise strictly from reward shaping and hardware-safe early termination criteria, not via explicit motion templates.

Mapping Quality and Interpretability

Qualitative analyses highlight the superiority of the neural mapping pipeline relative to recurrent memory-based alternatives, with the former assigning high uncertainty to occlusions (rather than overfitting to priors) and better preserving fine-grained terrain features (Figure 8).

Figure 8: Comparison: our pipeline accurately predicts terrain elevations and uncertainties, whereas the temporal recurrent model overfits and loses details on novel scenes.

Inspection of attention and feature patterns within the policy demonstrates selective focus on terrain characteristics critical for locomotion—attention weights focus dynamically on supports, edges, and gaps—explaining the observed generalization to novel terrain composites (Figure 9).

Figure 9: Visualization of neural feature patterns; local attention tracks fine structure, global features select terrain-defining elements.

Ablation and Benchmarking

Quantitative ablation experiments substantiate the architectural choices:

• AME-2 outperforms AME-1 and mixture-of-experts (MoE) baselines, especially on complex and composite unseen terrains.
• The combination of RL and teacher supervision (action and representation losses) for the student is critical—naive distillation or omission of RL leads to marked generalization gaps.
• The mapping pipeline’s use of extensive terrain diversity in training and robust uncertainty modeling yields the lowest $L_{0.5}$ errors and highest student policy success rates on test sets.
• The system displays resilience under increased sensor noise, missing data, and camera failures, with performance degradation localized to extreme occlusion or perception loss.

Implications and Future Directions

AME-2 demonstrates that modular, interpretable neural encoding of spatial context—bridging model-based generalization and end-to-end agility—can break the trade-off dominating previous approaches. Explicit uncertainty modeling and structured attention are essential for robust deployment in unstructured, partially observed environments.

Practically, this enables broader deployment of legged robots in real-world scenarios with on-the-fly adaptation, safety, and minimal environment-specific customization. Theoretically, AME-2 supports the case for structured, attention-based feature fusion as a design principle for robot perception–control interfaces, especially where generalization beyond the training domain is paramount.

Looking forward, several open questions and limitations exist:

• Extending the mapping and policy to fully 3D volumetric representations for environments with overhangs or non-elevation topologies.
• Scaling emergent contact patterns for humanoids or high-DoF systems to support multi-modal whole-body interactions.
• Integrating semantic awareness for higher-level navigation and task planning.
• Establishing mechanisms for fully scalable zero-shot generalization for complex task/terrain transitions.

Conclusion

AME-2 advances the state of RL-based perceptive locomotion by integrating attention-based map encoding and uncertainty-aware neural mapping into a unified, deployable framework. The result is a highly agile, robust, and generalized controller for legged systems, validated across extensive simulation and real-world experiments. The approach exemplifies how modular neural architectures—when carefully designed for interpretability and uncertainty—enable the next generation of adaptive, scalable robot control.