BarlowWalk: Terrain-Adaptive Biped Locomotion

• The paper introduces BarlowWalk, a method combining PPO with the Barlow Twins algorithm for efficient terrain-adaptive bipedal control.
• The methodology decouples latent proprioceptive features from explicit terrain perception, enabling a self-supervised loss that promotes temporal continuity.
• Experimental results indicate accelerated mid/late convergence and enhanced stability across varied terrains through an asymmetric actor-critic design.

BarlowWalk is a legged-locomotion method that integrates self-supervised representation learning into Proximal Policy Optimization (PPO) for terrain-adaptive bipedal control. It is defined as an improved PPO method in which the Barlow Twins algorithm constructs a decoupled latent space from historical proprioceptive observations, so that the actor can rely on proprioceptive information together with a learned latent representation rather than explicit external terrain perception. The method is positioned against mainstream reinforcement-learning pipelines for bipedal robot terrain traversal, particularly teacher-student policy knowledge distillation, which are described as suffering from long training times that limit development efficiency (Huang et al., 31 Jul 2025).

1. Formalization of the locomotion task

BarlowWalk models bipedal locomotion as a partially observed Markov decision process (POMDP),

$$(\mathcal{S},\;\mathcal{O},\;\mathcal{A},\;\mathcal{R},\;p,\;\gamma),$$

where $\mathcal{S}$ is the hidden true state space, $\mathcal{O}\subset\mathbb{R}^n$ is the observation space accessible to the agent, $\mathcal{A}=\mathbb{R}^8$ denotes joint-torque commands for the two legs, $\mathcal{R}$ is the scalar reward, $p(s_{t+1}\mid s_t,a_t)$ is the transition kernel, and $\gamma\in[0,1]$ is the discount factor. At each time step, the agent observes $o_t\in\mathcal{O}$, samples $a_t\sim\pi_\theta(a_t\mid o_t)$, transitions to $s_{t+1}\sim p(\cdot\mid s_t,a_t)$, and receives reward $\mathcal{S}$0. The optimization target is the expected discounted return

$\mathcal{S}$1

Terrain variability is handled through two mechanisms. First, the training process uses a curriculum over terrain families arranged in increasing difficulty, including slopes, rough slopes, stairs up/down, and discrete obstacles. Second, domain parameters such as mass, friction, restitution, and PD gains are randomized during training to improve sim-to-real robustness. This formulation makes explicit that the control problem is partially observed and that terrain adaptation is not treated as a separate perception module but as part of the policy-learning problem itself (Huang et al., 31 Jul 2025).

2. Self-supervised representation learning with Barlow Twins

The self-supervised component forms two views of the same proprioceptive history. The first, obs_old_hist, is the 10-step history ending at $\mathcal{S}$2; the second, obs_new_hist, is the 10-step history ending at $\mathcal{S}$3. Both exclude linear velocity. Each view is passed through shared MLP and latent encoders to produce batch features $\mathcal{S}$4 and $\mathcal{S}$5 of dimension $\mathcal{S}$6, which are standard-normalized across the minibatch. The cross-correlation matrix is then computed as

$\mathcal{S}$7

The Barlow Twins objective is

$\mathcal{S}$8

No explicit negative pairs are required; decorrelation is enforced through the off-diagonal term.

The architecture used for historical proprioceptive feature extraction has three stages: MLP_enc, with two fully connected layers of sizes $\mathcal{S}$9 and ReLU activations; latent_enc, with two FC layers $\mathcal{O}\subset\mathbb{R}^n$0 and ReLU; and Barlow_enc, with two FC layers $\mathcal{O}\subset\mathbb{R}^n$1 used to compute $\mathcal{O}\subset\mathbb{R}^n$2. For a window of length 5 steps, specifically obs_new_hist[5:10], the representation pipeline is

$\mathcal{O}\subset\mathbb{R}^n$3

Temporal continuity is encouraged by correlating $\mathcal{O}\subset\mathbb{R}^n$4 with $\mathcal{O}\subset\mathbb{R}^n$5 via the Barlow Twins loss. In the formulation given for BarlowWalk, this forces adjacent-time representations to maintain high autocorrelation on each feature dimension while minimizing cross-feature correlation. The resulting latent space is therefore decoupled and low-dimensional, and the method implements self-supervision directly over continuous time steps rather than through externally labeled terrain categories (Huang et al., 31 Jul 2025).

3. PPO integration and the asymmetric actor-critic design

BarlowWalk combines PPO with the self-supervised loss by optimizing a total objective

$\mathcal{O}\subset\mathbb{R}^n$6

where $\mathcal{O}\subset\mathbb{R}^n$7 is the actor’s clipped surrogate loss. The training loop initializes the policy $\mathcal{O}\subset\mathbb{R}^n$8, critic $\mathcal{O}\subset\mathbb{R}^n$9, and the MLP, latent, and Barlow encoders; gathers rollouts; computes advantages $\mathcal{A}=\mathbb{R}^8$0 with $\mathcal{A}=\mathbb{R}^8$1; updates the policy and Barlow encoders using minibatches; computes $\mathcal{A}=\mathbb{R}^8$2, the cross-correlation matrix $\mathcal{A}=\mathbb{R}^8$3, and $\mathcal{A}=\mathbb{R}^8$4 over corresponding $\mathcal{A}=\mathbb{R}^8$5; and updates the critic by minimizing TD loss. All parameters of MLP_enc, latent_enc, and the policy $\mathcal{A}=\mathbb{R}^8$6 are updated jointly under the gradient of the total loss.

A central architectural feature is the asymmetric information structure. The actor receives the current observation excluding linear base velocity $\mathcal{A}=\mathbb{R}^8$7 together with the latent representation $\mathcal{A}=\mathbb{R}^8$8, for a total input dimension of 51. No explicit terrain, height-field, or velocity from external sensors is used by the policy $\mathcal{A}=\mathbb{R}^8$9. The critic, by contrast, receives the full 225-dimensional observation, including privileged terrain height-map information. This is an important clarification because the phrase “proprioceptive-only actor” does not imply that privileged information is absent from training; rather, privileged information is reserved for the critic while the deployed policy remains independent of external terrain sensing. The paper characterizes this design as substantially reducing reliance on external perception at deployment (Huang et al., 31 Jul 2025).

4. Reward shaping and terrain-adaptive behavior

The per-step reward is composed of tracking, action, and constraint terms. The summary gives the structure as

$\mathcal{R}$0

with specific components listed in Table 4 of the paper. The linear-velocity tracking term is

$\mathcal{R}$1

and the angular-velocity tracking term is

$\mathcal{R}$2

Additional terms include action smoothness,

$\mathcal{R}$3

a body-height penalty,

$\mathcal{R}$4

an orientation penalty,

$\mathcal{R}$5

and a feet-clearance bonus,

$\mathcal{R}$6

The reward also includes additional action-cost and safety-constraint terms such as torque cost and contact forces.

Terrain adaptation is described as emerging from the combination of these dense rewards with curriculum switching among six terrain types of graded difficulty. This suggests that BarlowWalk does not isolate terrain adaptation in a separate module; rather, adaptation is an emergent property of jointly optimizing locomotion objectives, temporal representation consistency, and exposure to increasingly difficult terrain families during training (Huang et al., 31 Jul 2025).

5. Experimental configuration and baselines

The reported simulated terrains are rough ground, slope-up, slope-down, stairs-up, stairs-down, and discrete obstacles. Training uses a curriculum in which agents encounter progressively harder terrains. The baseline methods are SLR, described as a privileged-information quadruped baseline adapted to TRON1, and a second baseline with no privileged information and only a five-step proprioceptive MLP, without the Barlow module.

The key hyperparameters listed in Table 3 are: batch size $\mathcal{R}$7 steps, clipping parameter $\mathcal{R}$8, $\mathcal{R}$9, GAE $p(s_{t+1}\mid s_t,a_t)$0, PPO epochs $p(s_{t+1}\mid s_t,a_t)$1, and Barlow $p(s_{t+1}\mid s_t,a_t)$2. Training time is reported as 4 hours on RTX 4090 with 4096 parallel agents in IsaacGym. These details locate BarlowWalk within a large-scale simulation regime rather than a low-data robotic setting, and they also identify the computational context in which the reported convergence behavior should be interpreted (Huang et al., 31 Jul 2025).

6. Reported results, ablations, and latent-space structure

The reported results indicate success up to terrain difficulty level 6/10. Over the first 50 steps, BarlowWalk slightly outperforms SLR in linear-error RMSE and matches it in angular-error, whereas Baseline 2 exhibits much larger tracking error. The cumulative reward curves show faster mid/late convergence when the Barlow Twins term is included.

The ablation study distinguishes the optimization behavior with and without $p(s_{t+1}\mid s_t,a_t)$3. Without $p(s_{t+1}\mid s_t,a_t)$4, early reward grows faster but plateaus lower and less stably. With $p(s_{t+1}\mid s_t,a_t)$5, reward climbs steadily after approximately 2000 steps and converges to a higher value. Terrain-level progression analysis shows that the BT-enabled policy reaches Level 6 sooner and remains stable. A t-SNE visualization of the latent space yields distinct clusters for incline versus stairs, which the summary interprets as semantically meaningful embeddings.

Taken together, these findings support a specific characterization of the learned representation: it is not merely compressive, but temporally coherent and terrain-informative in a way that correlates with improved convergence during the middle and late phases of training. A plausible implication is that the self-supervised loss regularizes policy learning even when it does not maximize reward growth in the earliest training stage (Huang et al., 31 Jul 2025).

7. Scope, limitations, and projected extensions

The strengths listed for BarlowWalk are one-stage end-to-end training without teacher/student distillation, a proprioceptive-only actor that can be deployed without terrain sensors, compact and temporally coherent latent features produced by self-supervised Barlow Twins learning, accelerated mid/late training, and robust performance on six complex terrains and in hardware tests on TRON1. At the same time, the limitations are explicit: all training was performed in simulation; real-world transfer remains to be fully validated; the method is restricted to walking; transitions such as running and jumping are not addressed; and the fixed 10-step history may be insufficient for longer-term dependencies such as stepping patterns.

The future directions listed in the summary are correspondingly targeted. They include deployment and fine-tuning on physical biped platforms with sim-to-real techniques such as reinforcement-learning fine-tuning, extension of the self-supervised module to multimodal inputs including vision and contact sensing, incorporation of richer temporal models such as Transformers or time-series VAEs to capture gait periodicity over longer windows, and adaptation of the framework to dynamic tasks including trotting, running, and recovery from pushes. These directions indicate that BarlowWalk is presented not as a complete solution to legged mobility, but as a specific synthesis of PPO and self-supervised temporal representation learning whose main contribution lies in terrain-adaptive walking under an asymmetric, proprioception-centered control design (Huang et al., 31 Jul 2025).