Demonstration-Guided Humanoid Stand-Up on an Emulated Deformable Surface

Abstract: This paper presents a reference-guided reinforcement learning framework to generate stand-up motion for a 29-DOF Unitree G1 humanoid on deformable soft ground, using a human demonstration recorded on hard ground. The terrain compliance is modelled using solref and solimp parameters from MuJoCo's rigid body soft-contact model. The rewards consists of (i) reference motion tracking through residual joint-position control and (ii) explicit recovery objectives such as pelvis height, torso uprightness, and the final posture. First, the policy is trained with the specified rewards considering hard ground. Next, the terrain stiffness is lowered by updating solref and the nominal surface penetration zone is expanded using solimp. Subsequent training enables the policy to adapt to the delayed support force generation due to significant surface penetration during contact-intensive phases while preserving the original demonstration pattern. The learned policy successfully completes the fallen-to-standing task in simulation, reaching the targeted pelvis height and uprightness, with a maximum contact penetration of approximately 40 mm during the process. The proposed method is demonstrated on two stand-up sequences and successfully achieves the final recovery objective on both hard and soft ground. Ablation studies show that reference tracking alone is insufficient for successful stand-up, and that explicit recovery rewards are essential.

• The paper introduces a reference-guided reinforcement learning framework that allows a humanoid robot to stand up from a fallen posture on both hard and soft surfaces, demonstrating a degradation policy that extends the robotic range
• The method uses a two-stage training procedure, involving tracking retargeted human demonstrations on rigid ground and fine-tuning on compliant terrain, leading to a maximum penetration of ~40 mm in penetration assessment.
• The ablation study reveals that accurate joint-space tracking alone does not guarantee whole-body recovery, validating the necessity of explicit recovery objectives beyond reference tracking and extending the project.

Overview

The paper presents a reference-guided reinforcement learning (RL) framework that adapts human stand-up demonstrations, recorded on hard ground, to enable a 29-DOF Unitree G1 humanoid to recover from a fallen posture on deformable soft terrain in simulation. The core idea is a two-stage training procedure: the policy first learns to track a retargeted human demonstration on rigid ground, then is fine-tuned on compliant terrain modelled through MuJoCo's phenomenological soft-contact parameters. The work targets a gap the authors identify explicitly: existing learned recovery methods treat terrain compliance as one of many randomisation dimensions rather than as the primary contact-control challenge, and motion-imitation methods generally assume rigid support (Schulman et al., 2017, Luo et al., 11 Nov 2025).

Method

The stand-up task is formulated as an MDP with fixed episode duration. A PPO policy operating at 60 Hz outputs residual adjustments $s_a \mathbf{a}_t$ (with $s_a = 0.25$) on top of a reference joint-position trajectory $\mathbf{q}^{\mathrm{ref}}_t$, executed by joint-level PD controllers with task-specific gains. Observations comprise simulated and reference joint states, tracking errors, deviation from the final standing pose, normalised motion phase $\phi_t$, and previous action. Two stand-up trajectories from the BONES-SEED dataset (Luo et al., 11 Nov 2025), retargeted via GMR (Araujo et al., 2 Oct 2025) and PyRoki, serve as references; one policy is trained per trajectory.

The reward combines three groups of terms. Reference-tracking rewards cover joint pose, velocity, root planar position, pelvis height, and root orientation using exponential kernels. Explicit recovery rewards — standing-pose tracking, torso uprightness, and standing height — are activated only after phase $\phi > 0.65$ through a phase-dependent weight that gradually shifts emphasis from tracking to final standing. Regularisation penalties address foot slip, action magnitude, and action smoothness. This decomposition is deliberate: the ablation results show that tracking alone does not produce successful recovery.

The compliant floor is modelled by repurposing MuJoCo's rigid body soft-contact formulation. The default parameters solref = (0.02, 1) and solimp = (0.9, 0.95, 0.001) are changed to solref = (0.1, 1), solimp = (0.0, 0.95, 0.02), yielding a critically damped response with a 0.1 s time constant and solver impedance ramping from 0 to 0.95 over 0.02 m of penetration depth. The authors argue this width is physically plausible for a roughly 5 cm foam slab under representative human loading. Fine-tuning runs for 20 million steps after 70 million pre-training steps, with reduced learning rate and entropy coefficient, plus a linear reset-noise curriculum on initial joint positions ($\sigma_q$: 0.005 → 0.030 rad) and velocities ($\sigma_{\dot q}$: 0.005 → 0.100).

Results

Both policies completed their respective fallen-to-standing motions on hard and soft flat ground while preserving the demonstrated coordination pattern; deviations appear only during contact-intensive phases, attributable to the policy accommodating delayed support-force generation. Quantitatively, one representative policy achieved a final pelvis height of 0.792 m against a 0.794 m target and a final uprightness of 0.991. Peak contact penetration was 39.4 mm (~40 mm), consistent with the intended impedance ramp width, and penetration increased precisely as the robot attempted to rise, making the movement harder than on hard ground.

The ablation study yields the paper's most instructive result: the tracking-only policy matched the complete-reward policy's aggregate RMS joint error almost exactly (0.1571 rad vs. 0.1560 rad), yet failed the task entirely — maximum pelvis height 0.130 m, final height 0.059 m, and final uprightness collapsing from 0.917 to 0.543. Accurate joint-space tracking therefore does not guarantee whole-body recovery under compliant contact, which directly justifies the explicit recovery objectives.

Limitations

The authors concede several constraints. The MuJoCo soft-contact model captures compliance primarily in the normal direction; real deformable surfaces such as foam or grass also exhibit tangential compliance and submersion-induced lateral resistance that cannot be represented by the existing contact model. All findings are simulation-only, with no physical validation. Robustness claims are local to the applied position/velocity noise ranges, since yaw perturbations, horizontal root displacement, pushes, gravity variation, and friction randomisation were disabled. Additionally, each policy is trained for a single demonstration trajectory, so generalisation across stand-up sequences or initial postures remains untested.

Conclusions

This work demonstrates a two-stage RL procedure that adapts hard-ground human stand-up demonstrations to substantially compliant terrain by combining residual joint-position control, phase-weighted explicit recovery rewards, and MuJoCo soft-contact parameter tuning. The policies reached the targeted standing configuration on both hard and soft ground with ~40 mm maximum penetration while retaining the demonstrated motion pattern, and the ablation establishes that explicit recovery objectives are essential beyond reference tracking. Open questions include modelling combined normal-tangential compliance, validating the approach on hardware, and extending beyond per-trajectory policies to multi-demonstration or posture-general recovery.