---
title: 'Stable Sim2Real Transfer: Strategies and Insights'
url: https://www.emergentmind.com/topics/stable-sim2real
type: topic
---

# Stable Sim2Real Transfer: Strategies and Insights

Stable-Sim2Real denotes a strand of simulation-to-reality research in which the central objective is not merely transfer, but transfer that remains reliable under model inaccuracies, environmental variations, asynchronous software execution, perceptual shift, and limited real-world adaptation data. In the recent literature, this objective appears across humanoid teleoperation, legged locomotion, underwater docking, autonomous driving data synthesis, manipulation, and model-based control, with stability pursued through abstraction, explicit mismatch modeling, staged simulator-to-hardware validation, and selective use of real data rather than through a single canonical algorithm [2605.12347] [2503.01255] [2407.04328].

## 1. Operational meaning and scope

The literature does not use a single universal definition of stability. In humanoid teleoperation, one paper defines “stable” operationally as continuous execution without visible oscillation, loss of balance, abrupt joint-velocity discontinuities, joint-limit violations, self-collisions, or perceptible teleoperation delay, and treats these as the practical failure modes that must be eliminated before unchanged sim2real deployment can be claimed [2605.12347]. In reinforcement-learning transfer, a different formulation defines the Sim2Real performance gap as the discrepancy between the return achieved by a policy in simulation and the return achieved by that same policy in the real world, thereby shifting attention from simulator fidelity alone to the stability of downstream control performance [2510.17709].

A second recurring theme is that instability often originates outside nominal rigid-body dynamics. EAGERx emphasizes asynchronous control, concurrent sensing and actuation, and communication/computation delays as first-order transfer variables, arguing that a delay-free sequential simulator can misrepresent the temporal structure of real robot software stacks even when physical modeling is adequate [2407.04328]. Work on robotic locomotion similarly shows that unmodeled joint static friction can dominate transfer behavior: on Saturn Lite, conventional domain randomization that omitted static friction allowed backward walking on flat ground but caused the robot to fall during forward walking, and remained brittle on stairs [2503.01255].

Taken together, these results suggest that Stable-Sim2Real is best understood as a deployment-oriented notion. It concerns whether the transferred system preserves feasibility, synchronization, and task competence under the dominant mismatch variables of the target setting, rather than whether the simulator is globally “more realistic” in an undifferentiated sense.

## 2. Architectural strategies based on abstraction and decomposition

A major route to stable transfer is to move the learning or teleoperation interface into a space that is less sensitive to simulator-specific low-level dynamics. In real-time humanoid teleoperation, the retargeting layer is explicitly kinematic and deterministic: human motion from a Virdyn IMU suit is mapped to robot joint targets by a morphology-aware function
\[
q_r = \mathcal{R}(q_h),
\]
followed by soft joint-limit clipping and an exponential moving average
\[
y_t = \alpha u_t + (1-\alpha) y_{t-1}.
\]
The same motion-processing stack is first validated in MuJoCo and then deployed on the physical Unitree G1 without modification, with low-level balance delegated to the onboard servo layer rather than to a learned whole-body stabilizer [2605.12347].

A closely related argument appears in visual navigation. “Rethinking Sim2Real” shows that lower-fidelity kinematic simulation can yield higher transfer than higher-fidelity dynamic simulation when the deployed policy operates in a high-level velocity-command space. Under the same broad training setup, kinematic policies reached about 500M steps while dynamic policies reached about 50M steps, and the kinematic policies transferred more reliably to a real Spot robot. In the reported real-world evaluation, the Habitat Kinematic and iGibson Kinematic policies both achieved 100.0 SR, whereas Habitat Dynamic achieved 40.0 SR and iGibson Dynamic 67.7 SR, with substantially higher action counts and collision counts for the dynamic policies [2207.10821].

Other work pushes the same principle into command-space adaptation. The Schwarz-Christoffel Mapping framework transfers commands from a teacher to a degraded learner by geometrically mapping teacher control inputs into the learner’s command space while respecting learner limits. Its claims are geometric rather than Lyapunov-theoretic, but the intended benefit is stable maneuver preservation when the learner is less capable than the teacher and closed-form learner dynamics are unavailable [2503.16634]. In manipulation, “Sim2Real Transfer for Reinforcement Learning without Dynamics Randomization” similarly reduces brittleness by learning in operational space with explicit joint-space and Cartesian-space saturation, so that the policy acts in a task-aligned Cartesian interface rather than in raw joint torque coordinates [2002.11635].

This architectural pattern extends to software structure. EAGERx treats both simulators and physical hardware as engines within the same graph-based computation and communication pipeline, so that state, action, and time-scale abstractions can be shared across sim and real while delays and asynchronous execution are modeled explicitly [2407.04328]. A plausible implication is that Stable-Sim2Real often depends less on maximizing fidelity everywhere than on choosing the abstraction layer at which the dominant mismatch becomes manageable.

## 3. Explicit mismatch modeling, calibration, and simulator adaptation

A second line of work stabilizes transfer by identifying the specific variable that dominates the gap and then representing it explicitly. The clearest example is joint friction. In “Impact of Static Friction on Sim2Real in Robotic Reinforcement Learning,” the single-joint model
\[
I_j \ddot{\theta}_j(t) + B_j \dot{\theta}_j(t) = \tau_j(t) + f_j(t)
\]
is augmented with a sign-dependent friction term, and the resulting static friction-aware domain randomization becomes the only evaluated method that achieves stable walking on flat ground and successful stair traversal in sim2real. The paper reports a static friction mean of \(0.442\) for Saturn Lite shank motors versus \(0.0481\) for Go1, and an \(f/\tau_{\max}\) ratio mean of \(0.98\%\) versus \(0.13\%\), respectively; it also reports that removing sealing rings reduced static friction by 70%, reinforcing the claim that omitted friction was a first-order transfer variable [2503.01255].

In underwater docking, the emphasis is more conservative. The BlueROV2 Heavy study evaluates zero-shot transfer from nominal simulation to payload-shifted disturbed simulation and finds that a naively trained policy is already quite robust when relative pose, orientation, linear velocity, angular velocity, and previous action are observed directly. Large domain randomization improves positional robustness in the Hard scenario with a 7.0 kg payload at 0.3 m along the \(x\)-axis, while Large DR plus history marginally improves angular performance; Small DR performs worst on angular error [2506.17823]. This suggests that robustification should be matched to the actual failure mode rather than applied indiscriminately.

Several papers formalize simulator adaptation itself. “Closing the Sim2Real Performance Gap in RL” argues that optimizing simulator fidelity or randomization only as proxies can be misaligned with the true objective, and therefore formulates a bi-level RL problem in which simulator dynamics and reward parameters \(\theta\) are adapted according to real-world return. The inner problem learns a policy in simulation, and the outer problem adjusts \(\theta\) so that the in-sim optimum induces higher real performance, using implicit differentiation of the inner stochastic policy-gradient fixed point [2510.17709]. The executable-digital-twin NMPC paper pursues a different but related path: controller hyperparameters such as \(Q\), \(R\), and \(S\) are adapted online using a high-fidelity xDT and real data, while preserving the constrained NMPC structure. In the reported double-lane-change experiment, the adapted controller reduced the velocity tracking error by almost 91% and the path tracking error by 63% [2208.06226].

A more formal gap-bounding approach appears in “Quantification of Sim2Real Gap via Neural Simulation Gap Function,” which defines a state- and input-dependent bound
\[
|\hat f_i(x,u)-f_i(x,u)| \le \gamma_i(x,u)
\]
between a nominal mathematical model and a high-fidelity simulator. The learned \(\gamma_i\) is then lifted into a bounded-uncertainty dynamics model suitable for robust or symbolic control synthesis, with a full-domain guarantee obtained from finite sampled data under Lipschitz assumptions [2506.17675]. Sym2Real adopts a complementary decomposition at the dynamics-model level:
\[
\hat{s}_{t+1} = f_{\text{SR}}(s_t,a_t) + f_{\text{res}}(s_t,a_t),
\]
where a symbolic base model is learned from low-fidelity simulation and a small residual network is trained from a handful of target-domain trajectories. The paper reports robust control with about 10 trajectories in the real world, six OOD sim2sim scenarios, and five real-world conditions, and attributes the stability gains to preserving the symbolic action-to-state structure while regularizing the residual to remain small [2509.15412].

An older but still representative methodology is Sim2Real2Sim, in which an initial simulator is used to design the task solution, the solution is transferred to hardware, and the resulting discrepancies are fed back to update the simulator. In the DRC plug task, this loop was used to identify cable stiffness and damping from real measurements and then update the Gazebo model, yielding static deformation errors on the order of a few thousandths of a radian and sagging-angle errors mostly below about 4% [2002.02538].

## 4. Perception-side stability and simulation-ready data generation

Stable transfer can also fail because the observation channel is unstable even when the control law is adequate. Several papers therefore relocate the stability problem to perception. In autonomous-driving data synthesis, simulator-generated semantic maps are translated into realistic images by Pix2pixHD, OASIS, or ControlNet. The main conclusion is not that diffusion universally dominates, but that ControlNet degrades more gracefully when the conditioning labels come from the simulator rather than from the manually annotated Cityscapes distribution used for training. On SHIFT-conditioned outputs, ControlNet achieves BRISQUE \(14.625\), NIQE \(9.175\), and mIoU \(34.1\), while OASIS is best on FID with \(112.405\); the paper interprets this as evidence that diffusion-based conditioning can be more stable under label-source shift [2404.09111].

TRITON tackles a different perception failure mode: temporal inconsistency in unpaired sim2real image translation. It represents object appearance with per-object implicit textures \(\mathcal{V}^i(u,v)\), projects them into images, and enforces an unprojection consistency loss across views so that the same surface coordinates retain the same appearance over indefinite timescales. The method is explicitly motivated by the observation that framewise plausible translation can still be unusable for robot learning if object identity, texture, or occlusion relations flicker across time [2206.13500].

The same logic appears in deployment-time adaptation. “Real2Sim or Sim2Real” proposes translating real monocular RGB observations into simulator-like images at inference time and then feeding them into a frozen simulation-trained insertion policy. The reported real-world outcome is 12 successes in 13 trials, or about 92.3%, using only a low-cost RGB camera, but the paper also states that the method is camera-position specific and that important features such as the hole may disappear during CycleGAN transfer [2206.02679]. “Best of Sim and Real” pushes the decoupling further: control is learned in simulation with privileged state, then frozen, while a real-world visual bridge \(f_\phi\) is trained from 10–20 expert demonstrations so that
\[
\mathcal{L} = \mathbb{E}_{(o_t,a_t^*) \sim \mathcal{D}_{\text{demo}} \left[\left\|a_t^* - \pi_\theta(f_\phi(o_t))\right\|^2\right].
\]
With 10 real demonstrations, the reported success rates are 73.3% on Stacking Cube, 43.3% on Opening Drawer, and 88.3% on Closing Door, substantially above the strongest end-to-end baseline reported in the paper [2509.25747].

Stable simulation itself can require stable scene generation. STABLE addresses this by separating semantic scene reasoning from physics correction. A fine-tuned LLM generates coarse tabletop layouts, and a flow-based physics corrector adjusts object poses using mesh geometry and SDF-based penalties for object-object collision, object-table penetration, and floating. The reported outcome is Object Collision \(=0\), Align with Task \(=99.4\), and Align with Scene Graph \(=99.0\), making the generated layouts “simulation-ready” in the paper’s operational sense [2605.16137].

Beyond robotics, environmental neural processes show that the same perception/data principle holds in sparse observational domains. Sim2Real training from ERA5 reanalysis to DWD stations substantially outperforms both Sim Only and Real Only when real spatial coverage is sufficient, but FiLM-style lightweight adaptation underperforms global fine-tuning when the real domain contains shorter spatial length scales than the simulator exposed during pre-training [2310.19932]. This suggests that stable transfer on the observation side depends on matching the adaptation capacity to the structure of the domain gap.

## 5. Validation regimes and evaluation practice

A distinguishing feature of the Stable-Sim2Real literature is its preference for staged validation. The humanoid teleoperation system validates the IMU suit first, then runs sim2sim verification in MuJoCo using the official Unitree G1 model, and only then deploys the identical retargeting software on hardware. In simulation, the stability criteria are absence of joint-limit violations, self-collisions, and abrupt velocity discontinuities; on hardware, the criteria remain qualitative but centered on synchronized, balanced motion without visible delay [2605.12347].

Several papers stress that simulator-only success can be misleading. In the friction study, domain randomization without static friction performs best in Webots sim2sim, yet falls during forward walking in sim2real; Actuator Net repairs flat-ground forward walking but still fails on stairs; friction-aware randomization is the only method that generalizes to the harder terrain on hardware [2503.01255]. In visual navigation, dynamic policies trained in one simulator transfer poorly across simulator/controller changes, while lower-fidelity kinematic policies maintain far more consistent behavior across Habitat, iGibson, and real Spot deployment [2207.10821]. These results undermine the assumption that better in-simulator behavior is automatically predictive of better transfer.

Evaluation metrics are correspondingly heterogeneous. Depending on the domain, the literature uses SR and SPL in navigation [2207.10821], NLL and MAE in environmental interpolation [2310.19932], AP/mIoU/PQ and image-quality metrics in driving synthesis [2404.09111], Object Collision and Align with Task in scene generation [2605.16137], and distance or angular error over time in underwater docking [2506.17823]. This diversity reflects real differences in task demands, but it also complicates cross-paper comparison.

A persistent limitation is that many stability claims remain qualitative or only weakly quantified. The Unitree G1 teleoperation paper reports no control-loop frequency, no measured end-to-end latency, and no tracking-error metrics [2605.12347]. The static-friction study provides structured qualitative comparisons but no success-rate table or large-scale statistics [2503.01255]. The underwater docking study is entirely simulation-based and omits success rate, collision rate, and time-to-dock [2506.17823]. A plausible implication is that Stable-Sim2Real has matured faster as an engineering design culture than as a uniform evaluation discipline.

## 6. Limitations, controversies, and emerging directions

The literature is notable for several recurring controversies. One is whether higher simulator fidelity is always beneficial. “Rethinking Sim2Real” answers negatively for high-level navigation, arguing that high-fidelity dynamics can slow learning and induce overfitting to simulator-specific inaccuracies [2207.10821]. “Sim2Real Transfer for Reinforcement Learning without Dynamics Randomization” similarly shows that, for a KUKA LBR iiwa peg-in-hole task, operational-space control, explicit constraints, and pre-training system identification can transfer successfully without dynamics randomization, and that randomization may slow learning and make training less stable [2002.11635]. By contrast, the static-friction study shows that omitting a single first-order effect can make transfer fundamentally unreliable, so abstraction or reduced fidelity is not a universal remedy [2503.01255]. Taken together, these papers suggest that stability depends on whether the omitted detail is nuisance structure or a dominant task variable.

A second limitation is the weakness of formal guarantees. The SCM command-mapping framework motivates stability through angle-preserving geometric consistency but does not provide a closed-loop stability theorem [2503.16634]. The xDT-based NMPC paper preserves a constrained controller structure and uses energy-based validation, but explicitly does not offer a full Lyapunov proof for the adaptive loop [2208.06226]. The neural simulation gap paper provides a formal transition-mismatch bound, yet its guarantees terminate at the high-fidelity simulator rather than at physical hardware [2506.17675]. Stable-Sim2Real therefore remains largely empirical outside a small subset of model-bounding work.

The open problems identified by the papers are correspondingly concrete. Humanoid teleoperation points to adaptive Kalman filtering, online whole-body momentum control, and model-predictive footstep planning for more dynamic motions [2605.12347]. Underwater docking points to currents, waves, richer disturbances, and eventual real-vehicle deployment [2506.17823]. Driving synthesis leaves temporal consistency as an open problem at the video level [2404.09111]. The SCM framework is fundamentally restricted to 2D command spaces [2503.16634]. Sym2Real explicitly leaves contact-rich systems and higher-dimensional settings open [2509.15412]. STABLE presently restricts tabletop poses to translation plus yaw and a fixed three-stage semantic curriculum [2605.16137].

The broad synthesis suggested by these works is that Stable-Sim2Real is less a doctrine of “more realism” than a doctrine of selective fidelity. Stability improves when the abstraction level, adaptation mechanism, and evaluation protocol are chosen to match the dominant deployment mismatch: kinematic retargeting when low-level servos are reliable, explicit friction modeling when stick-slip dominates, delay-aware graph execution when software timing is the gap, real-domain perception bridges when visual appearance is the bottleneck, and iterative simulator correction when the main uncertainty is a compact set of physical parameters.

Source: https://www.emergentmind.com/topics/stable-sim2real