---
title: 'JT-SPI: Robust Joint Torque Perturbation'
url: https://www.emergentmind.com/topics/joint-torque-space-perturbation-injection-jt-spi
type: topic
---

# JT-SPI: Robust Joint Torque Perturbation

Joint Torque Space Perturbation Injection (JT-SPI) is a sim-to-real robustness methodology in which, during simulation-time policy training, learned, state-dependent perturbations are injected directly into the joint-torque inputs of the robotic forward dynamics engine. JT-SPI recasts the sim-to-real “reality gap” as an unknown nonlinear mapping from nominal torques to realized torques and exposes the control policy to a much richer family of actuator and contact-force mismatches than can be achieved by randomizing a standard, finite set of simulation parameters. The resulting policies demonstrate increased robustness to complex and previously unseen reality gaps, facilitating successful transfer of motor skills from simulation to hardware for high-DOF legged robots [2504.06585].

## 1. Formal Definition and Underlying Principles

JT-SPI addresses reality gap challenges by modeling the mapping from commanded joint torques to actual joint torques as an unknown, potentially nonlinear and state-dependent signal. Traditional domain randomization approaches typically randomize simulator parameters (such as link masses or friction coefficients) across fixed finite sets. JT-SPI, by contrast, introduces perturbations in the joint torque space that are state-conditioned and drawn from a wide functional class, parameterized via a universal function approximator (specifically, a multi-layer perceptron, or MLP).

The joint-space dynamics under domain randomized torque noise are:

$$
M(q; p_{DR})\,\ddot q + C(q,\dot q; p_{DR}) + G(q; p_{DR}) + \tau_{contact}(s; p_{DR}) = \tau_{input} + \tau_{DR}(s; p_{DR})
$$

JT-SPI injects a perturbation $\tau_\phi$ from a function class $\mathcal{J}T$ (implemented as an MLP) with weights $\phi$ re-sampled each episode. At timestep $t$:

- Policy output (nominal torque): $\tau_\pi(o_t)$
- JT-SPI perturbation: 
  $$
  \tau_\phi(s_t) = \sigma_{lim} \cdot \tanh(\text{MLP}(\hat{o}_{privileged}(t); \phi))
  $$
  where $\sigma_{lim}$ limits the maximum perturbation, inputs are normalized privileged full-state observations, and $\phi$ is re-sampled per episode from XavierUniform.

The policy observes partial state $o_t \in \mathbb{R}^{48}$, while the perturbation generator accesses privileged full state (normalized). This exposes the policy to a diverse, high-dimensional set of actuator and contact deviations.

## 2. Algorithmic Implementation

JT-SPI is integrated into on-policy, high-throughput simulation frameworks (e.g., IsaacGym) using parallel rollouts. A high-level pseudocode is:

```python
Initialize policy parameters θ, critic parameters, discriminator, etc.
for iteration = 1 … N_updates:
    for env = 1 … N_parallel:
        # Episode start
        sample φ_env ~ Xavier    # MLP weights for perturbation
        s_env = s0
        step = 0
        while not done and step < max_steps:
            o_env = partial_observation(s_env)
            a_env = π_θ(o_env)            # action in [-1,1]^12
            τ_π = τ_limit ⊙ a_env         # nominal torques
            o_priv = privileged_observation(s_env)
            ĥo_priv = o_priv / running_std
            τ_φ = σ_lim * tanh(MLP(ĥo_priv; φ_env))
            (s_next, r, done) = simulator.step(s_env, τ_π + τ_φ)
            store_transition(s_env, o_env, a_env, τ_φ, r, s_next, done)
            s_env = s_next
            step += 1
    # Policy optimization (PPO+AMP+gradpen)
```

Key scheduling aspects include re-sampling $\phi$ only at episode boundaries and perturbing 50% of rollouts to maintain learning stability. The perturbation MLP comprises two 256-unit ReLU hidden layers, a final tanh layer for bounded outputs, and uses zero bias in all layers to ensure zero input yields zero perturbation.

## 3. Experimental Setup and Comparative Analysis

JT-SPI was validated using the TOCABI humanoid platform (100 kg, floating-base, high-ratio harmonic drives with gear ratio 100), controlled at 125 Hz. Training utilized PPO with Adversarial Motion Prior (AMP) imitation and a gradient penalty regularizer.

Comparison was made to:

- **Domain Randomization (DR):** Randomizes parameters such as terrain friction, link masses, center-of-mass offsets, armature inertia, damping, motor constant, latency, random pushes, and observation noise within prescribed ranges.
- **ERFI Baseline:** (from Campanaro et al.) Applies untargeted additive torque noise at each joint.

JT-SPI differs fundamentally by generating state-dependent, potentially highly nonlinear perturbations rather than white-noise (ERFI) or a finite augmented parameter set (DR).

Perturbation and DR method characteristics:

| Method      | Perturbation Type             | State Dependence | Functional Family    |
|-------------|------------------------------|------------------|---------------------|
| DR          | Parameter randomization       | No               | Predefined          |
| ERFI        | Untargeted torque noise       | No               | White noise         |
| JT-SPI      | Learned, bounded perturbation | Yes              | Universal approx.   |

## 4. Empirical Evaluation and Scenario Results

JT-SPI was evaluated on velocity-commanded humanoid walking at $v_x = 0.4$ m/s, with yaw commands in $[-1, 1]$ rad/s and zero lateral command. Key performance metrics included forward velocity tracking error (mean, variance), lateral/yaw tracking, and success/failure at maintaining balance.

Test conditions covered:

- **Nominal simulation:** All methods exhibited similar velocity tracking and gait quality.
- **Unseen actuator stiffness (250 Nm/rad, not seen during training):** JT-SPI and ERFI succeeded; DR failed.
- **Unseen contact compliance (Mujoco solref time constant 0.2 s, soft ground):** Only JT-SPI succeeded robustly; DR and ERFI failed (robot falls).
- **Sim-to-real transfer (lab, uneven/slippery floor):** JT-SPI succeeded in all seeds, DR in 2/3, ERFI in none.

No additional ablations on fraction of perturbed environments or MLP size were reported. These results indicate a substantial robustness improvement of JT-SPI to both actuator and contact-variation reality gaps, with successful transfer under challenging real-world perturbations.

## 5. Hyperparameters and Training Guidelines

Recommended settings and procedures include:

- **MLP architecture:** Two hidden layers of 256 ReLU units each; tanh output layer; all layers have zero bias (ensuring zero-motion state yields zero perturbation).
- **Observation normalization:** Normalize privileged observations by running standard deviation only, omitting mean subtraction.
- **Perturbation magnitude (σ_lim):** Start at 20 Nm, gradually increase to 50 Nm for joints. For base-force perturbations, use up to 80 N. Adjust based on observed gait stability in nominal simulation.
- **Fraction of perturbed rollouts:** 50% perturbation recommended to balance robustness and learning stability.
- **Perturbation sampling:** Newly sample $\phi$ once per episode; re-sampling per timestep is discouraged as it produces high-frequency noise that harms policy learning.

The design rationale leverages the Universal Approximation Theorem, stating the MLP class can model any continuous, bounded torque perturbation mapping $\tau_{reality}(s)$. By randomizing $\phi$ each episode, the policy is forced to generalize over a large subspace of such mappings. Empirically, policies withstand higher unmodeled disturbances, supporting robust sim-to-real transfer [2504.06585].

## 6. Distinguishing Characteristics and Theoretical Implications

The JT-SPI approach enables policies to encounter a vastly richer set of actuator/force deviations compared to approaches based solely on predefined parameter randomization. State-dependent perturbations allow simulation of complex, context-sensitive discrepancies (e.g., nonuniform friction, actuator nonlinearities) which fixed-parameter or white-noise models cannot represent.

A plausible implication is that as simulator fidelity and robot actuation complexity increase, sim-to-real transfer robustness will demand perturbation schemes with sufficient expressiveness, such as the universal function class utilized in JT-SPI. The finding that half-perturbed rollouts maximize learning stability, and that re-sampling at episode rather than step level is crucial, highlights important scheduler design considerations. JT-SPI thereby contributes a generalizable framework for bridging reality gaps in modern high-DOF robotic locomotion [2504.06585].

Source: https://www.emergentmind.com/topics/joint-torque-space-perturbation-injection-jt-spi