---
title: Multi-Modal Delay Randomization (MMDR)
url: https://www.emergentmind.com/topics/multi-modal-delay-randomization-mmdr
type: topic
---

# Multi-Modal Delay Randomization (MMDR)

Multi-Modal Delay Randomization (MMDR) is a training-time protocol for multi-modal reinforcement learning that injects independently randomized, modality-specific observation delays so that a policy is optimized under temporally misaligned inputs rather than under a synchronous, delay-free assumption. In the quadrupedal locomotion literature, MMDR was proposed for vision-guided end-to-end control to emulate the asynchronous, stochastic latencies of proprioceptive sensing, depth perception, network inference, and actuation on real hardware, and it was later incorporated into the QuadKAN framework as part of a Proximal Policy Optimization (PPO) pipeline for vision-guided quadruped motion control [2109.14549, 2508.19153].

## 1. Origins and motivating problem

MMDR emerged from a specific sim-to-real failure mode in vision-guided quadrupedal locomotion: simulation typically provides perfectly synchronized observations at each step, whereas deployed systems do not. In a real quadruped, the end-to-end perception–action loop contains sensor acquisition, preprocessing, network inference, and motor actuation, and these stages introduce modality-dependent latencies that vary over time. The 2021 formulation emphasized that high-dimensional vision is especially affected by computation latency, while proprioceptive streams such as joint states and IMU measurements are typically faster and lower latency [2109.14549].

On a Unitree A1 platform, the reported measurements were: depth map sensor, \(0.033 \pm 0.004\) s per frame (\(\sim 30\) Hz); joint state and IMU sensors, \(0.0025 \pm 0.001\) s (\(\sim 400\) Hz); network inference with vision, \(0.040 \pm 0.007\) s; network inference state-only, \(0.004 \pm 0.026\) s; and actuation time, \(0.0025 \pm 0.001\) s [2109.14549]. The later QuadKAN study described the same class of mismatch in terms of heterogeneous latencies and asynchronous sampling rates: IMU and joint feedback stream at high rates with low latency, while cameras stream at lower rates and visual perception may incur additional inference time; the perception-to-action pipeline and the control loop also run at distinct frequencies [2508.19153].

The central claim of MMDR is therefore not merely that observations may be delayed, but that multiple modalities are delayed differently and asynchronously. This differs from training assumptions in synchronous simulation and from constant-delay compensation schemes that assume a fixed offset. MMDR addresses that gap by randomizing the temporal selection of each modality during training, so the policy experiences cross-modal misalignment as part of its optimization regime rather than as an out-of-distribution artifact at deployment [2109.14549].

## 2. Formal definition

The formal structure of MMDR is a delayed multi-modal observation model. In the general formulation used in QuadKAN, modalities are indexed by \(m \in \{1,\ldots,M\}\), with observation \(o_t^{(m)}\) for modality \(m\) at time \(t\). MMDR constructs delayed observations via independently delayed streams,
\[
o_{t-d_m}^{(m)},
\]
and forms the fused input
\[
x_t = \left[o_{t-d_1}^{(1)}, \ldots, o_{t-d_M}^{(M)}\right],
\]
which is consumed by the policy
\[
\pi_\theta(a_t \mid x_t).
\]
For the quadruped setting with two modalities, proprioception and depth vision,
\[
x_t = \left[s_{t-d_{\mathrm{prop}}}^{\mathrm{prop}},\ I_{t-d_{\mathrm{vis}}}^{\mathrm{depth}}\right].
\]
The delayed input used by the controller is correspondingly written as
\[
\tilde{x}_t = \left[\tilde{s}_t^{\mathrm{prop}},\ \tilde{I}_t^{\mathrm{depth}}\right]
\]
in QuadKAN, or
\[
\hat{o}_t =
\begin{bmatrix}
p_{t-\Delta_p}\\
v_{t-\Delta_v}
\end{bmatrix}
\]
in the original 2021 formulation [2109.14549, 2508.19153].

This formulation makes the training objective explicitly delay-conditioned. QuadKAN states that \(\pi_\theta\) is optimized under a distribution over delays \(\{d_m\}\), encouraging the policy to perform well in expectation across delay-induced temporal misalignments rather than overfit to a synchronous setting [2508.19153]. In the 2021 system, the same idea appears operationally: the policy is trained end-to-end in simulation using past observations sampled with randomized periods for both proprioception and vision, then deployed directly without fine-tuning [2109.14549].

A key implication is that MMDR changes the temporal indices of the inputs, not the underlying control objective. In QuadKAN, delayed inputs are encoded into fused features \(h_t\), and the PPO loss is applied using those features. The policy uses a Tanh-squashed Gaussian action model,
\[
\tilde{a}_t \sim \mathcal{N}\!\big(\mu_\theta(h_t), \operatorname{diag}(\sigma_\theta^2(h_t))\big), \qquad
a_t = a_{\max}\tanh(\tilde{a}_t),
\]
with clipped surrogate, value, entropy, and spline-regularization terms in the total loss [2508.19153].

## 3. Modality-specific delay processes

The defining property of MMDR is that proprioception and vision are delayed differently. In both quadruped implementations, proprioception uses an episode-wise sampled delay and vision uses a per-step stochastic process, but the details of buffering and sampling are paper-specific [2109.14549, 2508.19153].

| Modality | Delay construction | Temporal behavior |
|---|---|---|
| Proprioception | \(\Delta^{\mathrm{prop}} \sim \mathcal{U}[0,0.04]\) s; linear interpolation from a simulator-rate FIFO buffer | Sampled once per episode and held fixed |
| Vision | Randomized frame selection from recent buffered depth frames, one uniformly sampled index per contiguous block | Sampled at each control step; time-varying within episode |

For proprioception, the simulator runs at approximately \(400\) Hz. In QuadKAN, with simulator step size \(\delta_{\mathrm{sim}} = 1/f_{\mathrm{sim}}\), the sampled delay \(\Delta^{\mathrm{prop}}\) is decomposed into integer and fractional parts, \(k = \lfloor \Delta^{\mathrm{prop}}/\delta_{\mathrm{sim}} \rfloor\) and \(\alpha = \Delta^{\mathrm{prop}}/\delta_{\mathrm{sim}} - k\), and the delayed input is computed by linear interpolation,
\[
\tilde{s}^{\mathrm{prop}}_t = (1-\alpha)s^{\mathrm{prop}}_{t-k} + \alpha s^{\mathrm{prop}}_{t-k-1}.
\]
The 2021 formulation described the same mechanism in continuous-time terms: at control time \(t\), \(p_{t-\Delta_p}\) is obtained by linear interpolation between the two buffered entries bracketing \(t-\Delta_p\) [2109.14549, 2508.19153].

For vision, the 2021 paper maintained the most recent \(4k\) depth frames in a buffer at control rate, partitioned the buffer into four contiguous sub-buffers of length \(k\), and sampled one index uniformly from each sub-buffer. The resulting delayed visual input was
\[
v_{t-\Delta_v} =
\left[
I_{t-\delta^{(1)}},
I_{t-\delta^{(2)}},
I_{t-\delta^{(3)}},
I_{t-\delta^{(4)}}
\right],
\]
where for \(j \in \{1,2,3,4\}\),
\[
\delta^{(j)} \sim \mathrm{UniformDiscrete}\big(\{(j-1)k,\ldots,jk-1\}\big).
\]
With control at \(25\) Hz, \(k \in \{4,8,16\}\) corresponds to visual coverage of \(0.64\) s, \(1.28\) s, and \(2.56\) s, respectively; \(k=4\) worked best [2109.14549].

QuadKAN retained the same principle but parameterized vision in terms of a camera frame rate \(f_{\mathrm{vis}}\) of approximately \(30\) Hz and a rolling buffer of the most recent \(4k_{\mathrm{vis}}\) frames. At each control step, one index is sampled uniformly from each of four contiguous blocks to form a 4-frame depth stack,
\[
\tilde{I}^{\mathrm{depth}}_t =
\left[
I_{t-i_1}, I_{t-i_2}, I_{t-i_3}, I_{t-i_4}
\right].
\]
The paper states that this induces a time-varying visual delay within the episode, emulating stochastic perception-to-action latencies and asynchronous capture [2508.19153].

The asymmetry between interpolation for proprioception and discrete selection for vision is deliberate. The 2021 work explicitly stated that no interpolation is performed for vision due to low sensor rate and non-smooth per-frame changes [2109.14549].

## 4. Integration with locomotion policies and architectures

In the original MMDR locomotion system, the observation space combined an \(84\)-D proprioceptive input and a stack of four \(64 \times 64\) depth images. The proprioceptive observation comprised \(12\)D joint rotations, \(4\)D IMU, and \(12\)D last executed action, stacked across three recent time slices. The vision encoder was a 3-layer CNN producing a \(256\)D feature vector; the proprioception encoder was a 2-layer MLP producing a \(256\)D feature vector; fusion was by concatenation; and a 2-layer MLP produced either the action distribution or the scalar value. The action was a \(12\)D vector of target joint angles converted to torques via a PD controller. PPO was used with batch size \(16{,}384\), divided into \(16\) mini-batches, and Adam with learning rate \(1 \times 10^{-4}\); training used \(10\) million samples in PyBullet [2109.14549].

QuadKAN preserved the MMDR idea but changed the policy class. Delayed proprioception is encoded by a spline-parameterized Kolmogorov–Arnold Network (KAN) encoder,
\[
\phi_j(x)=\sigma\!\left(b_j+\sum_{m=1}^{M} w_{jm} B_m(a_j^\top x + c_j)\right), \qquad j=1,\ldots,d_p,
\]
with curvature and Jacobian regularizers,
\[
\mathcal{R}_{\mathrm{curv}} = \lambda_c \sum_{j=1}^{d_p}\sum_{m=2}^{M-1}(w_{j,m+1}-2w_{j,m}+w_{j,m-1})^2,
\]
\[
\mathcal{R}_{\mathrm{jac}} = \lambda_L \mathbb{E}_x\!\left[\left\|\frac{\partial \phi(x)}{\partial x}\right\|_F^2\right],
\]
and \(\mathcal{R}_{\mathrm{spline}}=\mathcal{R}_{\mathrm{curv}}+\mathcal{R}_{\mathrm{jac}}\). The visual encoder is a lightweight CNN patchifier operating on \(\tilde{I}^{\mathrm{depth}}_t \in \mathbb{R}^{4\times 64 \times 64}\), and fusion is performed by a token-wise KAN backbone followed by pooling and a head \(f_{\mathrm{head}}\) that produces \(h_t\) for the policy and value networks [2508.19153].

A central architectural point in QuadKAN is that MMDR is orthogonal to the KAN design: delayed modalities are fed into the same encoders and fusion head used without delay; MMDR changes only the temporal indices of the inputs, not the architecture [2508.19153]. This is important because it frames MMDR as a protocol for temporal robustness rather than as a particular network topology.

## 5. Empirical findings

The 2021 study evaluated MMDR against No-Delay, Frame-Extract, Fixed-Delayed, and Interpolation baselines. In simulation with delay variability, described as random delays \(0.04\)–\(0.12\) s, MMDR attained higher moving distance and fewer collisions than baselines while retaining similar training sample efficiency. In a static environment, moving distance was \(28.7 \pm 7.7\) for MMDR, versus \(26.5 \pm 5.0\) for No-Delay, \(24.9 \pm 3.1\) for Frame-Extract, \(18.5 \pm 0.8\) for Fixed-Delayed, \(21.4 \pm 2.7\) for Interpolation, and \(2.9 \pm 0.5\) for State-Only. In a dynamic environment with moving obstacles, MMDR achieved moving distance \(11.4 \pm 1.9\) and collision steps \(84.4 \pm 22.9\), compared with \(6.0 \pm 1.7\) and \(401.5 \pm 118.0\) for No-Delay, and \(9.9 \pm 2.7\) and \(545.9 \pm 119.9\) for Fixed-Delayed [2109.14549].

The same paper reported direct deployment on a Unitree A1 in four outdoor environments: Box., Dense Box., Moving Box., and Forest. Moving distance for MMDR was \(859.9 \pm 271.4\), \(641.2 \pm 49.9\), \(973.0 \pm 148.4\), and \(992.5 \pm 335.0\), respectively, compared with No-Delay values of \(444.7 \pm 115.0\), \(447.6 \pm 147.6\), \(505.7 \pm 120.5\), and \(733.8 \pm 118.0\). Collision Count remained low: \(0.0 \pm 0.0\), \(0.9 \pm 0.6\), \(0.33 \pm 0.0\), and \(0.22 \pm 0.16\) for MMDR [2109.14549].

QuadKAN was trained with MMDR as part of a shared training protocol across methods and did not present a no-MMDR ablation. Its reported comparisons therefore concern the combined regime of MMDR, domain randomization, and spline-structured fusion. On the trained Thin Obstacle terrain, QuadKAN versus MLP Proprio–Vision showed \(+10.1\%\) return (\(874.37\) vs. \(794.30\)), \(-75.9\%\) collisions (\(15.23\) vs. \(63.27\)), and \(+1.3\%\) distance (\(37.50\) vs. \(37.03\)). On unseen terrains with static obstacles plus rugged ground, QuadKAN achieved \(787.98\) return, \(35.17\) collisions, and \(36.13\) m distance; on dynamic obstacles plus rugged ground, it achieved \(261.52\) return, \(395.03\) collisions, and \(17.77\) m distance [2508.19153].

These results support two distinct conclusions. First, the original MMDR study provides direct evidence that randomized modality-specific delays improve sim-to-real robustness relative to synchronous or fixed-delay alternatives. Second, the QuadKAN study indicates that MMDR remains compatible with later spline-parameterized cross-modal policies, although that paper does not isolate MMDR’s individual contribution [2109.14549, 2508.19153].

## 6. Relation to adjacent methods, limitations, and broader extensions

The quadruped papers distinguish MMDR from several nearby ideas. Standard domain randomization perturbs physical parameters, textures, contacts, or disturbances but typically assumes synchronous observations; it does not capture modality-specific latency and asynchrony. Single-modal delay handling delays only one stream and therefore does not match the deployment regime in which both modalities are delayed differently. Constant-delay compensation assumes a fixed time shift, whereas real latencies are stochastic and load-dependent. Sensor dropout and corruption perturb values rather than timing, and frame skipping reduces input frequency uniformly rather than randomizing delays per modality [2109.14549, 2508.19153].

The limitations are also explicit. In the 2021 study, performance can degrade if deployment delays exceed the randomized ranges, such as \(\Delta_p > 0.04\) s or vision stalls far beyond the \(4k\) horizon; independent uniform sampling may also mismatch real deployments with highly non-uniform or correlated delay statistics [2109.14549]. In QuadKAN, the visual delay distribution is implicit via block sampling; explicit modeling of end-to-end perception inference latency could further improve realism. Proprioception delay is fixed per episode, so adaptive or state-dependent latency models might better capture real-time operating-system scheduling or bus congestion. The paper also states that all experiments are in simulation, making real-world validation an important next step for assessing sim-to-real transfer and confirming MMDR’s effectiveness under hardware latencies [2508.19153].

A broader systems perspective appears in later work on distributed multimodal inference under communication delay uncertainty. That framework models per-packet communication delay with
\[
\Pr\{T_i^{j_s}=R\Gamma_s\}=\varepsilon_s^{R-1}(1-\varepsilon_s),
\]
derives
\[
\mathbb{E}[T_i^{j_s}] = \frac{\Gamma_s}{1-\varepsilon_s},
\qquad
\mathrm{Var}[T_i^{j_s}] = \frac{\varepsilon_s \Gamma_s^2}{(1-\varepsilon_s)^2},
\]
and uses adaptive Temporal Windows of Integration (TWIs),
\[
T_W = \max_{s \in \{a,v\}} \frac{P_s \Gamma_s}{1-\varepsilon_s},
\]
to trigger non-blocking inference under heterogeneous stream delays [2511.16225]. This suggests a broader generalization of MMDR-style reasoning beyond embodied RL: modality-specific delay distributions can be modeled not only at the observation level, but also at packet and token granularity, with runtime adaptation of the inference schedule. In that broader view, MMDR is part of a larger class of methods that treat asynchronous multi-modal timing as a first-class component of the learning and inference problem rather than as a nuisance external to the model [2511.16225].

Source: https://www.emergentmind.com/topics/multi-modal-delay-randomization-mmdr