---
title: Discrepancy of Environment Dynamics (DoE)
url: https://www.emergentmind.com/topics/discrepancy-of-environment-dynamics-doe
type: topic
---

# Discrepancy of Environment Dynamics (DoE)

Searching arXiv for recent and directly relevant papers on “Discrepancy of Environment Dynamics” and closely related formulations.
Discrepancy of Environment Dynamics (DoE) denotes a family of concepts for describing a mismatch between environment dynamics as rendered, assumed, estimated, or previously learned and the dynamics that are actually perceived, encountered, or induced in operation. In human-in-the-loop teleoperation, the term refers to the difference between the “true” dynamics of a remote or virtual environment and the dynamics actually perceived by the operator through the teleoperator transmission [2107.10784]. In multi-environment learning, it refers to variations in the underlying dynamics across environments that invalidate training in a single nominal environment [2106.04546; 2206.09314]. In non-stationary reinforcement learning, it can be defined directly as the change in the action-value function caused by an environment shift under a fixed policy, \(DoE(s_k,a_k)=Q_i(s_k,a_k)-Q_{i-1}(s_k,a_k)\) [2509.15032]. Across these usages, the common object is a dynamics-level gap rather than a purely observational or reward-level mismatch.

## 1. Definitions and semantic scope

Across the cited literature, DoE is not a single universally standardized scalar quantity. Instead, it is instantiated in several mathematically distinct ways depending on the problem class. In teleoperation, the discrepancy is between rendered environment dynamics and perceived environment dynamics, and is quantified through transfer-function identification, angular displacement tracking, Bode plots, and step responses rather than by a single scalar metric [2107.10784]. In dynamical-systems learning, the discrepancy is an environment-specific correction term \(g_e\) in the additive decomposition \(f_e=f+g_e\) [2106.04546]. In robust imitation learning, the relevant object is variation in transition dynamics across sampled environments, handled by minimizing a weighted sum of Jensen–Shannon divergences between the learned policy and multiple experts [2206.09314]. In non-stationary RL, the discrepancy is explicitly defined as the environment-induced value shift under a fixed policy [2509.15032]. In Bayesian calibration, discrepancy is either an additive Gaussian-process correction to simulator output or a Gaussian-process parameter drift internal to the simulator [2603.11960].

| Setting | What is discrepant | Representative formulation |
|---|---|---|
| Teleoperation | Rendered vs perceived environment dynamics | \(G(s)=\frac{\theta(s)}{\tau(s)}=\frac{1}{a s^2+b s+c}\) |
| Multi-environment dynamics learning | Shared vs environment-specific dynamics | \(f_e=f+g_e\) |
| Robust imitation learning | Agent policy vs experts across varying dynamics | Weighted JS-divergence objective |
| Non-stationary RL | Pre-shift vs post-shift value under fixed policy | \(DoE(s_k,a_k)=Q_i(s_k,a_k)-Q_{i-1}(s_k,a_k)\) |
| Bayesian calibration | Simulator output vs observations, or baseline parameters vs parameter drift | \(y=\eta+\delta_\eta+e\) or \(y=\eta(x,\theta+\delta_\theta(x))+e\) |

This suggests that DoE is best understood as a modeling lens for dynamics mismatch rather than as a single metric with invariant semantics.

## 2. Teleoperation and perceived environment dynamics

The most literal use of DoE in the provided corpus appears in the teleoperation study “A novel testbed for investigating the impact of teleoperator dynamics on perceived environment dynamics” [2107.10784]. That work studies human-in-the-loop telerobotic systems in which dexterous manipulation depends on how well the telerobot is incorporated into the operator’s sensorimotor control scheme. The central claim is that leader-follower transmission dynamics can distort what the operator feels, thereby creating a discrepancy between the environment that is rendered at the follower side and the environment that is perceived at the leader side.

The experimental platform is a modular, 1-DoF rotational teleoperation testbed with reconfigurable transmissions. The available leader-follower couplings are a rigid shaft, an elastic shaft made of neoprene, a damped shaft using a dual-direction oil-filled rotary damper, a combined elastic-plus-damped transmission, and an electromechanical bilateral transmission implemented with DC motors and a position-position PD control law. High-resolution torque sensors are mounted at both leader and follower interfaces, and the environment can be free space or a virtual spring of \(1\ \mathrm{mNm/deg}\). For each configuration, the felt dynamics are identified by second-order transfer functions of the form
\[
G(s)=\frac{\theta(s)}{\tau(s)}=\frac{1}{a s^2+b s+c},
\]
with reported percent fit greater than \(99\%\) [2107.10784].

The empirical findings are configuration-specific. The rigid transmission provides the best transparency and minimal discrepancy; leader and follower move almost in perfect synchrony, and the rendered spring is perceived accurately. The damped transmission produces heavy low-frequency damping, substantial follower lag, and attenuation of the environment so that the operator feels a softer or less stiff spring than actually exists. The elastic transmission introduces substantial slippage and additional blurring due to viscoelasticity of the neoprene. The combined damped-plus-elastic case resembles the damped-only configuration and masks environment properties even more. In the electromechanical case, controller gains must be tuned for stability, often producing excessive closed-loop damping; the difference between free space and spring becomes negligible at the operator interface because the controller dynamics dominate [2107.10784].

Within this formulation, DoE is not merely a transparency deficit in the narrow bilateral-control sense. It is a perceptual and task-relevant mismatch between environmental mechanics and the dynamics that the operator can actually exploit. A plausible implication is that teleoperation architectures can preserve nominal stability while still substantially degrading environment discrimination.

## 3. Multi-environment learning and generalization

In machine learning for dynamical systems, DoE is used to denote differences in the underlying dynamics across environments. LEADS formalizes this by decomposing environment-specific dynamics as
\[
\frac{dx_t^e}{dt}=f_e(x_t^e), \qquad f_e=f+g_e,
\]
where \(f\) captures common dynamics and \(g_e\) captures environment-specific discrepancy [2106.04546]. The framework is motivated by the observation that learning a single model across all environments disregards discrepancies and leads to biased solutions, while learning one model per environment ignores commonalities and is prone to scarcity problems. LEADS therefore optimizes a shared model jointly with discrepancy terms \(g_e\), penalizing the complexity of \(g_e\) through \(\Omega(g_e)\). For neural-network hypothesis spaces, the penalty is
\[
\Omega(g_e)=\|g_e\|_\infty^2+\alpha\|g_e\|_{\mathrm{Lip}}^2.
\]
The theoretical analysis gives a sample-complexity bound in which the samples required per environment decrease as the number of environments increases, provided the discrepancy complexity is controlled. Empirically, LEADS improves generalization on ODEs and PDEs and supports lower-sample adaptation to unseen environments by keeping \(f\) fixed and fitting only a new \(g_{e'}\) [2106.04546].

Robust imitation learning against variations in environment dynamics addresses a related but policy-centric version of the same problem [2206.09314]. The RIME framework samples \(N\) environments \(\mathcal E_1,\ldots,\mathcal E_N\), collects expert demonstrations \(\tau_E^i\) from expert policies \(\pi_E^i\), and trains the agent in all sampled environments. The core objective is
\[
\min_\pi \; \mathbb{E}_{s \sim \frac{1}{N} \sum_{i=1}^N \mu_{\pi}^i} \left[ \sum_{j=1}^N \lambda_j(s)\,\mathcal{D}(\pi(\cdot|s), \pi_E^j(\cdot|s)) \right],
\]
with \(\mathcal D\) specialized to the Jensen–Shannon divergence. The paper argues that occupancy matching is ambiguous in multiple environments because the mapping from policy to average occupancy measure is not injective, and therefore uses direct policy matching via JS divergence. Numerical results on MuJoCo locomotion tasks show that standard IL trained in a single environment can catastrophically fail under perturbed dynamics, whereas RIME maintains high mean and minimum returns across a continuous range of seen and unseen dynamics [2206.09314].

A further development is DynaInfer, which removes the assumption that environment labels are available during training [2510.19784]. It treats prediction error as a discrepancy measure: for trajectory \(x^i\) and candidate environment \(e\),
\[
L(i,e;\theta,\phi_e)=\int_{t\in I}\left\|\frac{dx_t^i}{dt}-h(x_t^i;\theta,\phi_e)\right\|_2^2 dt.
\]
Environment assignment and model parameters are then optimized alternately. The assignment step selects the environment with minimal prediction error, and the parameter step updates the shared and environment-specific parameters. The reported results show rapid convergence to true labels and performance that outperforms existing environment assignment techniques, sometimes even exceeding oracle training with provided labels [2510.19784].

## 4. Reinforcement learning under non-stationarity and partial observability

The paper “Sample Efficient Experience Replay in Non-stationary Environments” introduces a particularly explicit definition of DoE for RL [2509.15032]. The environment changes at discrete times \(T_i\), inducing a sequence of MDPs \(\langle \mathcal S,\mathcal A,P_i,R_i,T_i,\gamma\rangle\). For a transition \((s_k,a_k)\), the discrepancy of environment dynamics is defined as
\[
DoE(s_k,a_k)
= \mathbb{E}_{P_i,\pi}\!\left[\sum_{j=k}^{\infty}\gamma^{j-t}R_i(s_j,a_j)\right]
- \mathbb{E}_{P_{i-1},\pi}\!\left[\sum_{j=k}^{\infty}\gamma^{j-t}R_{i-1}(s_j,a_j)\right]
=Q_i(s_k,a_k)-Q_{i-1}(s_k,a_k).
\]
The point of this construction is to isolate the effect of environment change from the effect of policy updates. Standard TD-error prioritization conflates both. DEER therefore assigns replay priority differently before and after detected change points. Pre-change transitions are prioritized when \(|DoE|\) is small, using
\[
p_k=2\sigma(-|DoE(s_k,a_k)|),
\]
whereas post-change transitions interpolate between TD-error and \(|DoE|\) according to a divergence score \(S\) derived from a binary classifier over reward-sequence windows. Experiments on four non-stationary benchmarks report an \(11.54\%\) improvement over the best-performing state-of-the-art ER methods [2509.15032].

A related but distinct discrepancy notion appears in the \(\lambda\)-discrepancy framework for partial observability [2407.07333]. There, discrepancy is the difference between two TD\((\lambda)\) value estimates computed with different \(\lambda\) values. Since TD\((\lambda=0)\) makes an implicit Markov assumption and TD\((\lambda=1)\) does not, the discrepancy detects non-Markovian state representations. The paper proves that the \(\lambda\)-discrepancy is exactly zero for all MDPs and almost always non-zero for a broad class of partially observable environments. It is then minimized as an auxiliary loss between recurrent value networks to learn better memory functions, yielding improved performance in challenging POMDPs [2407.07333].

Taken together, these formulations separate two distinct sources of dynamics mismatch in RL: exogenous environment shifts and endogenous representational insufficiency.

## 5. Calibration, discrepancy modeling, and data assimilation

In Bayesian model calibration, discrepancy refers to the mismatch between a simulator and observed data. The classical Kennedy–O’Hagan formulation is
\[
y(x_i)=\eta(x_i,\theta_i)+\delta_\eta(x_i)+e_i,
\]
where \(\delta_\eta(x)\) is an additive discrepancy GP external to the simulator [2603.11960]. The integrated discrepancy approach instead moves the GP inside the simulator,
\[
y(x_i)=\eta(x_i,\theta_i+\delta_\theta(x_i))+e_i,
\]
so that discrepancy is interpreted as context-dependent calibration-parameter drift rather than as a catch-all output correction. Applied to dislocation dynamics, this allows the mismatch between Discrete Dislocation Dynamics and Molecular Dynamics to be expressed through effective drift in material parameters. The paper argues that this formulation is appropriate when the simulator is believed to be structurally correct and model-form errors can reasonably be attributed to uncertainty in the input parameters [2603.11960].

A closely related hybrid-modeling strategy appears in “Learning Discrepancy Models From Experimental Data” [1909.08574]. There, the discrepancy between measurement and a first-principles model is defined as
\[
\delta \Phi(t)=\Phi^*(t)-\Phi_m(t),
\]
and, for dynamics,
\[
\delta \dot{\mathbf x}(t)=\dot{\mathbf x}^*(t)-f_m(\mathbf x(t),\mathbf u(t);\mathbf p_1).
\]
SINDy is then used to learn a sparse discrepancy model from a library \(\Theta(\mathbf X,\mathbf U)\) via
\[
\delta \dot{\mathbf X}=\Theta(\mathbf X,\mathbf U)\Xi.
\]
The double-pendulum-on-a-cart experiments show that the learned discrepancy captures dissipative effects missing from a conservative model, and that incorporating the discrepancy improves feed-forward control in simulation [1909.08574].

Environmental data assimilation introduces another operator-level treatment of discrepancy. Interpolated Discrepancy Data Assimilation modifies the governing PDE not only by adding a nudging term but also by injecting interpolated discrepancy into the nonlinear operator:
\[
\frac{\partial v}{\partial t}=F[v+\tilde d]+D[v]+\lambda \tilde d.
\]
Under the stated dissipativity, Lipschitz, and interpolation assumptions, the error \(d=u-v\) satisfies
\[
\|d(x,t)\|\le e^{-\gamma t}\|d(x,0)\|,\qquad
\gamma=\lambda\alpha-\frac{L^2C^2h^2}{2\mu},
\]
with admissible parameter range
\[
\frac{L^2C^2h^2}{2\alpha\mu}<\lambda<\frac{\mu}{C^2h^2}.
\]
The reported numerical tests on Burgers flow, Kuramoto–Sivashinsky dynamics, and two-dimensional Navier–Stokes show faster attainment of target accuracy and greater stability than standard interpolated nudging, especially with sparse observations [2510.24944].

The climate-sensitivity study provides a statistical caution: the inferred quantity can depend materially on the assumed discrepancy structure [1809.04068]. When fitting an energy balance model to CMIP5 abrupt \(4\times CO_2\) output, the discrepancy process \(E_t\) was modeled as IID, AR(1), GP, or AR(1)+GP. For seven of twenty-one ESMs, the posterior ECS depended weakly on the discrepancy model; for four the difference was large; and for the rest it was moderate. The paper recommends AR(1) as an appropriate default because it captures temporal correlation while avoiding the severe GP identifiability problems observed with short time series [1809.04068].

## 6. Methodological themes, limitations, and related terminology

Several recurrent methodological patterns emerge. One pattern treats discrepancy as an additive correction external to a baseline model, as in KOH calibration and classical nudging. Another embeds discrepancy inside the dynamics, either as parameter drift \(\delta_\theta(x)\), as an environment-specific residual \(g_e\), or as operator modification through \(F[v+\tilde d]\). A third treats discrepancy as a policy-invariant value shift, \(Q_i-Q_{i-1}\), in order to decouple environment change from learning dynamics. A fourth treats discrepancy as disagreement between estimators, as in the \(\lambda\)-discrepancy for partial observability. These are non-equivalent constructions, but each is designed to separate environment-induced effects from confounding factors.

The literature also identifies several limitations. Teleoperation work shows that there may be no single scalar DoE metric adequate for perception and task performance; the discrepancy has to be inferred from transfer functions and response characteristics [2107.10784]. Bayesian calibration and climate inference show that flexible discrepancy models can introduce identifiability problems; GP hyperparameters can become weakly identified, and catch-all discrepancy terms can confound parameter interpretation [2603.11960; 1809.04068]. In integrated discrepancy calibration, the central assumption is restrictive: not all model inadequacy can be plausibly reduced to context-dependent parameter variation [2603.11960]. In multi-environment learning, ignoring discrepancy leads to bias, but over-parameterizing it risks data inefficiency [2106.04546].

A terminological caution is also necessary. The acronym “DoE” is overloaded. In the design-of-experiments literature, DoE refers to Design of Experiments rather than Discrepancy of Environment Dynamics, and “discrepancy” denotes sample-set uniformity metrics such as \(C^2\)-discrepancy or the proposed Newcomb-Benford discrepancy, whose computational complexity is reported as \(\mathcal O(N_s\times d)\) versus \(\mathcal O(N_s^2\times d)\) for classical discrepancy measures [2103.08705]. That usage is conceptually separate from dynamics discrepancy.

In aggregate, DoE names a broad research program concerned with how environment-dependent variation enters perception, identification, inference, control, and learning. The unifying question is not whether a model fits data in the abstract, but whether the dynamics that matter for action and inference remain invariant across interfaces, environments, and time.

Source: https://www.emergentmind.com/topics/discrepancy-of-environment-dynamics-doe