Adaptive Dynamics Planning (ADP)

• Adaptive Dynamics Planning is a family of methods that adapt planning by treating system dynamics as flexible, variable elements rather than fixed assumptions.
• It integrates Bellman/HJB frameworks, finite-horizon control, and uncertainty-aware replanning to develop near-optimal feedback laws and efficient trajectory tracking.
• Applications span robot navigation, manipulation, and vehicle path tracking, yielding measurable improvements in speed, collision reduction, and overall performance.

Adaptive Dynamics Planning (ADP) denotes a family of methods in which planning or feedback synthesis is adapted to dynamics, trajectory structure, or uncertainty rather than treating those elements as fixed design assumptions. In control and reinforcement learning, closely related work uses Adaptive Dynamic Programming to compute optimal or near-optimal feedback laws from Bellman or Hamilton–Jacobi–Bellman (HJB) equations, either from system models or from measured trajectories (Xing et al., 2020, Jiang et al., 2013). In robotics, the phrase is also used for learning-augmented planners that dynamically adjust dynamics fidelity in navigation (Yuanjie et al., 6 Oct 2025), for test-time adaptation of diffusion-policy sampling dynamics by geometric manifold and initial-noise constraints (Li et al., 8 Aug 2025), and for uncertainty-triggered replanning with generative models (Jutras-Dubé et al., 2024). This suggests a common thread: dynamics are treated as explicit objects of adaptation rather than fixed background assumptions.

1. Terminology and scope

The literature does not use the acronym ADP in a single, uniform way. In nonlinear optimal control, it usually denotes Adaptive Dynamic Programming, a continuous-time or discrete-time value-function methodology closely related to reinforcement learning and approximate dynamic programming; the ADPT toolbox paper explicitly states that this is “very close in spirit to what some authors call adaptive dynamic planning” (Xing et al., 2020). In trajectory-tracking control, the same lineage appears in model-free Q-function methods that learn optimal tracking controllers for unknown systems while incorporating a parametrized reference trajectory (Köpf et al., 2019). In robot navigation, Adaptive Dynamics Planning is a distinct learning-augmented paradigm in which a reinforcement-learning meta-controller selects the dynamics modeling configuration used by a planner at each planning step (Yuanjie et al., 6 Oct 2025). In diffusion-based robot manipulation, Adaptive Diffusion Policy is presented as a test-time adaptation method whose sampling dynamics are constrained by geometry and structured initialization, and it is explicitly interpreted as a concrete instantiation of “Adaptive Dynamics Planning” in action space (Li et al., 8 Aug 2025).

The acronym is also overloaded. In robust reinforcement learning, Active Dynamics Preference uses the same acronym ADP, but refers to active selection of training dynamics parameters using informativeness and density rather than Bellman-based optimal control or planner-level dynamics adaptation (Xu et al., 2022). For encyclopedia use, the term therefore requires contextual disambiguation: in some papers it is fundamentally a value-function method, while in others it is a meta-planning mechanism that adapts fidelity, sampling, or replanning cadence.

2. Bellman and HJB foundations

In the optimal-control lineage, ADP starts from a standard infinite-horizon control problem. For continuous-time input-affine systems,

$\dot{x} = f(x) + g(x)u,$

the objective is typically

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$

with $q$ positive definite and $R$ symmetric positive definite. The optimal value function $V^*(x)$ satisfies the static HJB equation, and the optimal feedback law is

$u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$

Because solving the HJB PDE exactly is generally intractable, ADP replaces exact value functions and policies by approximations and updates them through policy iteration, actor–critic learning, or least-squares Bellman-error minimization (Xing et al., 2020).

One important branch seeks stronger guarantees by exploiting structure. For continuous-time nonlinear polynomial systems, global ADP relaxes the HJB equality to the inequality $\mathcal{H}(V)\le 0$ and solves an optimization problem over polynomial value functions using sum-of-squares constraints. The induced policy

$$\bar u(x) = -\frac{1}{2}R^{-1}(x)g^\top(x)\nabla V(x)$$

is globally stabilizing for any feasible $V$, and the resulting policy iteration yields a monotone sequence $V_i$ with $$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$0, converging in the chosen polynomial class (Jiang et al., 2013). This distinguishes the SOS-based formulation from neural-network ADP methods that are typically local or semiglobal.

The same Bellman/HJB perspective appears in discrete time. In PMSM torque control, the infinite-horizon discounted objective

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$1

is solved by value iteration with critic and actor networks trained offline, then deployed online as a static nonlinear feedback map. The method is explicitly compared with FOC and DTC-SVM, and under realistic parameter uncertainties the ADP controller yields better torque and speed response than FOC (Khiabani et al., 2019). In stochastic optimization, Q-learning is similarly described as an ADP method that approximates the optimal value function by adaptively sampling state and action space; importance-sampling exploration is proposed there as an action-selection mechanism driven by the approximate value function (Kumar et al., 2021).

3. Tracking, preview, and finite-horizon control

A major development in ADP has been the treatment of tracking as a genuine trajectory-planning problem rather than regulation to a fixed point. For unknown discrete-time systems

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$2

one model-free formulation parameterizes the future reference by

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$3

with a shifted parameter matrix $$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$4 satisfying

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$5

This leads to a parametrized reference Q-function $$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$6, and the optimal control is recovered from

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$7

The key point is that the Q-function depends on a compact description of the entire future reference rather than on a fixed exo-system. Once learned, the controller handles time-varying reference trajectories without further training, and in the linear quadratic tracking special case the optimal Q-function is exactly quadratic in $$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$8 (Köpf et al., 2019).

This reference-parameterized idea was transferred from simulation to hardware in a large-scale ball-on-plate system. There, each axis is treated independently with state

$$J(x_0,u) = \int_0^\infty \big(q(x(t)) + u(t)^T R u(t)\big)\,dt,$$9

and the future reference is approximated by a quadratic polynomial

$q$0

The learned policy takes the affine form

$q$1

so the controller contains state feedback, trajectory-preview feedforward, and a constant offset term. The experiments show that the trajectory-tracking controller significantly reduces control cost compared to setpoint controllers and automatically compensates constant offset terms caused by plate asymmetry (Köpf et al., 2020).

Finite-horizon ADP has also been developed for automated vehicle path tracking. In that setting, continuous-time policy iteration is implemented with value and policy neural networks defined on $q$2, and the actor minimizes a finite-horizon Hamiltonian while the critic enforces a self-consistency relation derived from the HJB equation. On the reported path-tracking benchmark, the method obtains a near-optimal policy with $q$3 error and is almost $q$4 times faster than the nonlinear MPC IPOPT solver; for a horizon length $q$5, ADP is about $q$6 faster than the fastest QP solver in the linear comparison (Lin et al., 2020).

4. Uncertainty-aware replanning with learned models

A different ADP strand treats planning itself as an adaptive scheduling problem over when and how often to replan. In model-based offline RL with diffusion models, the planner first samples a long-horizon state trajectory

$q$7

then converts predicted consecutive states to actions through an inverse-dynamics model. Instead of replanning at every step, uncertainty is estimated by a deep ensemble. In the NLL-based formulation, the ensemble mean action is

$q$8

and the predictive uncertainty is

$q$9

with replanning triggered when $R$0 exceeds a threshold $R$1 (Jutras-Dubé et al., 2024).

This yields an adaptive receding-horizon policy: the same generated trajectory can be reused for multiple consecutive actions when uncertainty remains small, but replanning frequency automatically increases when the prediction becomes unreliable. On D4RL locomotion tasks, the method reduces replanning frequency to only about $R$2 of the steps without compromising performance, and on a 1000-step Hopper episode it yields approximately a $R$3 speedup over Decision Diffuser in wall-clock time (Jutras-Dubé et al., 2024). The formulation is notable because the adaptation occurs at deployment, not by retraining the generative model.

5. Robotics-specific formulations: navigation and manipulation

In mobile robot navigation, Adaptive Dynamics Planning is explicitly defined as a learning-augmented paradigm in which a reinforcement-learning policy selects the dynamics modeling configuration used by a classical planner. The physical robot evolves according to

$R$4

where $R$5 is the dynamics configuration chosen at time $R$6. The ADP state is

$R$7

combining laser-scan observations, the previous dynamics configuration, and goal-related features; the ADP action is the next configuration $R$8. In the decremental strategy, time discretization is parameterized by

$R$9

so the planner can allocate high fidelity near the present and coarser fidelity later in the rollout. The policy is trained with TD3 and integrated into DWA, MPPI, Log-MPPI, and a standalone navigation system (Yuanjie et al., 6 Oct 2025).

The reported gains are substantial in cluttered environments. On the 225-environment BARN test set, the ADP navigation system at $V^*(x)$0 increases success from $V^*(x)$1 to $V^*(x)$2, reduces average traversal time from $V^*(x)$3 to $V^*(x)$4, and lowers collisions from $V^*(x)$5 to $V^*(x)$6 relative to the fixed decremental-dynamics baseline (Yuanjie et al., 6 Oct 2025). The same paper reports that the learned decremental strategy outperforms both a hand-crafted decremental schedule and an “incremental” schedule that places high fidelity late rather than early.

In robot manipulation, ADP appears as a test-time adaptive diffusion policy. A pretrained diffusion policy generates actions by reverse denoising, but the reverse dynamics are modified by two constraints. First, an initial noisy action is replaced by a structured initialization obtained through Fast Global Registration,

$V^*(x)$7

where $V^*(x)$8 and $V^*(x)$9 are the gripper and scene point clouds. Second, denoising is guided by a geometric manifold constraint derived from the Chamfer-distance gradient between the transformed gripper cloud and the scene cloud. The resulting update is

$u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$0

which constrains the reverse diffusion dynamics to remain aligned with a task-relevant geometric direction (Li et al., 8 Aug 2025).

This manipulation formulation requires no retraining and is compatible with pretrained diffusion policies. The reported implementation, ADPro, improves success rates, generalization, and sampling efficiency, achieving up to $u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$1 faster execution and $u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$2 points over strong diffusion baselines (Li et al., 8 Aug 2025). In this usage, ADP is not a Bellman method; it is a test-time planner that adapts the generative sampling dynamics themselves.

6. Guarantees, limitations, and adjacent developments

The guarantee profile of ADP depends sharply on which lineage is under discussion. The strongest global results are available for polynomial systems with SOS relaxations, where each policy generated by the iteration is globally stabilizing and the limit value function remains in $u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$3 (Jiang et al., 2013). Safety-critical variants of adaptive dynamic programming go further by augmenting the state with a barrier state $u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$4, writing the safe-set dynamics as

$u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$5

and defining the policy through a value function that explicitly penalizes the barrier coordinate. In that framework, safety is embedded directly into the Bellman structure, and closed-loop stability and safety are established with a barrier-state Lyapunov function (Satharasi et al., 1 Jan 2026).

More common ADP implementations make weaker claims. The ADPT toolbox emphasizes local optimality around a stabilizable equilibrium, polynomial basis functions, and dependence on sufficiently rich data; it notes the curse of dimensionality for monomial bases and the lack of a formal proof for convergence under arbitrary data and approximation degree (Xing et al., 2020). The time-varying-reference tracking method provides exact recovery in LQT and LSPI-type bounds, but for general nonlinear systems it does not provide full Lyapunov proofs (Köpf et al., 2019). The uncertainty-triggered replanning scheme based on diffusion models measures uncertainty only through the inverse-dynamics ensemble rather than the generative model itself, and its threshold $u^*(x) = -\frac{1}{2} R^{-1} g(x)^T \nabla V^*(x).$6 remains a deployment hyperparameter (Jutras-Dubé et al., 2024). The navigation formulation reports strong simulation and real-world improvements but does not present formal convergence or optimality theorems (Yuanjie et al., 6 Oct 2025).

Adjacent developments show how broad the “adaptive dynamics” idea has become. In robust RL, Active Dynamics Preference quantifies informativeness by GAE magnitude and density by visitation counts, then selects training dynamics with high informativeness and low density; this yields an implicit curriculum over system parameters rather than an HJB solver (Xu et al., 2022). In stochastic optimization, importance-sampling-based exploration in Q-learning uses the approximate Q-function itself to shape exploration in continuous action spaces, framing ADP as adaptive sampling of both action and state space (Kumar et al., 2021). Across these variants, a recurrent misconception is that ADP names a single algorithm. The literature instead presents a spectrum of methods united by one design choice: the representation, fidelity, or sampling of dynamics is adapted as part of planning rather than fixed in advance.