DEM Control Law in Discrete-Time Systems

• Discrete-time Error Minimization (DEM) control law is a feedback strategy that minimizes error signals at each time step using local optimization techniques, including gradient and least-squares updates.
• It employs methods such as the Pontryagin Maximum Principle and adaptive tuning to ensure convergence and stability, with rigorous error estimates and Lyapunov-based guarantees.
• DEM is applied in diverse scenarios like deep learning, embedded control, and distributed systems to optimize performance, manage quantization errors, and enhance real-time regulation.

Discrete-time Error Minimization (DEM) control law constitutes a family of feedback designs and algorithmic strategies for minimizing error signals in discrete-time dynamical systems, motivated by optimal control, adaptive control, and robust real-time regulation. These approaches – spanning from gradient-based and least-squares parameter adaptation to Hamiltonian-based layerwise optimization – share the common objective of explicit minimization of error dynamics or prediction/tracking losses on a stepwise or cumulative basis. DEM formulations have proven influential in fields such as optimal adaptive regulation, embedded control under quantization error, deep learning interpreted as optimal control, and distributed implementation error management.

1. Core Principles and Mathematical Formulation

The central tenet of DEM control law is the minimization, at each discrete time step (or stage/layer), of a function measuring the immediate or predicted system error – typically via a local optimization, projection, or adaptive update.

In its canonical form for a discrete-time system: $x_{k+1} = f_k(x_k, u_k)$ with $x_k$ the state and $u_k$ the control, the task is to select the control sequence $\{u_k\}$ to minimize a cost functional: $J(u) = \Phi(x_N) + \sum_{k=0}^{N-1} L_k(x_k, u_k)$ where $\Phi$ penalizes terminal state error and $L_k$ encodes intermediate penalties or regularization (Li et al., 2018). DEM principles are then operationalized by local minimization or update rules exploiting discrete-time optimality conditions, e.g., Pontryagin maximum principle (PMP), gradient/recursive least-squares updates, or per-step error projections (Rodrigues, 2022, Fisher et al., 2022, Tao, 2023, Zhao et al., 25 Mar 2024).

2. Pontryagin-Based Discrete-Time DEM (Method of Successive Approximations)

A system-theoretic foundation of DEM control is established via the discrete-time PMP applied to sequence optimization in high-dimensional spaces, notably in optimal training of neural networks (Li et al., 2018). Here, state and costate (adjoint) recursions are:

• State: $x_{k+1} = f_k(x_k, u_k)$
• Costate: $$\lambda_k = \partial_{x_k} H_k(x_k, u_k, \lambda_{k+1})$$
• Hamiltonian: $$H_k(x, u, \lambda) = L_k(x, u) + \lambda^\top f_k(x, u)$$

The DEM step is: $$u_k^{(i+1)} = \arg\max_{u \in \mathcal U_k} \left[ H_k(x_k^{(i)}, u, \lambda_{k+1}^{(i)}) - \frac{\rho}{2}\|u - u_k^{(i)}\|^2 \right]$$ where $\rho$ is a proximity parameter. In unconstrained, smooth settings, this yields a (projected) gradient ascent step: $$u_k^{(i+1)} = u_k^{(i)} + \eta_k \partial_u H_k(x_k^{(i)}, u_k^{(i)}, \lambda_{k+1}^{(i)})$$ A rigorous error estimate guarantees descent under regularity conditions: $J(u^{(i+1)}) \le J(u^{(i)})$ and ensures accumulation points satisfy PMP optimality (Li et al., 2018).

3. Error-Minimizing Adaptive and Least-Squares DEM Schemes

Adaptive DEM laws incorporate real-time parameter updates to minimize error in uncertain or drifting environments. Discrete-time high-order tuner (HOT) algorithms implement a Nesterov-accelerated gradient scheme to minimize convex stepwise prediction error: $$L_k(\Theta) = \frac{1}{2}\|\varepsilon(k+1)\|^2, \quad \varepsilon(k+1) = \Theta(k)\phi(k)$$ The DEM law for control is: $u(k) = \Theta(k)\,\phi(k) + r(k)$ with $\Theta$ updated by normalized momentum and gradient steps (Fisher et al., 2022). Stability is guaranteed by a composite Lyapunov function $V(k)$, with $\|\varepsilon(k)\|\to 0$ and tracking error converging to zero under mild plant and regressor assumptions.

In least-squares-based adaptive DEM (Zhao et al., 25 Mar 2024), parameters are updated to minimize the accumulative estimation error: $$J(\theta(t)) = \frac{1}{2}\sum_{\tau=0}^{t-1}\frac{1}{\kappa}\sum_i [\mu_i(\tau) + \theta(t)^\top\zeta_i(\tau)]^2 + \frac{1}{2}(\theta(t)-\theta_0)^T P_0^{-1} (\theta(t)-\theta_0)$$ With closed-form recursive update: $$\theta(t+1) = \theta(t) - P(t-1)Z(t)N^{-1}(t)\epsilon(t)$$ and control law: $u(t) = \Theta_2^{-1}(t)(\Theta_1^T(t)x(t) + r(t))$ Stability and asymptotic tracking are established via parameter convergence and operator-small-gain arguments (Zhao et al., 25 Mar 2024, Tao, 2023).

4. Practical DEM in Control-Affine and Embedded Systems

DEM as a stepwise error-minimization explicit quadratic program arises naturally in control-affine systems subject to quadratic costs: $x_{k+1} = f(x_k) + g(x_k)u_k$ The one-step stage cost: $$J_1(u_k; x_k) = u_k^T R(x_k) u_k + \gamma [f(x_k) + g(x_k) u_k]^T P [f(x_k) + g(x_k) u_k]$$ is minimized by: $$u_k^* = -\gamma [R(x_k) + \gamma g(x_k)^T P g(x_k)]^{-1} g(x_k)^T P f(x_k)$$ This characterizes DEM as a regularized projection of $-f(x_k)$ onto the control directions, trading off error reduction in the reference metric and input regularization (Rodrigues, 2022).

In embedded software, DEM addresses quantization error by jointly minimizing conventional LQR/LQG costs and the size of the practical stability region induced by fixed-point arithmetic (Majumdar et al., 2012). Static analysis tools compute the implementation error, and PSO drives the controller design toward Pareto-optimality with respect to both dynamic performance and error robustness.

5. DEM and Invariant-Set Error Management in Real-Time Control

DEM is also deployed as “error-diffusion” or “accumulated error correction” in practical real-time distributed control contexts, particularly when exact implementation of requested setpoints is infeasible. At each step, the system selects the next feasible action $y_n \in \mathcal S_n$ that minimizes the predicted accumulated deviation: $$y_n = \arg\min_{s \in \mathcal S_n} \|x_n + E_n - s\|, \quad E_{n+1} = E_n + (y_n - x_n)$$ This greedy greedy error-minimization ensures that the accumulated error $E_n$ evolves within a computable minimal convex invariant set $Q^*$, leading to average error vanishing at rate $O(1/n)$ under polytope and bounded cell conditions (Bernstein et al., 2016). This invariant-set characterization provides strong guarantees of long-term tracking fidelity in uncertain, time-varying, or quantized networked settings.

6. Convergence Properties and Stability Guarantees

The convergence and robustness of DEM laws are substantiated through rigorous Lyapunov-based, operator-theoretic, and Pontryagin-based analyses:

• For Pontryagin-based DEM, error estimates (by discrete Gronwall and Taylor expansions) show that exact or sufficiently accurate argmax Hamiltonian updates guarantee monotonic non-increase of the cost and convergence to critical points of the PMP system (Li et al., 2018).
• Adaptive and least-squares DEM schemes establish parameter and tracking error boundedness via strict Lyapunov functions, with convergence in the $\ell^2$-sense for error signals and stability under plant-model matching and bounded reference/initialization (Fisher et al., 2022, Tao, 2023, Zhao et al., 25 Mar 2024).
• In quantized embedded systems, the practical stability radius is explicitly controlled and can be minimized far below standard LQR design at marginal cost in quadratic performance (Majumdar et al., 2012).
• For real-time discrete DEM error-diffusion, the accumulated error remains in a predefined bounded set, yielding $O(1/n)$ convergence of the average error norm (Bernstein et al., 2016).

7. Applications and Implementation Considerations

DEM control law methodologies have been realized in various contexts:

• Deep learning, where the method of successive approximations based on the discrete-time PMP interprets layerwise weight updates as DEM steps, even for discrete-weight constraints (Li et al., 2018).
• Embedded controller synthesis, where DEM yields fixed-point control code with jointly optimized performance and minimized quantization-induced error regions, using static analysis and PSO (Majumdar et al., 2012).
• Adaptive tracking for nonlinear, unknown, or time-varying plants, via normalized high-order gradient and least-squares DEM adaptation (Fisher et al., 2022, Tao, 2023, Zhao et al., 25 Mar 2024).
• Distributed resource allocation and networked control, where DEM as error diffusion guarantees long-term tracking despite implementation imprecision (Bernstein et al., 2016).
• Control-affine and quadratic optimal policy computation, both in standard settings and with connections to inverse optimal control and reinforcement learning (Rodrigues, 2022).

Practical deployment relies on choosing appropriate normalization, step-sizes, gain bounds, and, where necessary, bound or constraint projections to ensure stability and avoid numerical pathologies. In multi-agent systems, DEM controllers can be augmented with external safety mechanisms (e.g., artificial repulsive fields) for additional robustness and constraint handling (Zhao et al., 25 Mar 2024).

---

References:

• (Li et al., 2018) An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
• (Rodrigues, 2022) Inverse Optimal Control with Discount Factor for Continuous and Discrete-Time Control-Affine Systems and Reinforcement Learning
• (Majumdar et al., 2012) Synthesis of Minimal Error Control Software
• (Fisher et al., 2022) Discrete-Time Adaptive Control of a Class of Nonlinear Systems Using High-Order Tuners
• (Tao, 2023) Discrete-Time Adaptive State Tracking Control Schemes Using Gradient Algorithms
• (Zhao et al., 25 Mar 2024) A Discrete-Time Least-Squares Adaptive State Tracking Control Scheme with A Mobile-Robot System Study
• (Bernstein et al., 2016) Real-Time Minimization of Average Error in the Presence of Uncertainty and Convexification of Feasible Sets