Minimax optimal dual control -- The single input case

Abstract: An explicit solution is derived for the Bellman inequality corresponding to minimax optimal dual control. The minimizing player determines control action as a function of past state measurements and inputs. The maximizing player selects disturbances and model parameters for the underlying linear time-invariant dynamics. The optimal minimizing policy is a dual controller that optimizes the tradeoff between exploration and exploitation. Once sufficient data has been collected, the policy becomes a deterministic certainty equivalence controller. However, when data is insufficient, the policy introduces a randomized term to improve excitation.

• The paper derives a dual control strategy that blends deterministic exploitation with randomized exploration to guarantee a minimax ℓ2-cost bound under uncertainty.
• It formalizes the control trade-off using a Bellman inequality while characterizing the uncertainty set through convex spectral cone constraints.
• The study challenges conventional adaptive control by demonstrating that linear time-invariant feedback fails under sign uncertainty and instability, necessitating nonlinear randomized approaches.

Minimax Optimal Dual Control for Single Input Systems

Problem Formulation and Theoretical Framework

The paper "Minimax optimal dual control -- The single input case" (2604.18550) rigorously addresses the problem of robust dual control in discrete-time, linear time-invariant systems with uncertain input matrices. The controller is designed to achieve a minimax optimal trade-off between performance and learning under adversarial disturbance and parameter uncertainty. The system dynamics are given by

$x_{t+1} = Ax_t + Bu_t + w_t,$

where $A$ is known, $B$ lies in a bounded set $\mathcal{B}$ (norm-constrained and possibly sign-uncertain), and $w_t$ represents disturbances. The dual control problem is cast as a game where the minimizing player (controller) acts causally based on accumulated data, and the maximizing player simultaneously selects disturbances and unknown $B$ within prescribed uncertainty bounds.

The minimax control objective is formalized through a Bellman inequality:

$$E\sum_{t=0}^T \left(|x_t|_{S}^2 + |u_t|_{R}^2\right) \le \gamma^2 \sum_{t=0}^T |w_t|^2,$$

for all admissible $B \in \mathcal{B}$, providing an explicit $\ell_2$ gain bound from disturbances to cost.

The uncertainty set $\mathcal{B}$ is structurally characterized using spectral properties of $A$0 and quadratic constraints, resulting in a union of convex second-order cones. This delineates fundamental feasibility limits: when sign uncertainty and instability are present, linear time-invariant feedback is insufficient, necessitating randomized nonlinear strategies.

Explicit Minimax Dual Control Policy

The main theoretical advance is the derivation of an explicit feedback law $A$1 that achieves minimax optimality with respect to the Bellman inequality. The controller adapts its operation based on the informativeness of accumulated data:

• Exploitation: When parameter uncertainty is sufficiently reduced by past excitation, the policy converges to deterministic certainty equivalence control, using the best estimate $A$2:

$A$3

• Exploration: When data is insufficient to confidently estimate $A$4, the controller introduces a randomized excitation term. This actively excites the system to improve parameter identification, ensuring persistent excitation for learning while guaranteeing robust performance.

The dual controller thus integrates optimality in exploitation with necessary excitation for exploration directly into a minimax framework, with the transition between regimes governed by explicit thresholds derived from system-theoretic quantities and the Bellman inequality.

Strong Numerical Guarantees and Contradictory Claims

The paper provides strong quantitative performance guarantees: for any sequence of disturbances and for all admissible $A$5, the expected total cost over finite horizon is upper-bounded by $A$6 times the summed disturbance energy. This bound holds whether the controller operates in the certainty equivalence regime or the randomized excitation regime.

A notable theoretical claim is that, in scenarios where sign uncertainty of $A$7 and instability of $A$8 coexist, stabilization cannot be achieved by linear time-invariant feedback, directly contradicting common adaptive control assumptions. Nonlinear, randomized strategies are necessary, and the explicit controller synthesis offers a constructive route.

Implications and Future Directions

Practically, the explicit minimax dual controller enables robust performance in settings where model uncertainty is adversarial and non-probabilistic, relevant to safety-critical or adversarial environments. The approach applies to adaptive and dual control scenarios where exploration cannot be decoupled from exploitation.

Theoretically, this formulation advances the rigorous synthesis of dual controllers under norm-bounded uncertainties, extending previous results for sign uncertainties to general stabilizable sets. The use of Bellman inequalities, spectral cone characterization, and minimax game theory offers a template for further generalizations—including multi-input/multi-output systems, non-convex parameter sets, and nonlinear dynamics.

Speculatively, future developments may include removal of the assumption on known $A$9, extension to more general uncertainty sets for $B$0, and integration with data-driven online learning architectures. Application to RL scenarios with robust safety guarantees and adaptation in continuous control will leverage these foundational results.

Conclusion

The paper delivers an explicit minimax optimal dual control policy for single-input LTI systems with norm-bounded parameter uncertainty and adversarial disturbances. By seamlessly blending deterministic exploitation with randomized exploration, the controller achieves robust performance guarantees encapsulated in a Bellman inequality. Theoretical findings highlight limitations of conventional adaptive strategies and provide constructive nonlinear control design under worst-case uncertainty scenarios, offering a rigorous basis for robust dual adaptive control in future AI-enabled systems.