Stable Transformer-Actor-Critic Model Predictive Control: A Contraction Analysis Approach

Abstract: Actor-Critic Model Predictive Control (MPC) effectively addresses complex, non-convex control problems, but guaranteeing the closed-loop stability of sequence-based learning models within these pipelines remains challenging. This paper introduces a novel Transformer-Actor-Critic MPC architecture with formal robustness guarantees. First, we prove that Transformer networks can satisfy global incremental Input-to-State Stability ($δ$ISS). We then leverage Riemannian contraction theory to analyze the interconnected dynamics between the physical plant and the predictive neural network. Finally, we integrate these theoretical bounds as a training regularizer to yield a certifiably robust policy. The framework is validated on a nonlinear 3D drone model executing target-reaching and obstacle-avoidance maneuvers.

• The paper presents a certifiably robust Transformer-Actor-Critic MPC design that guarantees closed-loop stability through contraction analysis and incremental input-to-state stability (SISS).
• It replaces standard MLPs with transformer sequence models to capture temporal context, employing a small-gain condition to ensure interconnection stability in nonlinear environments.
• Empirical validation on a 3D drone model demonstrates improved safety and predictable performance, highlighting a trade-off between stability guarantees and agile maneuverability.

Certifiably Robust Transformer-Actor-Critic Model Predictive Control via Contraction Analysis

Introduction

The paper "Stable Transformer-Actor-Critic Model Predictive Control: A Contraction Analysis Approach" (2606.20197) systematically addresses the challenges of guaranteeing closed-loop stability in learning-based MPC architectures, specifically those leveraging sequential neural models for dynamic optimization of nonlinear systems. Existing Actor-Critic MPC pipelines, though powerful in shaping the control cost landscape, lack rigorous stability assurances and are typically constructed from simple feedforward MLPs, which do not leverage temporal consistency critical for robust performance under time-correlated disturbances. The proposed architecture replaces MLPs with transformer sequence models, enabling context-aware predictions while yielding formal guarantees rooted in incremental Input-to-State Stability (SISS) and contraction theory.

Technical Approach

Model Architecture

The proposed architecture comprises three key stages:

1. Initial MLP for Feature Extraction: Physical history of past states and references is embedded into latent variables.
2. Transformer Sequence Model: Temporal context is exploited via attention mechanisms to predict structured parameters $\{Q_{i|k}, P_{i|k}\}_{i=0}^T$ for the MPC cost function, fully enabling sequence-based adaptation.
3. Final MLP Projection: Ensures output dimensionality and positivity constraints on MPC weights (notably via softplus mapping of diagonals).

The choice of transformer sequence modeling is crucial for environments with significant temporal correlation in disturbances and requirements for history-dependent control synthesis.

Stability Analysis

Incremental Input-to-State Stability (SISS) of Transformers

A rigorous discrete dynamical system interpretation is leveraged, providing an explicit SISS proof for transformer blocks under bounded inputs and stabilizing layer normalization. The derived Lipschitz constants characterize transformer sensitivity to input trajectories, enabling quantification of bounded state deviations.

The main result states that, under bounded input and appropriately chosen stabilization $\epsilon$, transformer blocks satisfy:

$$|S z_{k}|_\infty \leq A_s|S z_0| + (1 - A_s)^{-1} B_s |S p|_\infty$$

where $A_s$ and $B_s$ are explicit functionals of transformer weights and attention-projection matrices.

Riemannian Contraction Theory for Closed-Loop Analysis

Closed-loop dynamics are analyzed by anchoring the system trajectory in a Riemannian manifold defined by contraction matrices, computed via Riccati recursions with terminal weights derived from transformer outputs. This framework circumvents the conservatism of infinity-norm bounds for physical state transitions.

A critical result is the derivation of a global uniform contraction rate:

$p_{\mathrm{mpc}} = 1 - \frac{q}{m}$

where $q$ is the minimum eigenvalue of the stage cost and $m$ the maximum eigenvalue of the contraction metric. This rate guarantees exponential decay of trajectory deviations, provided $q > 0$ and $m < \infty$.

Small-Gain Condition for Interconnected Stability

The full interconnected system—physical plant, predictive neural network, and differentiable solver—is subject to a contraction-based small-gain condition:

$\epsilon$0

where cross-domain sensitivity terms $\epsilon$1 and $\epsilon$2 are analytically linked to MPC solver sensitivity and transformer’s SISS constants. This condition provides a certifiable bound for robust closed-loop contraction and is enforced as a regularization objective during network training.

Empirical Validation

A 3D drone model operating in highly nonlinear, obstacle-cluttered environments is used for validation. The drone's state includes position, linear and angular velocities, and quaternion-based orientation; control inputs are thrust and torques. The controller is trained end-to-end via differentiable MPC frameworks with horizon $\epsilon$3, subject to severe observation and process disturbances.

Key Numerical Results

• Robust Convergence: The regularized transformer-actor-critic network maintains robust flight stability and task completion under maximal injected noise ($\epsilon$4, $\epsilon$5). The tracking error remains tightly bounded and spatial trajectory rollouts are highly correlated, reflecting the theoretical SISS property.
• Safety Under Disturbance: In obstacle avoidance scenarios, the stable network yields zero collision events under maximal disturbance, whereas the baseline exhibits a 30% collision rate and catastrophic divergence.
• Attitude Tracking Trade-off: Strict enforcement of contraction bounds slightly increases the standard deviation of attitude tracking errors (approx. $\epsilon$6 rad), suggesting some loss in aggressive transient maneuverability. This demonstrates the trade-off between uniform stability guarantees and local agility, motivating future exploration of state-dependent contraction metrics.

Practical and Theoretical Implications

This work marks a significant advance in certifiable robust control with learning-based architectures in nonlinear domains. It provides a mathematical bridge between modern sequence models (transformers) and classical MPC stability theory through SISS and contraction analysis—enabling actor-critic MPC variants to be deployed in safety-critical scenarios. The approach offers a scalable and versatile pathway for integrating advanced temporal neural models into differentiable closed-loop optimization.

The small-gain contraction regularization can be generically extended to other sequence modeling frameworks, including Vision-Language-Action systems and reinforcement learning in robotics. The empirical demonstration confirms that enforcing structural stability properties at the network level substantially improves operational safety and predictability.

Future Directions

Future research will likely focus on:

• State-dependent contraction metrics: Relaxing global bounds to allow greater transient maneuverability while retaining local stability.
• Generalization to multi-agent and higher-dimensional domains: Extending the approach to distributed control and coordination.
• Integration with vision and language modalities: Leveraging context-aware transformers for end-to-end task adaptation in real-world robotic environments.
• Formal robustness certification for reinforcement learning models: Bridging theories such as SISS and contraction with RL policy stability.

Conclusion

The paper presents a certifiably robust Transformer-Actor-Critic MPC architecture that guarantees closed-loop stability via contraction analysis and SISS. The approach rigorously unites sequence modeling transformers with differentiable MPC, providing strong theoretical bounds and empirical evidence for robust performance under severe disturbances and dynamic uncertainty. The method is well-positioned for extension to broader classes of learning-based control in practical AI systems, with attention to flexibility-stability trade-offs in future developments.