Gradient-Based Model Predictive Control

• Gradient-Based Model Predictive Control is a method that uses derivative information to update control inputs and policies efficiently across convex and nonlinear problems.
• It leverages first- and second-order gradient techniques, warm-starting, and iterative descent algorithms to meet real-time performance in various control applications.
• Applications include robotics, distributed systems, and model-based reinforcement learning, offering proven convergence rates and closed-loop stability guarantees.

Gradient-Based Model Predictive Control

Gradient-Based Model Predictive Control (GB-MPC) refers to a class of algorithms for solving the optimization problems at the core of Model Predictive Control (MPC) using gradient-based numerical optimization techniques. Instead of solving the associated optimal control problem via direct enumeration, interior-point or active-set QP solvers, or sampling-based population methods, these approaches use first- or second-order derivative information—obtained either analytically or via automatic differentiation—to efficiently update controller parameters, control sequences, or policy surrogates. This paradigm applies to both linear/quadratic (convex) settings and to nonlinear, nonconvex, and learning-based MPC formulations, and encompasses both centralized and distributed scenarios.

1. Fundamental Principles

In standard MPC, the controller, at each time step, solves a finite-horizon optimal control problem of the form:

$$\begin{aligned} \min_{u_{0:N-1}} \quad & J(x_0,u_{0:N-1}) = \ell_f(x_N) + \sum_{k=0}^{N-1} \ell(x_k, u_k) \ \text{s.t.} \quad & x_{k+1} = f(x_k, u_k),\quad x_0 = \text{current state}, \ & x_k \in \mathcal{X},\quad u_k \in \mathcal{U} \; \forall k, \end{aligned}$$

where $\ell$ is the stage cost, $\ell_f$ the terminal cost, $f$ the system (possibly nonlinear) dynamics, and $N$ the prediction horizon. In GB-MPC, this finite-horizon problem is cast and solved as a differentiable optimization, leveraging:

• Analytical or automatic computation of the gradients (first or higher order).
• Iterative descent algorithms (e.g., gradient descent, projected gradient, Nesterov acceleration, Adam, L-BFGS, conjugate gradient, FISTA, proximal-gradient, or hybrid approaches).
• Potentially, warm-starting using solutions from previous time steps to accelerate convergence.

The gradient-based approach also extends, in various forms, to distributed MPC, primal-dual optimization, learning-based surrogates for explicit MPC, and hybrid schemes using gradient steps within sampling- or population-based frameworks.

2. Algorithmic Variants: Centralized, Primal-Dual, and Distributed Gradient Methods

First-Order Projected Gradient and Proximal Methods

Classical and accelerated projected gradient methods solve condensed or lifted QPs arising in MPC for convex quadratic problems with state and input constraints. Projected gradient descent or Nesterov-type acceleration is employed, often with constant or adaptively scheduled step-sizes, with convergence rates up to $O(1/k^2)$ for strongly convex objectives. Modern approaches further generalize to arbitrary order momentum parameters, yielding provable error bounds of $O(1/k^\alpha)$ for integer $\alpha \geq 2$ (Wang et al., 2021).

Table 1: Summary of Core Centralized Gradient-Based Methods

Algorithm	Gradient Step Type	Rate (Convex/Strong Convex)	Constraints Handling
Projected Gradient	First-order	O(1/k) / O(1/k²)	Simple projection
Nesterov/FISTA	Accelerated	O(1/k²)	Simple projection
High-order accelerated	Polynomial momentum	O(1/kᵅ), α≥2	Simple projection

Projected gradient methods are efficiently applicable to QPs with additional constraints (e.g., input rate/amplitude), often utilizing methods such as Dykstra's algorithm for projections onto intersections of box and diamond constraints (Kempf et al., 2020). For more complex constraints (e.g. semi-definite, continuous-time path), primal-dual methods or splitting algorithms (PDHG) are used, allowing full exploitation of block and parallel structure (Li et al., 2023).

Primal-Dual and Proportional-Integral Methods

Primal-dual gradient methods, including proportional-integral (PI) projected gradient and sampled-data primal-dual dynamics, alternate between primal updates (projected gradient step in the control/trajectory space) and dual updates (Lagrange multipliers for constraints), achieving convergence rates O(1/k) or improved rates under strong convexity (Yu et al., 2020, Moriyasu et al., 2024). Discrete-time PDG variants admit real-time parallel implementations with rigorous Lyapunov-based stability certificates under mild assumptions.

Distributed and Consensus-Based Gradient MPC

Gradient-based distributed MPC formulations utilize block-separable objective functions and coupling constraints, with each agent or subsystem updating its local primal and dual variables using gradient steps and exchanging information with neighbors to ensure consensus on coupling constraints. Dual decomposition methods with accelerated gradient ascent (Nesterov momentum) have been demonstrated on large-scale hydro power systems, yielding convergence rates O(1/k²) and near-centralized performance (Doan et al., 2013). Contractive distributed PDG methods with Laplacian consensus ensure recursive feasibility and closed-loop stability under inexact optimization (Su et al., 2019).

3. Nonlinear, High-Frequency, and Learning-Based GB-MPC

Real-Time Nonlinear and Accelerated First-Order Solvers

For general nonlinear systems, the full trajectory optimization problem is solved by unrolling the dynamics and back-propagating gradients through time. This enables single-shooting/adjoin-based computation of the cost and its gradient with respect to the entire control sequence, which can then be minimized via accelerated first-order solvers, such as Adam or L-BFGS, with warm-starting strategies to achieve high-frequency rates (e.g., 1 kHz on robotic manipulators) (Zhang et al., 2024). In these applications, first-order gradient methods (Adam, Nesterov) can yield closed-loop performance competitive with second-order (DDP, SQP) solvers but with reduced per-iteration complexity and eliminated need for line search.

Learning Surrogates for Explicit MPC: Gradient Supervision

To address the curse of dimensionality in explicit MPC (eMPC), which defines the optimal law as a piecewise affine (PWA) function over exponentially many regions, neural networks with ReLU activations are trained to mimic eMPC. Crucially, performance and data efficiency are dramatically improved when, in addition to state-control pairs, the Jacobian $\partial u^*(x)/\partial x$ from the differentiable QP solver is included in the loss function. This joint imitation- and gradient-matching approach accelerates learning and yields neural approximators with fixed capacity independent of the number of PWA regions, maintaining high closed-loop performance with much smaller datasets (Winqvist et al., 2021).

Model-Based RL and Latent-World MPC: Backpropagation-based Planning

Recent approaches in model-based reinforcement learning (MBRL) use fully differentiable world models (latent dynamics, reward, and value networks) enabling direct gradient-based optimization of action sequences within the MPC loop. Strategies such as Dream-MPC "imagination" pipelines warm-start candidate action sequences with policy priors, apply gradient ascent with uncertainty penalization, and reuse previous solutions via amortization to avoid local minima and reduce computational cost (Spieler et al., 6 May 2026). Hybrid CEM-plus-gradient methods interleave population-based search (CEM) with local gradient-based refinement, mitigating local minima and improving sample/compute efficiency in high-dimensional spaces (Bharadhwaj et al., 2020).

4. Advanced Problem Structures and Hardware Acceleration

Path Constraints, Barrier Functions, and Safety Guarantees

Gradient methods can be extended to handle complex path constraints and safety requirements by:

• Relaxing hard state constraints into cost penalties and optimizing the soft-constrained MPC problem with a first-order method (L-BFGS or Adam), followed by a Control Barrier Function Quadratic Program (CBF-QP) that enforces strict safety via minimal modification of the nominal control (Singh et al., 18 Jul 2025).
• Reformulating continuous-time linear MPC with differential flatness to polynomials, encoding path constraints via Sum-of-Squares relaxations, and solving the resulting block-structured SDP using parallel PDHG algorithms, fully leveraging GPU computation (Li et al., 2023).

Trajectory Optimization with Online Dimension Reduction

Gradient-based trajectory optimization in high-dimensional systems (e.g., non-prehensile manipulation, soft robotics) suffers from prohibitive computational cost. Online state vector reduction approaches prune task-irrelevant degrees of freedom based on feedback gain metrics extracted during iLQR backward passes, reducing update complexity and improving "policy lag," which directly improves closed-loop tracking and cost (Russell et al., 2024).

GPU Based Large-Scale and Real-Time NMPC

Accelerated real-time NMPC for complex robots is enabled by leveraging sparse structure in the Gauss-Newton (SQP) KKT system with custom parallel implementations of block-preconditioned conjugate gradient (PCG) solvers on GPU. The MPCGPU architecture achieves kilohertz solve rates for horizons with hundreds of steps, outperforming state-of-the-art CPU solvers by large factors (Adabag et al., 2023).

5. Analysis of Convergence, Stability, and Implementation

Gradient-based MPC solvers achieve rates of convergence depending on convexity, strong convexity, and the specific algorithmic variant:

• Standard projected gradient: $O(1/k)$ in convex, $\ell$0 in strongly convex scenarios.
• High-order acceleration: $\ell$1, α≥2 (for properly selected polynomial momentum).
• Proximal and primal-dual variants: rates match or improve upon classical schemes and are often provably stable for standard MPC cost and constraint structures (Wang et al., 2021, Yu et al., 2020, Moriyasu et al., 2024).

Closed-loop stability under inexact gradient optimization has been established via Lyapunov-based arguments, requiring that the projected gradient residual is proportional to the immediate stage cost, guaranteeing that suboptimality is sufficiently small to ensure contractive trajectories (Hall et al., 2022). For sampled-data primal-dual MPC, global asymptotic stability follows if the combined plant-controller system satisfies a certain dissipativity-based matrix inequality for the chosen time discretization (Moriyasu et al., 2024).

Implementation aspects include warm-starting to leverage temporal coherence, step-size and momentum tuning for convergence and numerical stability, exploitation of parallelism (e.g., block-diagonal projections and sparse tensor operations), and the need for careful constraint handling (e.g., Dykstra's algorithm, CBF-QP safety filters, or explicit projections).

6. Applications, Empirical Results, and Limitations

Gradient-based MPC methods have been validated in domains including classical linear and nonlinear control, embedded systems, robotics (1kHz tracking for manipulators), large-scale water-valley power tracking, non-prehensile manipulation, anesthesia drug delivery, and safe autonomous navigation. Highlights include:

• Closed-loop performance matching or exceeding second-order methods with lower per-iteration complexity in high-frequency control tasks (Zhang et al., 2024).
• Neural approximations of eMPC with gradient-matching loss drastically reducing required dataset size and computational overhead (Winqvist et al., 2021).
• In model-based RL, gradient-based latent-MPC approaches (Dream-MPC) outperforming or matching sampling-based planners on challenging continuous control benchmarks with an order of magnitude lower model query complexity (Spieler et al., 6 May 2026).
• Large-scale parallelized solvers (e.g., MPCGPU) scaling to sub-millisecond solve times over hundreds of-step horizons (Adabag et al., 2023), and cost-efficient constraint projections outperforming ADMM in rate+amplitude constrained MPC (Kempf et al., 2020).
• Closed-form and semi-analytical guarantees for constraint satisfaction, stability, and convergence, though limitations include the requirement for accurate first order dynamics/gradients, proper problem conditioning, and, in learning-based and highly nonconvex regimes, the risk of local minima and the need for auxiliary population/ensemble-based methods for robustness (Singh et al., 18 Jul 2025, Bharadhwaj et al., 2020).

7. Outlook: Hybrid and Adaptive Gradient-Based MPC Paradigms

Recent research in GB-MPC is moving towards flexible and hybrid approaches incorporating policy priors, uncertainty penalties, and action amortization (warm start) to overcome local minima and model errors in high-dimensional/enriched settings (Spieler et al., 6 May 2026). Hybridizations of sampling and gradient steps (CEM+Grad) yield performance increases without excessive computational cost (Bharadhwaj et al., 2020). Adaptive, automated state/vector reduction, and parallelism-aware implementations enable real-time applicability in domains previously restricted by “curse of dimensionality.” Integration with safety-critical filtering (CBF, SSM), learning surrogates, and hardware-aware accelerators continue to broaden applicability and performance envelopes of GB-MPC methods.