---
title: Gradient-Based Model Predictive Control
url: https://www.emergentmind.com/topics/gradient-based-model-predictive-control-mpc
type: topic
---

# Gradient-Based Model Predictive Control

Searching arXiv for recent and foundational papers on gradient-based MPC to ground the article in the literature.
Gradient-based model predictive control denotes a family of receding-horizon methods in which derivatives of a finite-horizon objective, a constraint residual, or a posterior density are used to update the decision variables of the MPC problem. Across the literature, those decision variables include open-loop action sequences, lifted state-input trajectories, dual multipliers, and particle representations of control distributions. The common operational pattern is to optimize over a finite horizon, execute only the first input, discard the rest, and replan at the next step [2004.08763] [2605.14937]. In that sense, gradient-based MPC is not a single algorithmic template but a broad computational regime spanning shooting-based planners, primal-dual first-order solvers, direct-transcription nonlinear MPC, and learning-based controllers that exploit differentiable predictive models [2203.07747] [2303.17889].

## 1. Canonical problem statements

In the most classical learned-model formulation, planning is posed as optimization of an open-loop action sequence under differentiable dynamics and reward models,
$$
\max_{a_{0:H}} \sum_{h=1}^H r_\psi(s_h)
\quad \text{s.t.} \quad
s_h = f_\phi(s_{h-1}, a_{h-1}),
$$
followed by receding-horizon execution of only the first action [2004.08763]. In goal-conditioned visual control, the objective may instead be terminal and task-agnostic, such as minimizing a latent distance between predicted terminal object slots and goal slots,
$$
\mathcal{J}_{\text{MPC}} = \|\hat S_{t+H} - S_{\text{Goal}}\|_2^2,
$$
with the action sequence updated directly by gradient descent through the latent rollout [2605.14937]. In safe optimal control, hard state constraints can be relaxed into stage penalties, for example by adding
$$
\lambda \max\{0,\,-l(x_k)+\delta\}
$$
to the horizon cost, and then re-imposing hard safety with a subsequent control-barrier-function filter [2507.13872].

These variants differ in objective semantics, but they share the same structural feature: gradients are taken with respect to finite-horizon decision variables rather than a stationary policy alone. In the surveyed literature, gradient information is exploited in at least four recurrent ways.

| Family | Core decision object | Representative papers |
|---|---|---|
| Direct shooting through differentiable models | Open-loop action sequence | [2004.08763], [2605.14937], [2203.07747] |
| First-order MPC solvers | Primal, dual, or condensed QP variables | [2003.05667], [2009.06980], [2109.04405] |
| Learning-based uncertainty-aware MPC | Mean/variance trajectories or posterior particles | [1612.01211], [1812.10579], [2011.07641], [2404.03689] |
| Safety-augmented MPC | Soft-constrained nominal plan plus safety filter | [2507.13872], [2303.17889] |

A useful interpretive distinction is between gradient-based MPC as an online optimizer and gradient-informed methods that only support MPC indirectly. The former use gradients in the receding-horizon loop itself; the latter may use gradient information offline, for instance to learn a surrogate controller that approximates an MPC law [2102.02173].

## 2. Direct trajectory optimization through differentiable models

The most literal form of gradient-based MPC differentiates the horizon objective through the model rollout and updates the control sequence directly. In the learned-dynamics setting studied by Bharadhwaj, Xie, and Shkurti, pure gradient planning is attractive because it receives a \(D\)-dimensional feedback signal—the gradient of return with respect to actions—whereas CEM receives only a scalar return per sampled sequence. The same paper also emphasizes the failure modes of pure gradient shooting: nonconvexity, vanishing or exploding gradients over long horizons, sensitivity to bad initialization, and difficulty under non-smooth contact [2004.08763].

Two strands of subsequent work make that template more practically effective. One is hybridization with population search. The Grad+CEM planner samples candidate sequences from a Gaussian distribution, performs one gradient-refinement step per sample, and refits the sampling distribution to the refined elites. Empirically, it converges faster than CEM in high-dimensional settings and avoids some local minima that trap pure gradient methods [2004.08763]. The other is to redesign the latent planning space itself. Slot-MPC performs gradient-based MPC in a differentiable object-centric latent state built from temporally aligned slots rather than pixels or large patch embeddings. The planner encodes the current image and a goal image into slots, rolls the learned cOCVP dynamics forward over horizon \(H\), aligns predicted and goal slots with Hungarian matching, and updates actions by
$$
a \leftarrow a - \eta \nabla \mathcal{J}_{\text{MPC}}.
$$
In the reported offline robotic manipulation setting with limited state-action coverage, full Slot-MPC attains success rates \(0.64\), \(0.52\), \(0.42\), and \(0.22\) on Button Press, Lever Pull, Stack, and Square, whereas replacing gradient-based MPC with MPPI yields \(0.04\), \(0.04\), \(0.00\), and \(0.00\). The same study reports planning times of \(0.42 \pm 0.01\) s per step on Meta-World and \(0.48 \pm 0.02\) s on robosuite, compared with \(4.22 \pm 0.06\) s and \(5.19 \pm 0.03\) s for Slot-MPC with MPPI, and roughly \(144\)–\(145\) s for DINO-WM [2605.14937].

A third line addresses online nonlinear MPC with learned residual dynamics but conventional SQP/RTI structure. Real-time Neural-MPC does not expose a large neural network directly to the MPC solver. Instead, it computes batched local linear or quadratic approximations of the learned residual model outside the solver and passes only local coefficients to the RTI-based QP preparation stage. This preserves the structure expected by acados/CasADi-style embedded solvers while allowing neural dynamics models of over \(4000\) times larger parametric capacity in a \(50\) Hz real-time window on embedded hardware, with up to \(82\%\) reduction in positional tracking error relative to state-of-the-art MPC approaches without neural-network dynamics [2203.07747].

Across these examples, the role of gradients is not merely to replace sampling. They reconfigure the optimizer around local model information, and their utility depends strongly on initialization, smoothness, and the geometry of the latent or state representation [2004.08763] [2605.14937].

## 3. First-order solver architectures for structured MPC programs

A second major meaning of gradient-based MPC concerns solver design for finite-horizon convex programs. Here the MPC problem is not optimized by generic NLP software but by tailored first-order methods that exploit condensed, dual, or saddle-point structure.

For linear MPC with input amplitude and slew-rate constraints, the fast gradient method can be applied directly to the condensed QP
$$
\min \frac12 u^\top J u + q^\top u
\quad \text{s.t.} \quad u \in \mathcal U,
$$
provided the nontrivial projection onto the joint amplitude-plus-rate constraint set is handled by Dykstra’s algorithm. The proposed FGM+Dykstra method avoids the augmented-variable formulation used by ADMM, greatly reduces computation time, and approximately halves the memory usage for large horizons [2003.05667].

For finite-horizon convex MPC with linear equality constraints and general convex state/input constraints, the proportional-integral projected gradient method updates
$$
v^k = w^k + \beta^k(Gz^k-g), \quad
z^{k+1} = \pi_{\mathbb Z}[z^k-\alpha^k(Hz^k+h+G^\top v^k)], \quad
w^{k+1} = w^k + \beta^k(Gz^{k+1}-g).
$$
Its distinctive feature is that each iteration requires only one projection onto the joint state-input constraint set \(\mathbb Z\). The paper proves \(O(1/k)\) convergence for both distance to optimum and constraint violation in the convex case, and \(O(1/k^2)\) for distance to optimum together with \(O(1/k^3)\) for constraint violation in the strongly convex case, along weighted averaged iterates [2009.06980].

At the level of dual QP solvers for condensed linear MPC, accelerated proximal-gradient methods are also prominent. One proposed variant modifies the usual FISTA momentum sequence by selecting parameters from the higher-order polynomial recurrence
$$
t_{p+1}^{\alpha}-t_{p+1}^{\alpha-1}-t_p^\alpha = 0,
$$
with the stated goal of improving the convergence-rate bound from \(O(1/p^2)\) to \(O(1/p^\alpha)\). On the tested random small MPC instances, it reduces iteration counts relative to FISTA and outperforms MOSEK and ECOS in the small-size regime. The same paper, however, explicitly notes that the claim of arbitrary-order acceleration is unusual and should be treated cautiously relative to standard first-order complexity theory [2109.04405].

For continuous-time path-constrained linear MPC, another route is to convert the problem into a conic program. Using differential flatness, polynomial parameterization, and Markov–Lukács/SOS conditions, continuous-time path constraints on the entire interval are transformed into an SDP with many small PSD blocks, which is then solved by a parallelizable PDHG scheme. The paper reports GPU iteration times from about \(0.318\) ms at \(N=2,d=3\) to \(0.776\) ms at \(N=40,d=3\), and argues that the method guarantees strict continuous-time constraint satisfaction that a standard discretized MPC controller may violate between sampling instants [2303.17889].

A related primal-dual viewpoint appears in sampled-data PDG-based instant MPC. There, the optimizer dynamics are themselves the controller, and the contribution is not better asymptotic optimization but preservation of closed-loop stability after discretization. The paper derives a sampled-data stability certificate of the form
$$
Q_{\rm all}^{\rm d} \prec 0
$$
for the discrete-time closed loop and reports \(0.048\) ms average computation time per step in an engine air-path example, with the abstract stating execution in only \(1\,\mu\)s on a standard laptop [2401.05100].

## 4. Learning-based uncertainty, latent geometry, and posterior methods

In learning-based MPC, gradients are often mediated by uncertainty propagation or by the representation chosen for the predictive model. Gaussian-process MPC is a canonical case. One GP-MPC line models unknown nonlinear discrete-time dynamics with a GP over state increments, propagates mean and variance over the horizon by moment matching, and then derives two tractable controllers. GPMPC1 relaxes the stochastic MPC problem to a deterministic nonlinear MPC based on a locally linearized GP model and solves it by SQP; GPMPC2 augments the state with uncertainty information and obtains a convex QP approximation solved efficiently by an active-set method. On the reported tracking problems, both controllers are effective, but GPMPC2 is much more efficient computationally, with about \(34.1\) s versus \(4.51\) s total for one benchmark and about \(24.72\) s versus \(5.38\) s for another [1612.01211].

A later development, linGP-SCP, replaces direct optimization over exact GP mean and variance with a local model obtained by linearizing the latent GP function itself. The resulting linGP has an affine predictive mean and a convex quadratic predictive variance, making the local MPC subproblem convex under stated assumptions. Its median solve time rises from \(0.0828\) s to \(0.2082\) s as model size grows from \(N=100\) to \(N=1500\), while Ipopt rises from \(0.0327\) s to \(4.2207\) s and Knitro from \(0.0364\) s to \(2.7341\) s; at \(N=1500\), Ipopt is about \(20.27\times\) slower and Knitro about \(13.13\times\) slower than linGP-SCP [1812.10579].

The GP-MPC tutorial literature systematizes the underlying approximation machinery. A nominal discrete-time model is corrected by a GP residual,
$$
x_{k+1}=f(x_k,u_k)+B_d(d(x_k,u_k)+\omega_k),
$$
and first-order Taylor approximations yield differentiable recursions for mean and covariance propagation. In the mean-only formulation, local linearization of both the nominal model and the GP mean reduces the MPC step to an explicit quadratic update
$$
\delta \mathbf u_k =
(\mathbf H'^{\!T}\mathbf Q\mathbf H' + \mathbf R)^{-1}
(\mathbf H'^{\!T}\mathbf Q\,\tilde{\mathbf x}_{k+1} - \mathbf R\bar{\mathbf u}_k),
$$
while uncertainty-aware variants retain smooth deterministic tightened constraints derived from propagated GP variances [2404.03689].

Another probabilistic interpretation of gradient-based MPC appears in Stein Variational MPC. There, planning is formulated as posterior inference over control parameters or open-loop control sequences, and Stein variational gradient descent moves particles according to
$$
\Theta^i \leftarrow \Theta^i + \epsilon\,\hat{\phi}^*(\Theta^i),
$$
where the update combines the gradient of the log posterior with a kernel-induced repulsive term. The paper shows that MPPI and CEM arise as single-particle special cases under particular likelihood choices, and reports \(96\%\) success on a 4x4 stochastic obstacle grid for SV-MPC with 32 particles, versus \(64\%\) for both MPPI and CEM under equal total sample budgets [2011.07641].

Taken together, these works indicate that in learning-based MPC the effectiveness of gradients is inseparable from how uncertainty, multimodality, and latent geometry are represented. Gradient access alone does not fix poor curvature or poor semantics in the planning space [1812.10579] [2605.14937].

## 5. Safety, constraints, and practical implementation

Safety and constraint handling expose a central tension in gradient-based MPC: gradients scale well, but hard state constraints, non-smooth contact, and feasibility restoration can make purely gradient-driven optimization brittle. One response is two-stage control. A recent safe-controller synthesis framework first computes a nominal control by gradient-based MPC with relaxed state-safety penalties, then solves a CBF-QP
$$
u^*(x)=\arg\min_{u\in\mathcal U}\|u-u_{\mathrm{mpc}}\|^2
\quad \text{s.t.} \quad
\mathcal L_f h(x)+\mathcal L_g h(x)u+\kappa(h(x))\ge 0
$$
to enforce hard safety with minimal deviation from the nominal action. On the reported unicycle problem, GMPC-CBF achieves cost \(1.403\), safety rate \(100\%\), and computation time \(0.14\) s, compared with MPPI-CBF cost \(2.071\), safety rate \(100\%\), and time \(0.20\) s. On the reported 6D planar quadrotor problem, GMPC-CBF achieves cost \(1.093\), safety rate \(100\%\), and computation time \(0.3\) s, compared with MPPI-CBF cost \(2.302\), safety rate \(100\%\), and time \(0.7\) s [2507.13872].

A more classical constraint-centric route is direct transcription. LGR-MPC formulates each receding-horizon optimal control problem as a nonlinear program using a Legendre–Gauss–Radau pseudospectral discretization. States and controls are collocated, defect constraints are imposed through the differentiation matrix, path constraints are enforced directly, and the resulting NLP is solved by SciPy. The package emphasizes warm-starting from the previous solution, scaling all state and control variables to \([-1,1]\), and simulating the dynamics over the applied interval before advancing the horizon. Its contribution is not a new gradient algorithm per se, but an accessible derivative-based direct-transcription MPC workflow [2310.15960].

Implementation practice across the literature converges on a small set of recurring devices. Warm starts are essential: policy-prior initialization in Slot-MPC, shifted-horizon priors in SV-MPC, local linearization around previous RTI iterates in Neural-MPC, and horizon-shift reuse in conic or direct-transcription methods all reduce sensitivity to initialization [2605.14937] [2011.07641] [2203.07747]. Variable scaling, box projection, and trust-region management are equally prominent. In the GP and linGP literature, trust regions and exact penalties prevent local convexification from wandering too far from the nominal trajectory; in projected-gradient and primal-dual methods, cheap projection is part of the algorithmic design itself [1812.10579] [2003.05667] [2009.06980].

## 6. Distinctions, limitations, and unresolved issues

Several recurring misconceptions are explicitly corrected in the literature. First, gradient-based MPC is not synonymous with any use of derivatives around MPC. Learning an approximate MPC law from controller Jacobians, as in neural imitation of explicit MPC with gradient data, is an offline controller-identification problem rather than online gradient-based MPC optimization [2102.02173]. Second, differentiability alone is not sufficient for good planning. Slot-MPC argues that planner performance depends not only on whether the model is differentiable, but on the geometry of the latent state space; the same study attributes the weakness of large patch-based latents partly to optimization burden and poor semantic alignment [2605.14937].

Third, pure gradient methods do not dominate sampling methods uniformly. The classic counterarguments remain local minima, exploding or vanishing gradients over long horizons, sensitivity to initialization, and non-smooth or discontinuous objectives such as contact-rich costs [2004.08763]. Slot-MPC softens this by showing that in an offline regime with limited state-action coverage, gradient-based MPC may actually outperform sampling-based MPC because it stays closer to the policy-prior distribution and avoids evaluating many far-out-of-distribution trajectories [2605.14937]. By contrast, Stein Variational MPC argues that particle-based gradient updates are preferable when multimodality itself is the core difficulty [2011.07641].

Fourth, solver-level convergence claims should be interpreted with care. PI-PG gives ergodic guarantees along averaged iterates rather than exact-feasibility guarantees for every truncated online iterate [2009.06980]. The accelerated proximal-gradient paper for dual MPC QPs explicitly presents an \(O(1/p^\alpha)\) rate claim but also notes that the result is unusual from a modern first-order-optimization perspective and should be viewed critically [2109.04405]. The sampled-data PDG literature likewise shows that continuous-time stability proofs do not transfer automatically to digital implementation; discretization can destabilize a PDG controller that continuous-time analysis would certify as stable [2401.05100].

Finally, safety mechanisms are themselves conditional. CBF-based safety filtering yields formal guarantees only when the barrier construction is valid and the resulting QP is feasible [2507.13872]. Goal-conditioned latent objectives can omit force-sensitive success criteria; Slot-MPC reports failure modes in which the robot reaches visually near the goal yet does not fully press a button or slips off a lever, suggesting that terminal-only latent distance can underrepresent contact-rich task completion [2605.14937].

The broad implication is that gradient-based MPC is best understood as a spectrum of techniques rather than a settled algorithmic doctrine. Its unifying idea is the exploitation of derivative information inside receding-horizon optimization, but its actual performance depends on representation, smoothness, uncertainty modeling, constraint treatment, warm-start quality, and the fidelity with which numerical implementation preserves the assumptions of the underlying analysis [2004.08763] [2404.03689].

Source: https://www.emergentmind.com/topics/gradient-based-model-predictive-control-mpc