---
title: Differentiable Quadratic Program (dQP)
url: https://www.emergentmind.com/topics/differentiable-quadratic-program-dqp
type: topic
---

# Differentiable Quadratic Program (dQP)

A differentiable quadratic program (dQP) is a mathematical programming primitive that allows the solution to a quadratic program to function as a differentiable layer within end-to-end optimization pipelines, particularly neural networks. dQP specifically refers to the integration of quadratic programming solving and differentiation, such that both forward (solution computation) and backward (sensitivity/gradient computation) passes are efficiently handled and scalable to medium and large-scale instances, typically leveraging first-order or splitting methods rather than classical interior-point schemes. This paradigm enables direct embedding of constrained optimization as trainable modules—a central component for applications in control, financial engineering, and learning-based prediction-optimization.

## 1. Quadratic Program Layer: Canonical Formulation

dQP layers operate on the canonical parametric convex quadratic program:
\[
\min_{x\in\R^n} \frac{1}{2}x^\top P x + q^\top x,\quad
\text{subject to}\quad A x = b,\quad G x \le h
\]
where $x$ is the decision variable, $P\in\R^{n\times n}$ (positive semidefinite), $q\in\R^n$, $A\in\R^{m\times n}$, $b\in\R^m$, $G\in\R^{p\times n}$, $h\in\R^p$. For convexity and well-posedness, $P\succeq 0$ and the feasible set is assumed nonempty [2112.07464]. Many extensions absorb equalities and inequalities into bounded forms ($\ell\le Ax \le u$), introduce auxiliary slack variables, or specialize to conic quadratic and quadratic cone programs (QCPs) [2508.17522].

In neural network architectures, the problem data $\theta$ (e.g., $P$, $q$, $A$, $b$, $G$, $h$) can be generated by earlier layers, and the mapping $x^\star(\theta)$ serves as a nontrivial, constrained activation or projection function.

## 2. Forward Pass: First-Order Splitting via ADMM

dQP layers predominantly eschew dense KKT or interior-point solves in favor of first-order operator-splitting techniques, notably the Alternating Direction Method of Multipliers (ADMM) [2112.07464][2308.08232]. A prototypical ADMM-based dQP layer reformulates the QP as a variable-split constrained minimization:
\[
\min_{x,z} f(x) + g(z)\quad\text{subject to}\quad x-z=0,
\]
with $f(x) = \frac{1}{2}x^\top P x + q^\top x + \mathbb{I}_{Ax = b}(x)$ and $g(z) = \mathbb{I}_{Gz \leq h}(z)$. The scaled-dual ADMM scheme performs alternating updates:
- $x^{k+1}$: equality-constrained QP via KKT solve
- $z^{k+1}$: projection onto the feasible polyhedron (often implemented as per-coordinate clamping)
- $u^{k+1}$: dual variable update

This splitting confines the dominant computational cost to sparse matrix factorizations of size $(n+m)\times(n+m)$, well below the cubic scaling of the full KKT system $(n+m+p)\times(n+m+p)$. Methods such as SCQPTH further exploit data scaling, adaptive penalty selection, and reuse of factorizations to support $n=100$–$1000$ and $m$ in the thousands, with routine use of batched GPU accelerators [2308.08232]. Empirically, ADMM-based dQP layers achieve 1–10$\times$ improvements in solve time over interior-point-based OptNet or conic-layer CVXPY implementations at comparable accuracy—even on dense, large-scale QPs [2112.07464][2308.08232].

## 3. Differentiation: Implicit Backward Sensitivity

A core attribute of dQP is the ability to efficiently compute the derivatives $\tfrac{\partial x^\star}{\partial \theta}$ for backpropagation. ADMM-based approaches structure their fixed-point iteration as $v^{*} = F(v^{*}; \theta)$ (where $v$ collects primal and scaled dual variables). Differentiation proceeds via the implicit function theorem:
\[
\frac{\partial v^*}{\partial \theta} = \left[I - \frac{\partial F}{\partial v}\bigg|_{v^*}\right]^{-1} \frac{\partial F}{\partial \theta}\bigg|_{v^*}
\]
[2112.07464][2308.08232]. This mechanism requires solving a linear system of size $(n+m)$ in the backward pass, independent of iteration count, contrasting with unrolled differentiation, which stores all intermediate iterates ($O(K n)$ memory), and KKT-based implicit differentiation, which necessitates a much larger $(n+m+p)\times(n+m+p)$ system [2112.07464].

Specialized structured-problems (e.g., graph cut QPs in object-centric representation learning) further exploit sparsity, enabling custom backward routines based on Schur complements and sparse Cholesky factors for image-scale problems ($m\sim 50\,000$), achieving forward+backward solves in milliseconds [2210.02159].

## 4. Alternative Splitting, Unrolling, and Learning-Based dQP Models

Recent advances demonstrate deep unrolling of operator algorithms, where each iterative step of a splitting scheme (e.g., Douglas–Rachford, PDHG) is interpreted as a neural network layer [2412.01051][2508.11869]. For example, unrolled Douglas–Rachford replaces the costly prox-linear system with a single gradient step, maintaining convergence and interpretability, and achieves up to 40%–55% reduction in solve times versus the classical algorithm [2508.11869].

Similarly, PDQP-net unrolls a primal-dual hybrid gradient method into a differentiable architecture, trained directly on KKT-residual-based losses rather than ground-truth labels—eliminating the need for expensive supervised solver-based training [2412.01051]. Theoretical results show network parameters can be chosen so the unrolled network precisely recovers each PDQP iteration, with polynomial size and guaranteed linear convergence under convexity. Empirically, warm-starting PDQP with PDQP-net predictions produces 30%–45% acceleration in solves, with negligible overhead.

## 5. Active-Set Sensitivity and Black-Box Differentiability

The dQP framework [2410.06324] generalizes sensitivity analysis: rather than requiring solver internals (duals, factorization, etc.), knowledge of the active set at QP optimum allows explicit differentiation. After identification of the active constraint set, the solution and all relevant sensitivities can be computed via a reduced KKT system (size $n+p+|J|$), yielding derivatives:
\[
\left(\frac{\partial z^*}{\partial \theta}\right) = -K_J^{-1} \left( \frac{\partial K_J}{\partial \theta}\begin{pmatrix} z^* \\ \lambda^* \\ \mu_J^* \end{pmatrix} - \frac{\partial v_J}{\partial \theta} \right)
\]
This architecture enables plug-and-play differentiation for any QP solver (Gurobi, OSQP, MOSEK, PIQP, etc.), rendering the system entirely agnostic to solver internals, with $2$–$3$ order-of-magnitude speedups versus unrolled or full-KKT implicit approaches on high-dimensional problems [2410.06324].

## 6. Application Domains and Empirical Performance

dQP layers have demonstrated significant impact in end-to-end learning for data-driven decision making, especially in portfolio optimization. For mean-variance portfolio construction ($n\approx 250$–$1000$ assets), ADMM-based dQP layers achieve Sharpe ratios (out-of-sample 2015–2020) of $\approx 1.34$, matching OptNet ($\approx 1.28$), while achieving $5\times$ faster training at $n\approx 255$ [2112.07464][2411.19285]. BPQP, a backward-pass QP reformulation for gradient calculation, achieves $13.5\times$ faster backward time than CVXPY/SCS, $30\times$ faster than OptNet, while retaining gradient norm-consistency for inexact forward solves [2411.19285].

For object-centric representation learning via differentiable graph cuts, problem instances with up to $52\,000$ primal variables and $23\,000$ constraints are integrated efficiently, with differentiable gradients and batched GPU execution [2210.02159].

## 7. Extensions, Limitations, and Best Practices

dQP layers support scaling to larger problems via exploitation of problem sparsity ($P$, $A$), GPU batch factorization, and custom first-order solvers. The splitting paradigm naturally extends to second-order cone and semidefinite constraints by introducing additional proximal mappings [2112.07464][2508.17522]. Adaptive penalty selection ($\rho$), residual balancing, and Anderson/Nesterov acceleration are best-practice techniques for convergence and robustness [2112.07464][2308.08232]. Hybrid methods—where ADMM outputs are used to warm-start interior-point solvers for high-accuracy requirements ("solution polishing")—are also prevalent.

BPQP is compatible with any advanced QP solver for backward passes and provides KKT-norm-preserving gradients matching the forward solve residual [2411.19285]. Active-set identification for reduced KKT sensitivity is guaranteed under strict complementarity, with regularization or soft-active-set approaches recommended otherwise [2410.06324][2411.19285]. Limitations arise in nonconvex settings; local minima can still be differentiated via local KKT systems, but global optimality is not assured.

---

In summary, differentiable quadratic programming constitutes an enabling technology for integrating large-scale, constrained optimization directly into differentiable computational graphs. Contemporary dQP layers prioritize first-order, splitting-based solvers and implicit fixed-point differentiation, yielding significant computational savings and scalable, robust integration across decision-making, financial, graphical, and control domains [2112.07464][2308.08232][2210.02159][2410.06324][2411.19285][2508.17522][2508.11869][2412.01051].

Source: https://www.emergentmind.com/topics/differentiable-quadratic-program-dqp