Operator-Splitting QP Solvers

• Operator-splitting QP solvers are algorithms that decompose structured convex quadratic programs into simpler sub-tasks using proximal steps and fixed-point iterations.
• They employ methods like ADMM and Krylov subspace acceleration to overcome slow convergence and ill-conditioning in active-set transitions.
• These solvers are applied in MPC, statistical learning, and optimization, with research focusing on enhancing iteration efficiency and large-scale scalability.

Operator-splitting quadratic programming (QP) solvers are a central class of algorithms for structured convex quadratic programs, leveraging proximal and monotone operator theory to decompose difficult problems into efficiently solvable sub-tasks. These solvers encompass ADMM, Peaceman–Rachford/Douglas–Rachford splitting, primal-dual hybrid gradient, and related first-order approaches. Recent research introduces Krylov subspace and Newton-type accelerations to overcome slow convergence near active-set transitions or ill-conditioned regimes, and provides comprehensive analysis of their behavior, convergence, and large-scale scalability.

1. Problem Formulation and Operator-Splitting Principle

The canonical “operator-splitting QP” is formulated as: $$\min_{x\in\mathbb{R}^n,\;s\in\mathbb{R}^m} \frac{1}{2} x^\top P x + c^\top x \;\text{subject to}\; A x + s = b,\; s \in \mathbb{R}_+^{m_1} \times \{0\}^{m_2}$$ with $x\in\mathbb{R}^n$ (primal), $s\in\mathbb{R}^m$ (slacks), $P$ positive semidefinite, $A\in\mathbb{R}^{m\times n}$, $c\in\mathbb{R}^n$, $m_1$ inequality and $m_2$ equality constraints ($m_1 + m_2 = m$) (Pereira et al., 9 Nov 2025).

Operator-splitting approaches rephrase this constrained convex QP as a fixed-point problem for a (typically piecewise-affine) non-expansive operator $T$, e.g.,

$x\in\mathbb{R}^n$0

where $x\in\mathbb{R}^n$1 are multipliers or dual variables. On each polyhedral active set $x\in\mathbb{R}^n$2, $x\in\mathbb{R}^n$3 is affine: $x\in\mathbb{R}^n$4. Most splitting methods reduce each iteration to evaluating simple proximal steps (projections onto cones or affine sets), and solving small, structured linear systems.

2. Core Algorithms: ADMM and Fixed-Point Formulation

A baseline splitting method is the (preconditioned) ADMM, applied to the QP's saddle-point form. The key ADMM updates are

$x\in\mathbb{R}^n$5

where $x\in\mathbb{R}^n$6 is projection onto the dual cone, $x\in\mathbb{R}^n$7, and $x\in\mathbb{R}^n$8. In fixed-point form, this is $x\in\mathbb{R}^n$9, where $s\in\mathbb{R}^m$0 is $s\in\mathbb{R}^m$1-averaged in the $s\in\mathbb{R}^m$2-norm with

$s\in\mathbb{R}^m$3

General convergence rate is $s\in\mathbb{R}^m$4, but practical iteration count may be much larger in ill-conditioned or active-set-changing regimes (Pereira et al., 9 Nov 2025).

3. Krylov Subspace Acceleration for Splitting Methods

The key recent contribution for operator-splitting QP solvers is the Krylov subspace acceleration scheme (Pereira et al., 9 Nov 2025), which addresses slow convergence in first-order methods. When the active set $s\in\mathbb{R}^m$5 remains constant, the operator $s\in\mathbb{R}^m$6 is affine and the subproblem reduces to solving $s\in\mathbb{R}^m$7. The Krylov method constructs

$s\in\mathbb{R}^m$8

where $s\in\mathbb{R}^m$9. Using the Arnoldi process, an orthonormal basis $P$0 and upper Hessenberg matrix $P$1 are assembled.

The optimal subspace step solves

$P$2

and applies $P$3. The Arnoldi orthogonalization guarantees the subspace system remains well-conditioned, even when the fixed-point residuals become nearly collinear—rectifying the ill-conditioning endemic to Anderson acceleration near active-set changes.

The algorithm alternates between vanilla fixed-point iterations and accelerated steps at prescribed Krylov dimensions, with safeguard parameters $P$4 and $P$5 to ensure convergence is not compromised by aggressive subspace corrections. Memory parameter $P$6--$P$7 suffices in practice.

4. Comparison to Anderson Acceleration and Robustness

Type-II Anderson acceleration employs an unorthogonalized least-squares problem over recent residuals, which can lead to severe ill-conditioning as residuals align (e.g., near active-set transitions). In contrast, the GMRES-style Arnoldi orthogonalization in the Krylov approach maintains conditioning and yields consistent improvement in both iteration count and overall computation time for strongly convex QPs (Pereira et al., 9 Nov 2025).

Walker & Ni (2011) proved AA and GMRES coincide for linear $P$8; the Krylov-accelerated scheme generalizes this robustness to piecewise affine (PWA) fixed-point operators arising in QPs.

5. Empirical Performance and Implementation Guidance

Extensive numerical benchmarks confirm Krylov acceleration achieves 2–5× fewer iterations than Anderson acceleration across classes of QPs derived from model predictive control and statistical learning. For high accuracy ($P$9), the method solved 70% of MPC problems faster than Anderson acceleration (40%), and consistently outperformed on iteration count. For large, stable active-set problems, wall-time advantages are even more pronounced (Pereira et al., 9 Nov 2025).

Practical recommendations include:

• Use $A\in\mathbb{R}^{m\times n}$0 in the range 10–20 for Krylov memory; larger is beneficial only for very slow convergence.
• Restart the acceleration procedure on significant active-set changes.
• Safeguard with $A\in\mathbb{R}^{m\times n}$1, $A\in\mathbb{R}^{m\times n}$2–100 for the step-size and residual norm tests in the $A\in\mathbb{R}^{m\times n}$3-norm.
• Efficient implementation can combine real and imaginary parts in one complex vector to reduce matrix–vector costs empirically to less than $A\in\mathbb{R}^{m\times n}$4 real cost.

6. Applicability and Integration

Krylov-accelerated operator-splitting is a drop-in upgrade for any QP solver whose underlying $A\in\mathbb{R}^{m\times n}$5 is piecewise affine, including ADMM, Douglas–Rachford, and the proximal-point method (Pereira et al., 9 Nov 2025). The acceleration framework is independent of the baseline splitting scheme, provided the fixed-point operator is $A\in\mathbb{R}^{m\times n}$6-averaged. The safeguard mechanism retains the convergence guarantees of the original method. Integration requires only the ability to extract the current active set and perform Arnoldi updates after each standard iteration.

Krylov subspace acceleration is particularly recommended for QPs arising in MPC subproblems, statistical learning, and any regime requiring high-accuracy solutions or robust iteration counts against ill-conditioning.

7. Limitations and Future Directions

Krylov subspace acceleration is fundamentally reliant on stable active sets; rapid or large-scale active-set changes may necessitate frequent restarts, reducing acceleration efficacy. Large memory parameters $A\in\mathbb{R}^{m\times n}$7 can increase orthogonalization and small least-squares overhead, offsetting per-iteration speedups in pathological cases.

Further investigation into adaptive memory management, higher-order safeguard strategies, and integration with continuation or warm-starting techniques could extend the method’s scalability, particularly in real-time or embedded contexts. A plausible implication is that hybrid methods combining Krylov and Newton-type steps could enable superlinear local convergence where the PWA operator $A\in\mathbb{R}^{m\times n}$8 is differentiable and the Jacobian is nonsingular.

---

References

• Krylov Subspace Acceleration for First-Order Splitting Methods in Convex Quadratic Programming (Pereira et al., 9 Nov 2025)