Unrolled Active Set Method

• The unrolled active set method is an optimization approach that uses controlled perturbations to predict and exploit active constraints in quadratic programming.
• It decouples global problem solving from local subproblem refinement using perturbed KKT conditions, enhancing prediction accuracy and numerical stability.
• The approach yields significant iteration reductions and improved performance in degenerate cases, making it effective for large-scale QPs.

The unrolled active set method is a class of optimization algorithms designed to reveal, predict, and exploit the combinatorial structure (such as the support of active constraints) in constrained optimization problems, most notably quadratic programming (QP). By unrolling or decoupling the identification and exploitation of the active set, these methods integrate predictive strategies—often by judicious perturbations or structured interior point iterates—to transition rapidly from global strategies to efficient, locally reduced subproblems corresponding to the active set. The approach is grounded in explicitly controlling the relationship between perturbed and original problems so the iterative sequence of active set predictions can be rigorously aligned with the true optimal partition, even for degenerate or non-strictly complementary QPs.

1. Controlled Perturbations for Active-Set Prediction

Central to the unrolled active set method is the introduction of controlled perturbations to the original inequality constraints. For a convex QP,

$$\min_{x} \;\; \frac{1}{2}x^T H x + c^T x \quad \text{subject to} \quad Ax = b, \;\; x \geq 0,$$

the feasible set can be enlarged by considering

$x \geq -\lambda,$

with $\lambda \geq 0$ a vector of small perturbations, and analogously for the dual,

$s \geq -\lambda.$

The perturbed primal and dual problems take the form

$$\begin{aligned} &\min_{x} \;\; \frac{1}{2}(x+\lambda)^T H (x+\lambda) + (c+(I-H)\lambda)^T(x+\lambda)\ &\text{subject to} \quad Ax = b, \; x \geq -\lambda\ & \max_{(x,y,s)} (b+A\lambda)^T y - \frac{1}{2}(x+\lambda)^T H (x+\lambda)\ &\text{subject to}\quad A^T y + s - Hx = c, \; s \geq -\lambda. \end{aligned}$$

Selection of $\lambda$ is critical—it must be small enough for tight prediction but large enough so interior point iterates can approach the boundary associated with the true active set.

Perturbed KKT (central path) conditions are expressed as

$$Ax = b, \; A^T y + s - Hx = c, \; (X+\Lambda)(S+\Lambda)e = \mu e, \; (x+\lambda, s+\lambda)>0$$

where $\Lambda = \text{diag}(\lambda)$, $e$ is the vector of ones, and $\mu > 0$ is the barrier parameter.

2. Construction of Primal–Dual Pairs Matching the True Active Set

The method provides existence theorems guaranteeing that for an appropriate $\lambda$, there exists a primal–dual pair $(x, y, s)$ nearly (or exactly) satisfying the perturbed KKT conditions, so that the shifted variables $(x+\lambda, s+\lambda)$ correctly reveal the combinatorial structure of the original QP: $$\mathcal{A}_\lambda(x) = \mathcal{A}(x^*), \quad \mathcal{I}_\lambda(x) = \mathcal{I}(x^*), \quad \mathcal{A}_\lambda(s)=\mathcal{A}(s^*), \quad \mathcal{S}_\lambda(s)=\mathcal{S}(s^*)$$ with feasibility residuals proportional to $\|\lambda\|$. The sets are defined as:

• Primal active: $\mathcal{A}_\lambda(x) = \{i : x_i = -\lambda_i\}$
• Primal inactive: $\mathcal{I}_\lambda(x) = \{i : x_i > -\lambda_i\}$
• Dual analogous definitions.

This precise structural alignment enables the algorithm to use simple thresholding on iterates to predict the solution's combinatorial structure well before convergence, facilitating early transition to small-scale subproblems.

3. Optimal Tripartition and Degeneracy Handling

Since QPs generally lack strict complementarity, the optimal index partition is a tripartition: $$\begin{aligned} &\mathcal{S}(s^*) \subseteq \mathcal{A}(x^*)\ &\mathcal{I}(x^*) \subseteq \mathcal{A}(s^*)\ &\mathcal{T}(x^*, s^*) = \{1, \dots, n\} \setminus (\mathcal{S}(s^*) \cup \mathcal{I}(x^*)) \end{aligned}$$ where $\mathcal{T}$ indexes degenerate components ($x^*_i = s^*_i = 0$). The controlled perturbation strategy guarantees that the predicted tripartition from the perturbed iterates matches that of the original QP via

$$\left(\mathcal{S}_\lambda(s), \mathcal{I}_\lambda(x), \mathcal{T}_\lambda(x,s)\right) = (\mathcal{S}(s^*), \mathcal{I}(x^*), \mathcal{T}(x^*, s^*))$$

thus robustly separating degenerate indices.

4. Error Bounds and KKT Residuals

Theoretical analysis provides error bounds on approximate optimality in terms of residuals: $$\| x - x^* \| \leq \tau_p ( r(x,s) + w(x,s) ), \quad \| s - s^* \| \leq \tau_d ( r(x,s) + w(x,s) )$$ where $r(x,s)$ and $w(x,s)$ are specific residual measures for (in)feasibility and complementarity, with constants $\tau_p,\tau_d$ depending on problem data. This links the controlled perturbation directly to predictive fidelity and ultimately to convergence guarantees.

5. Numerical Results and Practical Implications

Preliminary experiments validated the method:

• The correction ratio (fraction of true active set correctly predicted) is increased, sometimes more than doubled, by introducing perturbations.
• On random QP instances (including degenerate cases), the perturbed method's predicted active sets were closer to those found by active-set solvers.
• Using these predictions to reduce problem size led to savings of approximately 63% in active-set method iterations for random problems, and ~36% for degenerate cases.
• Feasibility residuals decayed faster in the perturbed approach, indicating not only prediction accuracy but also improved numerical stability of the Newton system arising in interior point methods.

6. Algorithmic Workflow and Integration with Interior Point Methods

The unrolled active set method typically comprises the following steps:

1. Solve the perturbed QP using an interior point method, tracking KKT central path iterates for $(x, y, s)$.
2. At each iterate, evaluate $x_i$ (and $s_i$) against $–\lambda_i$ to predict $\mathcal{A}_\lambda$, $\mathcal{I}_\lambda$, etc.
3. Once prediction stabilizes, focus on the reduced problem formed by eliminating inactive constraints/variables.
4. Apply a combinatorial active-set algorithm on the reduced space, exploiting the predicted support for fast local convergence.

This workflow is "unrolled" in the sense that the prediction and exploitation phases are visibly segregated and tightly coupled via controlled perturbation and primal–dual pair construction, yielding an explicit mapping from interior-point iterates to combinatorial structure.

7. Significance, Limitations, and Connections

The unrolled active set method advances active-set prediction far beyond heuristics by providing a systematic, theory-backed mechanism for revealing the active partition early, with controlled error in the KKT system. Its limitations are tied to the selection of perturbation vector $\lambda$ and to the properties of the underlying QP (e.g., non-strict complementarity, degeneracy). When properly implemented, it enables large speedups for active-set methods and sets a foundation for robust and scalable QP solvers.

This paradigm is closely related to interior-point, combinatorial, and hybrid methods and is especially relevant in applications demanding rapid solution of large QPs (e.g., control, machine learning, sparse recovery) where early structure prediction translates directly into computational savings.

---

Key mathematical constructs and formulas:

• Original QP:

$$\min_{x} \frac{1}{2} x^T H x + c^T x \;\; \text{s.t.} \;\; Ax = b,\; x\ge0$$

• Perturbed QP:

$$\min_{x} \frac{1}{2}(x+\lambda)^T H (x+\lambda)+(c+(I-H)\lambda)^T(x+\lambda) \;\; \text{s.t.} \;\; Ax=b,\; x\ge-\lambda$$

• KKT central path:

$$Ax=b,\quad A^Ty + s - Hx = c,\quad (X+\Lambda)(S+\Lambda)e = \mu e$$

• Error bound:

$\|x-x^*\| \leq \tau_p ( r(x,s) + w(x,s) )$

• Active set prediction:

$$\mathcal{A}_\lambda(x) = \{ i: x_i = -\lambda_i \}, \quad \mathcal{I}_\lambda(x) = \{ i: x_i > -\lambda_i \}$$

The method combines structural prediction, combinatorial problem reduction, and interior-point theory for rigorous and efficient solution of convex QPs with challenging constraint activity.