A Sequential Cubic Programming Method with Second-Order Complexity Guarantees for Equality Constrained Optimization

Abstract: We develop a new method for equality constrained optimization problems based on a sequential cubic programming framework. Each iteration utilizes a step decomposition based on the Jacobian of the constraints into a normal and a tangential component, the latter of which is found by solving a subproblem involving cubic regularization. The method incorporates second-order correction steps as necessary to ensure global convergence to second-order stationary points as well as local quadratic convergence. In addition, we show that the algorithm is the first to obtain worst case complexity guarantees on the order of $\mathcal{O}(ε_g^{-3/2})$ for the gradient of the Lagrangian, $\mathcal{O}(ε_H^{-3})$ in terms of second-order stationarity, and $\mathcal{O}(ε_c^{-1})$ in terms of the constraint violation. These are the best known complexity guarantees of any method for this class of problems.

• The paper’s main contribution is a single-phase SCP algorithm that attains optimal worst-case complexity bounds for both first- and second-order conditions in equality-constrained nonconvex problems.
• Its innovative decomposition into normal and tangential steps, coupled with cubic regularization and adaptive second-order corrections, ensures robust convergence and efficient constraint satisfaction.
• The method improves constraint violation bounds compared to prior approaches, simplifies hyperparameter tuning, and achieves local quadratic convergence under standard regularity.

Sequential Cubic Programming for Equality Constrained Optimization: Second-Order Complexity Guarantees

Introduction and Problem Formulation

The paper "A Sequential Cubic Programming Method with Second-Order Complexity Guarantees for Equality Constrained Optimization" (2604.02747) addresses global, worst-case iteration complexity for equality-constrained, smooth, nonconvex optimization problems. The focus is on finding points satisfying approximate second-order KKT conditions, i.e., points where the Lagrangian gradient is small, constraints are approximately feasible, and the Lagrangian Hessian is nearly positive semidefinite in the feasible directions. The main result is a single-phase Sequential Cubic Programming (SCP) algorithm that, for the first time, matches in order the best known complexity thresholds in the unconstrained setting—$\mathcal{O}(\epsilon_g^{-3/2})$ for the gradient, $\mathcal{O}(\epsilon_H^{-3})$ for the Hessian, and a $\mathcal{O}(\epsilon_c^{-1})$ bound for constraints—while maintaining local quadratic convergence under standard regularity.

The equality-constrained optimization problem considered is:

$$\min_{x\in\mathbb{R}^n} f(x) \quad \text{s.t.} \quad c(x) = 0,$$

where $f$ and each $c_i$ are assumed to be twice Lipschitz differentiable and possibly nonconvex.

Algorithmic Contributions

The proposed SCP method generalizes cubic regularization to the equality constrained setting. The method works as follows:

• Decomposition of Steps: At each iteration, the search direction $d$ is decomposed into a normal step $v$ (for approximate satisfaction of the linearized constraints) and a tangential step $u$ (restricted to the nullspace of the constraint Jacobian, to improve optimality over feasible directions).
• Cubic Regularization Subproblem: The tangential step solves a cubic subproblem—a cubicly regularized Lagrangian minimized over the tangent space—using an inexact Krylov method or similar, with guarantees on model reduction and use of negative curvature.
• Sequential Cubic Trust Mechanism: The algorithm uses a sequential scheme where the cubic penalty acts as a trust-region-like control. After each step, an l1-merit function is used to check for sufficient reduction, and the regularization parameter is adapted.
• Second-Order Correction/Maratos Effect: To recover quadratic convergence and avoid Maratos effect, the method includes a second-order correction step along constraint curvature in nearly feasible states, adaptively invoked as in trust-region SQP.
• Approximate Lagrange Multipliers: Dual iterates are maintained as least-squares estimators, ensuring both theoretical and local convergence properties.
• Single-Phase Procedure: The algorithm does not decouple feasibility and optimality (unlike two-phase methods) and does not require penalty or augmentation parameter tuning.

Complexity Results and Theoretical Guarantees

First-Order Complexity: For problems with only first derivatives available, the method achieves the following worst-case bounds:

• To reach $(\epsilon_g, \epsilon_c)$-FOSP (epsilon-gradient and epsilon-feasibility): $\mathcal{O}(\epsilon_H^{-3})$0 [Theorem 3, (2604.02747)]. This matches the lower bound for steepest descent in the unconstrained setting and aligns with the best single-phase and two-phase methods for equality constraints [berahas2025sequential; curtis2024worst].

Second-Order Complexity: With second derivatives (Lipschitz Hessian), the method attains for $\mathcal{O}(\epsilon_H^{-3})$1-SOSP:

• Gradient: $\mathcal{O}(\epsilon_H^{-3})$2
• Hessian: $\mathcal{O}(\epsilon_H^{-3})$3
• Constraints: $\mathcal{O}(\epsilon_H^{-3})$4

Achieving all simultaneously and optimally is new, especially for single-phase methods. Notably, the constraint violation bound improves over previous results such as [goyens2024computing], reducing $\mathcal{O}(\epsilon_H^{-3})$5 to $\mathcal{O}(\epsilon_H^{-3})$6 by leveraging a tangential step driven by cubic regularization and adaptively controlled normal step.

Local Convergence: Under standard constraint qualifications (LICQ) and second order sufficient conditions, the algorithm exhibits local quadratic convergence to strict local minima. This ensures practical fast convergence in the regime near optimality, a feature rarely established in methods with global complexity analysis.

Robustness: The method is initialization-free and does not require penalty parameter tuning or strong regularization. All core steps (tangent/normal/dual updates) allow inexact computation within prescribed error tolerances, supporting efficient Krylov-type implementations for the cubic subproblem.

Technical Comparison and Relation to Literature

This work resolves a key open question: whether the unconstrained worst-case complexity bounds for cubic regularization hold in the presence of smooth equality constraints, under standard constraint qualifications. It positively answers the issue raised in [cartis2013evaluation, curtis2018complexity], showing that orders remain unchanged unless infeasibility is prioritized.

Contrasts to Two-Phase and Augmented Lagrangian Approaches:

• Most two-phase algorithms optimize feasibility then optimality, inducing non-optimal constraint violation bounds or strong scaling requirements between tolerance parameters.
• Augmented Lagrangian and penalty-based approaches achieve optimality complexity for unconstrained instances but are less effective with complicated constraints or when seeking higher-order KKT points.
• The SCP method unifies the best features—tight constraint violation control and optimality reduction—in a single, globally convergent framework.

Practical Implications:

• The result implies no additional “complexity penalty” arises for nonconvex constraints, as long as only equality conditions and the required smoothness/regularity are present.
• The algorithm is suitable for large-scale applications (PDE-constrained optimization, control, resource allocation) due to inexact subproblem tolerance, and parallelization of sub-space computations.

Implications and Future Directions

Practical Impact: The SCP method offers a robust, theoretically sharp baseline for large-scale nonlinear programming solvers targeting high-quality solutions with rigorous stationarity guarantees. The absence of any required penalty/augmentation parameters or two-phase logic simplifies algorithm engineering and hyperparameter selection.

Open Theoretical Directions:

• Extending these optimal complexity guarantees to inequality-constrained, or general nonpolyhedral, constraints remains open. The magnitude of the best attainable bound for constraint satisfaction with inequalities is unresolved.
• Matching lower bounds for constraint violation complexity are unknown in the equality-constrained, nonconvex regime. If possible, further acceleration of constraint satisfaction while holding optimality complexity unchanged would be notable.
• Richer structure (e.g., manifold constraints, partial smoothness) may require adaptation of SCP to exploit geometry more fully, potentially improving constants or rates.

Conclusion

The SCP method presented in (2604.02747) establishes that, under LICQ and smoothness, equality constraints do not fundamentally worsen the global complexity of nonconvex, smooth optimization relative to the unconstrained regime, for both first- and second-order points. The algorithm is architecturally simple, robust to inexactness, and ensures both rigorous worst-case iteration complexity and local quadratic convergence—a combination not previously achieved in constrained optimization literature. The work fills a key theoretical gap and provides a strong template for future advances in the global theory and practice of nonlinear programming algorithms.