Arc-Search Interior-Point Methods

• Arc-search interior-point methods are primal-dual algorithms that compute second-order elliptical arcs to closely approximate the central path.
• They achieve improved iteration complexity and efficiency across linear, quadratic, and nonlinear programming problems.
• Enhancements like inexact Newton solves and momentum acceleration further boost performance for large-scale, sparse optimization tasks.

Arc-search interior-point methods (arc-search IPMs) are a class of primal-dual algorithms for convex, linear, quadratic, and nonlinear programming that advance each iterate along an ellipsoidal arc, computed to second order, designed to approximate the central path more accurately than traditional straight-line or affine-scaling steps. These methods leverage higher-order information from the interior-point framework, enabling larger or more central steps, and systematically incorporate properties of the central-path’s curvature. Arc-search IPMs have been extensively analyzed and increasingly implemented for linear programming, convex quadratic programming, and more general nonlinear constrained optimization, where they establish both improved iteration complexity in theory and practical reductions in iteration counts relative to line-search-based interior-point methods.

1. Theoretical Foundation and Central Path Approximation

Arc-search IPMs are formulated in the context of primal-dual path-following. In traditional interior-point methods, the central path is traced by solving parameterized Karush-Kuhn-Tucker (KKT) systems with a gradually decreasing barrier parameter. Line-search IPMs use step directions obtained via a Newton solve, then update the iterates along this direction with a scalar step size. In contrast, arc-search IPMs compute both the first- and second-order derivatives of a "central-path-like" parametric curve—a local ellipse—that simultaneously matches the tangent and curvature at the current iterate.

The main geometric principle is to replace a straight-line update $x_{k+1} = x_k + \alpha \dot{x}$ by a second-order arc:

$$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$

with similar formulas for all primal and dual variables. Here, $\dot{x}, \ddot{x}$ are the first and second derivatives along the central curve defined via the KKT system and associated residuals. This ensures that arc-search steps remain closer to the true central path, particularly in regimes of nonlinearity or strong curvature (Yang, 2014, Yang et al., 2015, Yang, 2022).

2. Algorithmic Structure and Polynomial Complexity

Arc-search IPMs share the following canonical structure across a broad class of linear, quadratic, and nonlinear programs (Yang, 2022, Yang, 2013, Yang, 2016, Iida et al., 2019, Yang, 2024):

1. Build the KKT system and compute both the first and second Newton directions. For general $n$-variable problems:
   • Solve $J \dot{w} = -F(w)$ for the first derivative (affine-scaling step).
   • Solve $J \ddot{w} = \text{curvature term}$ for the second derivative (corrector).
2. Parametrize the update along an ellipse as above, building $w(\alpha)$.
3. Choose the step length $\alpha$ to maximize progress while preserving strict positivity, centrality, and sufficient reduction of the duality gap. Often, this amounts to finding the minimal positive root of quartic or cubic polynomials in $\sin\alpha$, due to the quadratic form of slacks and multipliers under the arc (Yang, 2013).
4. (Optionally) Perform a corrector step by solving a third KKT-like system at $w(\alpha)$ to recover centrality or feasibility.
5. Update all variables and check stopping criteria.

Complexity bounds are proven by showing that, under mild assumptions (LICQ, strict complementarity, operator regularity), there exists a uniform lower bound for the step sizes, so each iteration contracts the duality gap and KKT residuals by a constant factor. This yields the following state-of-the-art iteration bounds:

• For convex quadratic programming with box constraints: $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$0 iterations (Yang, 2013).
• For general convex programs: $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$1 or, with further structural simplification, $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$2 (Yang, 2022, Iida et al., 2024).
• For linear programming: $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$3, where $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$4 is the input bit-length of the problem data (Yang et al., 2015, Yang, 2016).

These bounds match or nearly match the best-known results for path-following methods and, for large-scale or high-precision problems, constitute substantial theoretical progress.

3. Application Domains and Structural Exploitation

Arc-search IPMs have been successfully specialized in several major domains:

• Linear Programming (LP): The methodology is applied to both feasible and infeasible path-following models, yielding iteration complexity $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$5 or $$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$6, and practical iteration counts that consistently outperform or match Mehrotra's predictor-corrector algorithm (Yang, 2014, Yang et al., 2015, Yang, 2016, Iida et al., 2023).
• Convex Quadratic Programming (QP) with Box Constraints: The separability and diagonality of the box constraints enable explicit starting points, reduced KKT systems, and enhanced per-iteration efficiency; the arc-search method achieves the best-known complexity bounds and robust practical convergence (Yang, 2013).
• General Convex and Nonlinear Programming: The approach generalizes via Lagrangian- and barrier-based KKT perturbations, with first and second derivatives computed using automatic or analytic differentiation. Slacks are introduced to handle inequalities, and the solver adapts the arc-search paradigm to arbitrary smooth convex and even certain nonconvex objectives (Iida et al., 2019, Yang, 2022, Yang, 2024).

Structural table:

Domain	Arc-search Complexity	Special Features
Linear Programming	$$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$7	Explicit ellipse formulas, fast starts
Box-constrained QP	$$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$8	Diagonal barriers, no Phase-I required
NLP/Convex Prog.	$$x(\alpha) = x_k - \dot{x} \sin\alpha + \ddot{x} (1 - \cos\alpha)$$9	General KKT, second-order corrections

4. Enhancements: Inexact Solves and Momentum Acceleration

Recent developments incorporate two practical enhancements:

• Inexact Newton Solves: Iterative solvers (e.g., preconditioned conjugate gradients) are used to compute the arc directions inexactly, trading accuracy for speed. Complexity analysis shows the iteration bound improves over inexact line-search IPMs, giving $\dot{x}, \ddot{x}$0 in the linear case versus $\dot{x}, \ddot{x}$1 for inexact line-search (Iida et al., 2024).
• Momentum/Nesterov Restarting: Momentum terms are added to the iterate updates (as in accelerated gradient methods), either inside the arc-search neighborhood or in Mehrotra-type variants. Nesterov-style momentum can strictly reduce the average number of iterations and wall-clock time, with polynomial-time convergence guarantees maintained (Iida et al., 2023).

In complex scenarios, dropping some higher-order terms in the second-derivative system can further accelerate per-iteration cost with minimal impact on total iterations (Iida et al., 2019, Yang, 2024).

5. Implementation and Practical Performance

Arc-search IPMs have been implemented in MATLAB and Python for dense and sparse problems, with key implementation features:

• Two linear systems per iteration (with identical coefficient matrices), allowing factorizations or preconditioners to be reused.
• Step-size selection via small univariate polynomials, for which explicit roots are computed.
• Termination based on residual norms and duality gap.
• Empirical results on Netlib LP, standard quadratic, and nonlinear test suites show 20–50% fewer iterations than standard line-search IPMs, and typically 20–35% reductions in CPU time when leveraging sparse direct or iterative linear algebra (Yang, 2013, Yang, 2014, Iida et al., 2019, Iida et al., 2024, Yang, 2024).

Table: Representative test results (iteration counts, arc-search vs. line-search IPM, select benchmarks) (Iida et al., 2024, Iida et al., 2019, Yang, 2024):

Problem	Arc-search its	Line-search its	Time ratio (arc/line)
CRE-A (LP)	41	113	0.93
BT12 (QP/NLP)	4	20	1.56
HS19 (NLP)	22	35	0.64
HS80 (NLP)	20	29	0.70

Arc-search methods retain reliability and robustness, even in degenerate or highly ill-conditioned cases where line-search IPMs may require additional stabilization (Yang, 2014, Yang, 2016). Explicit interior starting points often allow elimination of phase-I procedures.

6. Impact, Limitations, and Future Directions

Arc-search IPMs resolve a gap noted by Todd (2002) regarding the existence of theoretically-justified and computationally efficient IPMs: these methods provide both polynomial-time iteration bounds and empirical iteration counts within 5–10% of heuristic best-known methods, such as Mehrotra's algorithm (Yang, 2016).

Limitations include:

• The need for twice or thrice continuously differentiable problem data for general nonlinear programs.
• Higher per-iteration cost due to solving two KKT-like systems, though this is mitigated by matrix reuse.
• In some general nonlinear cases, evaluation of third-order derivatives is nontrivial; strategies for approximating or omitting tensor terms have proved successful in practice (Iida et al., 2019, Yang, 2024).

Current research targets scalable implementations on very large, sparse instances (where iterative methods and inexactness are key), and extensions to specialized domains such as model predictive control, robust estimation, and quantum-enhanced optimization (Yang, 2013, Iida et al., 2024).

7. Related Methodologies and Algorithmic Comparisons

Arc-search IPMs are closely related to, but distinct from, predictor–corrector IPMs (e.g., Mehrotra-type) and other path-following algorithms. The main differentiator is the use of a two-term Taylor ellipse for local advancement, as opposed to straight-line steps. Compared with line-search IPMs:

• Arc-search permits systematically larger step sizes for equivalent or improved centrality, making global and practical convergence faster (Yang, 2014, Yang et al., 2015).
• The theoretical iteration complexity is at least as strong as line-search methods; in feasible-start scenarios, under box constraints or wide neighborhoods, it matches the best-known rates (Yang, 2013, Yang, 2016, Yang, 2022).
• Empirical studies on Netlib, CUTEst, and other standard suites confirm advantages in robustness, efficiency, and final residuals or objective accuracy (Yang, 2014, Yang, 2013, Iida et al., 2019).

A plausible implication is that arc-search IPMs are suitable for integration into existing optimization software requiring high reliability or facing large-scale, structurally regular problems, especially where step length and robustness are crucial metrics.