Kelley Cutting Plane Method

• Kelley cutting plane methods are iterative optimization frameworks that refine feasible regions through hyperplane cuts derived from subgradient oracles.
• They incorporate techniques like ACCPM, which leverages analytic centers and Newton methods to improve stability and reduce iteration complexity.
• Modern variants use randomized center selection and constraint pruning to enhance scalability and maintain robust convergence in high-dimensional problems.

A Kelley cutting plane-like method refers to a general strategy for convex (and, more generally, certain nonconvex) optimization in which the feasible region or the epigraph of the objective is iteratively approximated from the outside (“outer-approximation”) by accumulating supporting halfspaces (cuts) generated from first-order (subgradient or separation) oracles. The method originated with J.E. Kelley’s 1960 work, which introduced a practical approach to solving nondifferentiable convex programs via linear programming relaxations that are successively tightened with hyperplane cuts. Over the subsequent decades, Kelley-like strategies have underpinned optimization in diverse areas, leading to numerous algorithmic variants and application-specific implementations that preserve the iterative cutting-plane spirit.

1. The Classical Kelley Cutting-Plane Paradigm

In the basic Kelley method, the general convex optimization problem

$\min_{x\in\mathcal{X}} f(x),$

where $\mathcal{X} \subset \mathbb{R}^n$ is closed and convex and $f$ is convex, is solved by constructing a sequence of polyhedral outer approximations of $\mathcal{X}$ or the epigraph of $f$. At each iteration, the current approximation $Q_k$ is refined as follows:

• A trial point $x_k$ is generated—often, but not necessarily, by minimizing a piecewise-linear lower approximation of $f$ over $Q_k$ or by minimizing $f$ (or its lower model) over a trust region.
• A subgradient (or separation) oracle is called at $\mathcal{X} \subset \mathbb{R}^n$0. If $\mathcal{X} \subset \mathbb{R}^n$1 (or is optimal), the process can terminate or an “objective” cut is added. Otherwise, the oracle returns a hyperplane separating $\mathcal{X} \subset \mathbb{R}^n$2 from $\mathcal{X} \subset \mathbb{R}^n$3 (a “feasibility” cut), which is then added to $\mathcal{X} \subset \mathbb{R}^n$4, yielding $\mathcal{X} \subset \mathbb{R}^n$5.
• The process repeats, maintaining convergence by shrinking the localization region at each step.

Early methods selected the next iterate as the center of gravity or Chebyshev center of the current polyhedron, but these are computationally intractable in high dimension. Contemporary Kelley-like methods typically employ more tractable analytic centers or randomization strategies to select the next search point (Badenbroek et al., 2020).

2. ACCPM: Analytic Center Cutting Plane Method

A prominent Kelley-style method is the Analytic Center Cutting Plane Method (ACCPM), which leverages the analytic center of the current feasible region for point selection, improving stability and iteration complexity (Badenbroek et al., 2020). The general framework is as follows:

1. Initialization: Start with $\mathcal{X} \subset \mathbb{R}^n$6, for some large $\mathcal{X} \subset \mathbb{R}^n$7, and an initial point $\mathcal{X} \subset \mathbb{R}^n$8.
2. Iteration: At step $\mathcal{X} \subset \mathbb{R}^n$9, let $f$0.
3. Point Generation: Compute the analytic center $f$1, where

$f$2

1. Separation Oracle: If $f$3 is feasible, add an objective cut; otherwise, add the returned separating cut.
2. Refinement: Update $f$4 with the new cut. Terminate upon reaching a prescribed optimality gap.

Convergence is governed by self-concordance theory, requiring at most $f$5 iterations to achieve $f$6-accuracy, where $f$7 is the variable dimension and $f$8 the initial radius (Badenbroek et al., 2020).

ACCPM is not restricted to copositive programs but generally applies to any convex optimization problem equipped with an efficient separation oracle.

3. Algorithmic Components, Pseudocode, and Variants

The essential components in a Kelley cutting plane-like method comprise:

• Polyhedral localization: Iteratively maintain and update an outer approximation to the feasible region or the sublevel/epigraph set of the objective.
• Separation oracle: At each iterate, check feasibility or optimality; if violated, return a cut that supports the feasible set at (approximate) boundary points.
• Center selection: The next query point is often the analytic center of the localization polyhedron (ACCPM), a volume center, or, in randomized variants, a uniformly sampled point (e.g., as in random cutting-plane methods for SDPs (Marecek et al., 2021)).
• Modifications for stability: Prune “weak” constraints, limit the number of cuts, or employ infeasible-start Newton methods to address ill-conditioning.

A typical high-level outline for ACCPM is:

$\mathcal{X}$2 (Badenbroek et al., 2020)

Additional modernizations include randomized center selection (random hit-and-run sampling with boundary oracles in semidefinite programming (Marecek et al., 2021)) and constraint management policies (pruning, warm-starting, limited memory).

4. Theoretical Guarantees and Practical Scalability

Kelley-like cutting-plane methods possess strong polynomial-time convergence properties under standard convexity and separation oracle assumptions. For ACCPM, the volume of the feasible localization region decreases by a definite factor at each iteration. For problems in the copositive cone, the empirical number of oracle calls grows as $f$9 for $\mathcal{X}$0 matrices, substantially better than theoretical ellipsoid-method bounds (Badenbroek et al., 2020).

Notable practical features:

• Robustness: Well-conditioned center selection renders ACCPM insensitive to moderate ill-conditioning compared to classical Chebyshev or gravity-center methods.
• Cut management: Pruning cuts based on Dikin–relevance or violation, and capping the total number of active constraints, maintains computational efficiency and numerically stability.
• Oracle efficiency: For challenging membership queries (e.g., co-NP-complete copositivity checks), practical realization via specialized MILP subproblems makes the approach viable for moderate-to-large instances.

Numerical studies indicate that ACCPM requires significantly fewer oracle calls than classical approaches such as the ellipsoid method, especially for structured matrix cones (Badenbroek et al., 2020).

5. Comparison with Classical Kelley and Broader Algorithmic Context

Kelley’s original method is a generic outer-approximation framework, employing the center-of-gravity or Chebyshev center for selecting iterates. For high-dimensional problems, these are computationally prohibitive. ACCPM and its relatives replace these inefficient centers with analytic centers (using log-barrier minimization), enabling efficient Newton-based subproblems and scalable cut management (Badenbroek et al., 2020).

Distinguishing features relative to classical Kelley:

• Tractable center computation: ACCPM’s analytic center can be efficiently updated by damped Newton methods, while Chebyshev and gravity centers are infeasible for large $\mathcal{X}$1.
• Constraint management: Modern Kelley-like schemes actively manage cut sets, utilizing relevance measures and pruning to avoid unbounded constraint accumulation.
• Separation oracles: The approach is generic, accommodating subgradient, MILP-based, or spectral oracles depending on problem structure.

The broader framework encompasses analytic center, randomized, and supporting hyperplane methods, all adhering to the Kelley cutting-plane philosophy of iteratively refining polyhedral outer approximations with supporting halfspace cuts. These approaches have been successfully applied not only to matrix cone problems but also to structured optimization, semi-infinite programming, portfolio optimization, and semidefinite programs (Badenbroek et al., 2020, Hofmann, 2014, Bienstock et al., 24 Oct 2025, Marecek et al., 2021).

6. Applications and Extensions

Kelley cutting plane-like methods are foundational in diverse application domains:

• Copositive and semidefinite programming: ACCPM and randomized cuts for spectrahedral constraints (Badenbroek et al., 2020, Marecek et al., 2021).
• Portfolio optimization: Kelley-type approaches for enforcing Conditional Value-at-Risk constraints without reformulation overhead (Hofmann, 2014).
• Large-scale power systems optimization: Outer-approximation methods for SDP relaxations of AC optimal power flow, exploiting block PSD structure and modern cut management (Bienstock et al., 24 Oct 2025).
• Machine learning: Cutting-plane schemes for semi-infinite quadratic problems in manifold classification (Chung et al., 2017).
• Non-Euclidean optimization: Hyperbolic generalizations via the one-shot Klein cutting-plane method use Lorentzian structure to translate subgradient cuts to Euclidean central cuts in the Beltrami–Klein chart (Zhang et al., 17 May 2026).

The ongoing development of Kelley-like strategies includes variance-reduced and randomized sampling approaches, analytic center and volume-based refinements, and hybridization with bundle and level methods for enhanced performance in high dimensions and non-Euclidean geometries.

• "An Analytic Center Cutting Plane Method to Determine Complete Positivity of a Matrix" (Badenbroek et al., 2020)
• "A Cutting-plane Method for Semidefinite Programming with Potential Applications on Noisy Quantum Devices" (Marecek et al., 2021)
• "Accelerated Portfolio Optimization with Conditional Value-at-Risk Constraints using a Cutting-Plane Method" (Hofmann, 2014)
• "Advanced Cutting-Plane Algorithms for ACOPF" (Bienstock et al., 24 Oct 2025)
• "Learning Data Manifolds with a Cutting Plane Method" (Chung et al., 2017)
• "One-Shot Klein Cutting Planes for Lipschitz Geodesically Convex Optimization in Hyperbolic Space" (Zhang et al., 17 May 2026)
• "An optimal variant of Kelley's cutting-plane method" (Drori et al., 2014)