Beyond Averaging in John Ellipsoid Approximation: High-Accuracy Algorithms in the Leverage-Score Model

Abstract: The John ellipsoid of a symmetric polytope $$P={\mathbf{x}\in\mathbb{R}<sup>d:|\mathbf{A}\mathbf{x}|_\infty\le1}$$, $\mathbf{A}\in\mathbb{R}^{n\times d}$, is computed by a long line of leverage-score algorithms, from Cohen, Cousins, Lee and Yang (COLT 2019) to its successors [WY24, CLS+25], all reaching a $(1+\varepsilon)$-approximation in $Θ(\varepsilon<sup>{-1}\log(n/d))$ iterations. We separate this complexity into three costs the modern line conflates (certification, identification, and accuracy) and locate the historical $\varepsilon<sup>{-1}$ in the first alone. In the equivalent D-optimal-design form $$\min_{\mathbf{p}\inΔ_n}-\log\det(\sum_i p_i\mathbf{a}_i\mathbf{a}_i<sup>\top)$$, the leverage-score oracle is exactly the first-order oracle and the $(1+\varepsilon)$-John guarantee the Frank-Wolfe gap $g(\mathbf{p})\le\varepsilon d$; through this dictionary the costs come apart. The $\varepsilon<sup>{-1}$ is a certification artifact: the uniform average of the iterates, the certificate used throughout the line, has gap exactly $Θ(1/T)$, however cheap each iteration is made. Pointed instead at the last iterate the same oracle is fast: a warm-started accelerated method reaches the guarantee in $C(\mathbf{A})+O(\sqrtκ\log(1/\varepsilon))$ queries after an $\varepsilon$-independent setup $C(\mathbf{A})$, and once the optimal face is identified the facial problem is an unconstrained self-concordant minimization whose Hessian the oracle recovers exactly, so damped Newton needs only $O(\log\log(1/\varepsilon))$ steps, for a total of $C(\mathbf{A})+O(d<sup>2\log\log(1/\varepsilon))$ queries. The accuracy dependence is thus doubly logarithmic after an $\varepsilon$-independent, condition-dependent setup; the open problem is the remaining identification cost (a condition-free bound on reaching the optimal face) and lower bounds. Accuracy is not the obstruction.

• The paper reveals that uniform averaging in leverage-score algorithms creates an O(ε⁻¹) iteration barrier, independent of the problem's geometry.
• Accelerated techniques like restarted FISTA decouple the identification phase to achieve geometric decay in query complexity.
• Newton iterations on the optimal face enable quadratic convergence, reducing overall queries to O(d² log log(1/ε)) under ideal conditions.

High-Accuracy Algorithms for John Ellipsoid Approximation in the Leverage-Score Model

Background and Motivation

The computation of the John ellipsoid—a maximal-volume inscribed ellipsoid of a centrally symmetric polytope—is central to convex optimization, sampling, and experimental design. The problem is equivalent to optimizing the D-optimal design criterion, and the $\ell_\infty$ Lewis weight computation, with foundational links to preconditioning and geometric algorithms. State-of-the-art leverage-score algorithms, pioneered by Cohen et al. (CCLY), have achieved $(1+\varepsilon)$-approximation in $\Theta(\varepsilon^{-1}\log(n/d))$ iterations, with further improvements in per-iteration computational cost but not in overall query complexity.

Recent algorithmic advances have crystallized three granular costs conflated in previous analyses: certification (verifying approximate optimality), identification (finding the optimal face/support), and accuracy (driving the approximation error $\varepsilon$ to zero). The main technical challenge lies in decoupling these costs and understanding their oracle complexity.

Oracle Structures and Certification Boundaries

The leverage-score oracle computes $$v_i(\mathbf{p}) = \mathbf{a}_i^\top(\sum_j p_j \mathbf{a}_j \mathbf{a}_j^\top)^{-1}\mathbf{a}_i$$ for weights $\mathbf{p}$, forming the core computational primitive. Importantly, the gradient of the D-optimal design objective, $$f(\mathbf{p}) = -\log\det(\sum_i p_i \mathbf{a}_i \mathbf{a}_i^\top)$$, coincides with the negative leverage score vector ($\nabla f(\mathbf{p})_i = -v_i(\mathbf{p})$), making the leverage-score oracle equivalent to a first-order oracle for convex optimization.

The $(1+\varepsilon)$-John guarantee—given by $\max_i v_i(\mathbf{p}) \leq (1+\varepsilon)d$—translates directly to a Frank–Wolfe duality gap. The certificate traditionally used is the uniform average over all iterates, which, by Jensen’s inequality and intrinsic averaging behavior, imposes a $(1+\varepsilon)$0 decay in certificate gap, locking the iteration count at $(1+\varepsilon)$1. This averaging barrier is not dictated by the problem’s geometry or oracle power, but purely by the certification rule, notably visible even in instances with perfect conditioning.

Identification and Accelerated Accuracy Phases

The identification cost is the iteration required to locate and restrict to the optimal face (contact set). Classical away-step Frank–Wolfe (Wolfe–Atwood) achieves global linear convergence in both objective and gap, depending on the facial condition number. Existing work [zhao2023] quantifies this as $(1+\varepsilon)$2, with $(1+\varepsilon)$3 determined by the pyramidal width of the design polytope. However, accelerated methods—specifically restarted FISTA on the optimal face, after warm starting—reduce the query complexity to $(1+\varepsilon)$4, with $(1+\varepsilon)$5 never worse than the unaccelerated rate by more than a polynomial factor in dimension.

Once the contact set is identified and strict complementarity holds, the minimization over the optimal face becomes unconstrained and the objective self-concordant, eliminating all conditioning from the accuracy phase. The Hessian of $(1+\varepsilon)$6 restricted to the optimal face can be recovered exactly in $(1+\varepsilon)$7 leverage-score queries via an explicit Sherman–Morrison rank-one identity. Therefore, damped Newton iteration achieves quadratic convergence, with a doubly logarithmic dependence $(1+\varepsilon)$8 on accuracy, after a condition-dependent initialization phase.

Key Results

• Averaging Barrier: Uniform averaging of leverage-score iterates fundamentally cannot beat $(1+\varepsilon)$9 iteration complexity, even in perfectly conditioned instances; the gap decays exactly as $\Theta(\varepsilon^{-1}\log(n/d))$0.
• Accelerated Facial Phase: Restarted, warm-started FISTA yields geometric decay, with query complexity $\Theta(\varepsilon^{-1}\log(n/d))$1 after warm start, with $\Theta(\varepsilon^{-1}\log(n/d))$2 dependent on instance conditioning but independent of $\Theta(\varepsilon^{-1}\log(n/d))$3.
• Facial Newton Phase: Once the optimal face is identified, Newton steps converge doubly logarithmically in $\Theta(\varepsilon^{-1}\log(n/d))$4, and the required Hessian can be reconstructed precisely from oracle responses. Total query complexity becomes $\Theta(\varepsilon^{-1}\log(n/d))$5.
• Certification, Identification, Accuracy Separation: The historical $\Theta(\varepsilon^{-1}\log(n/d))$6 iteration barrier in leverage-score-based algorithms arises solely from the certification mechanism; both identification and accuracy phases admit fundamentally faster strategies when decoupled.

Implications and Outlook

This technical separation exposes certification via averaging as a fundamental bottleneck. Accelerated and Newton-type methods, when run on the optimal face in the leverage-score model, are exponentially more query-efficient for high accuracy than averaging-based algorithms, conditional on identification. Thus, algorithmic focus should shift toward robust, efficient identification mechanisms, possibly leveraging combinatorial or algebraic characterization of the contact set.

The condition number dependence in identification remains a primary barrier to uniform query complexity. Lower bounds for leverage-score oracles and characterization of degenerate cases lacking strict complementarity are important future directions. Additionally, stability under approximate leverage-score evaluation, as seen in sketching and streaming algorithms, warrants substantial investigation for practical deployment.

The separation of certification, identification, and accuracy costs is likely to catalyze new algorithmic paradigms across convex optimization, geometric rounding, statistical experimental design, and beyond. As leverage-score methods are increasingly embedded in large-scale data systems, understanding and exploiting this separation will be critical for optimal algorithmic design.

Conclusion

The paper precisely locates and explains the $\Theta(\varepsilon^{-1}\log(n/d))$7 complexity barrier in John ellipsoid approximation, showing it arises from certificate averaging rather than oracle or problem structure. By analyzing the D-optimal design problem in the leverage-score model and exploiting facial geometry, accelerated and Newton methods achieve exponential improvements in query complexity for high precision. This yields a formal separation of certification, identification, and accuracy complexities, establishing an architecture for further improvements and highlighting identification as the primary open problem. Future research should focus on uniform identification algorithms and extend these high-accuracy strategies to approximate oracle models and non-generic problem instances.