Performance Estimation Framework (PEP)

• Performance Estimation Framework (PEP) is a methodology that formulates worst-case performance analysis of iterative optimization algorithms as an optimization problem.
• It leverages semidefinite programming via Gram matrix formulations and interpolation constraints to derive tight and verifiable performance bounds.
• PEP has evolved to support decentralized optimization, algorithm design, and automated proof extraction, driving advances in various iterative methods.

Searching arXiv for recent and foundational papers on the Performance Estimation Problem framework and related extensions. The Performance Estimation Framework, usually centered on the Performance Estimation Problem (PEP), is a computer-assisted methodology for computing tight worst-case bounds for iterative optimization algorithms by formulating the analysis itself as an optimization problem. In its standard form, PEP maximizes a chosen performance criterion over all admissible initial conditions and all problem instances in a prescribed class, while replacing infinite-dimensional objects such as functions, operators, or networks by finite samples constrained through interpolation conditions. The resulting finite-dimensional model is typically a semidefinite program (SDP) in a Gram matrix of inner products and a set of scalar function values, and its solution yields both a worst-case bound and an extremal instance (Drori et al., 2012, Rubbens et al., 2023, Goujaud et al., 2022).

1. Historical formulation and conceptual scope

PEP was introduced from the observation that the worst-case behavior of a first-order black-box method is itself an optimization problem (Drori et al., 2012). In the original smooth convex setting, the objective was to analyze methods for unconstrained minimization over $\mathbb{R}^d$, under convexity and $L$-smoothness, with a first-order oracle returning $f(x_k)$ and $\nabla f(x_k)$ at each iterate (Drori et al., 2012). The framework subsequently expanded into a general methodology for fixed-step first-order methods, composite models, proximal and projection oracles, linear minimization oracles, Bregman variants, inexact oracles, and decentralized optimization (Goujaud et al., 2022, Colla et al., 2022).

A standard distinction in the literature is between the inner worst-case analysis problem and the outer algorithm-design problem. For a fixed method, PEP computes worst-case performance over all functions and trajectories consistent with the update rules and the class constraints (Colla et al., 2022, Goujaud et al., 2022). In later developments, the outer problem of choosing the method’s parameters is itself optimized. The paper "Branch-and-Bound Performance Estimation Programming: A Unified Methodology for Constructing Optimal Optimization Methods" formulates this joint design-and-analysis task as a nonconvex quadratically constrained quadratic program and solves it to certifiable global optimality with customized branch-and-bound (Gupta et al., 2022).

The framework is therefore both an analysis formalism and a design formalism. This suggests why the expression “Performance Estimation Framework” is often more informative than the narrower phrase “Performance Estimation Problem”: the same core machinery supports exact worst-case certification, automated parameter tuning, and the extraction of analytical proofs from numerical certificates (Gupta et al., 2022, Goujaud et al., 2022).

2. Generic SDP formulation

For a method $M$ run for $K$ steps on a function class $\mathcal F$ with an initial set $I^0$, the worst case is written as a finite-sample optimization problem over iterates, gradients, function values, and a minimizer. One generic form is

$$\sup_{f\in\mathcal F,\;x^0,\dots,x^K,\;x^*\in\arg\min f} \;\mathcal P(f,x^0,\dots,x^K,x^*)$$

subject to function-class interpolation constraints, algorithmic recursion constraints, and initial constraints such as $\|x^0-x^*\|^2\le R^2$ (Colla et al., 2022). In decentralized formulations the same structure appears with agent-indexed variables and additional optimality and consensus constraints (Colla et al., 2024).

The key modeling device is the Gram matrix. If $L$0 collects all vectors that appear in the description—iterates, gradients, auxiliary points, directions, operator outputs—then

$L$1

encodes all relevant inner products (Colla et al., 2024, Goujaud et al., 2022). Function values are stored in a separate vector. Method constraints, initialization constraints, interpolation inequalities, and many performance measures then become linear or linear-matrix-inequality constraints in the entries of $L$2 and the scalar function values (Goujaud et al., 2022, Rubbens et al., 2023).

This SDP representation is dimension-independent in the standard sense used in the literature: feasibility depends only on the Gram matrix, not on an explicit ambient dimension. One formulation states that the construction is lossless when one does not fix the dimension $L$3 and imposes no rank constraint on $L$4 (Colla et al., 2022). Another states that under the large-scale assumption $L$5, rank constraints on $L$6 can be dropped without relaxation (Gupta et al., 2022). These statements clarify an important point: tightness is not a generic consequence of semidefinite relaxation alone, but of exact interpolation and the absence of restrictive rank constraints.

3. Interpolation theory and exact class descriptions

Interpolation constraints are the central mechanism by which PEP replaces infinite-dimensional function or operator classes with finite inequalities on sampled data (Rubbens et al., 2023). For $L$7-smooth convex functions, a sampled set $L$8 is interpolable if the pairwise inequalities

$L$9

hold for all $f(x_k)$0 (Goujaud et al., 2022, Rubbens et al., 2023). Equivalent formulations include the descent lemma

$f(x_k)$1

and Baillon–Haddad cocoercivity,

$f(x_k)$2

both of which are routinely embedded in PEP models (Colla et al., 2024, Goujaud et al., 2022).

For $f(x_k)$3-strongly convex and $f(x_k)$4-smooth functions, the exact pairwise interpolation condition can be written as

$f(x_k)$5

for $f(x_k)$6 (Rubbens et al., 2023). Closely related equivalent forms are used internally in software such as PEPit (Goujaud et al., 2022).

The same interpolation viewpoint extends beyond smooth convex classes. The surveyed literature includes strong convexity alone, smooth functions with bounded subgradient, indicator functions of convex sets, monotone operators, cocoercive operators, Lipschitz operators, linear operators with bounded singular value, resolvents, proximal mappings, and consensus operators in decentralized optimization (Rubbens et al., 2023). Composite models such as $f(x_k)$7 are handled by introducing auxiliary samples $f(x_k)$8, $f(x_k)$9, $\nabla f(x_k)$0, and combining function interpolation with linear-operator interpolation constraints such as

$\nabla f(x_k)$1

for $\nabla f(x_k)$2 (Rubbens et al., 2023).

A recurrent source of confusion is the role of relaxation. The literature emphasizes that using tight interpolable class descriptions is crucial, because weaker necessary conditions can significantly degrade the bounds and mislead parameter selection (Rubbens et al., 2023). In standard smooth convex and smooth strongly convex settings, the interpolation theorems used in PEP are exact necessary-and-sufficient conditions, which is why the resulting SDP can be tight (Goujaud et al., 2022, Rubbens et al., 2023).

4. Performance criteria, algorithm classes, and representative bounds

PEP supports a broad range of performance measures. Canonical examples include final suboptimality $\nabla f(x_k)$3, distance to the solution $\nabla f(x_k)$4, gradient norm $\nabla f(x_k)$5, ergodic criteria, Frank–Wolfe dual gaps, and, in decentralized settings, consensus error and worst-agent metrics (Goujaud et al., 2022, Colla et al., 2024). Because these criteria are linear or convex quadratic in Gram entries and scalar values, they remain compatible with the SDP framework (Colla et al., 2022, Colla et al., 2024).

The original smooth convex analysis already showed that PEP can sharpen classical constants. For the gradient method with fixed normalized step size $\nabla f(x_k)$6, one tight analytical bound is

$\nabla f(x_k)$7

and explicit worst-case functions attain the bound (Drori et al., 2012). The same work developed semidefinite relaxations for a broader first-order black-box class including the heavy-ball method and fast gradient schemes, and used them to produce new numerical bounds and optimal step-size procedures (Drori et al., 2012).

Later formulations generalized this repertoire. PEPit covers gradient, projection, proximal, linear optimization, approximate, and Bregman oracles, and models methods such as Gradient Descent, Projected and Proximal Gradient, Douglas–Rachford and ADMM variants, Frank–Wolfe, Mirror Descent, composite models, and stochastic or randomized examples via expected metrics (Goujaud et al., 2022). The interpolation-centered survey likewise emphasizes accelerated methods, heavy-ball and momentum variants, primal–dual and splitting methods, coordinate descent, and decentralized algorithms (Rubbens et al., 2023).

A notable later development is the use of PEP for algorithm design. BnB-PEP poses the problem of finding the optimal method as a nonconvex QCQP, replaces the PSD proof matrix $\nabla f(x_k)$8 by a Cholesky factorization $\nabla f(x_k)$9, and solves the resulting design-and-analysis problem with a customized three-stage branch-and-bound algorithm (Gupta et al., 2022). The method is applied to smooth strongly convex gradient reduction, smooth convex function decrease without momentum, smooth nonconvex gradient reduction, and weakly convex nonsmooth analysis via the Moreau envelope (Gupta et al., 2022). An important consequence is that inner-dual solutions can become human-readable proofs: the paper explicitly uses BnB-PEP to find proofs with potential function structures and thereby systematically generate analytical convergence proofs (Gupta et al., 2022).

5. Software, proof extraction, and practical computation

The software ecosystem around PEP aims to make worst-case analysis close to the notation of an algorithmic paper. PEPit is a Python package in which one declares the function class, specifies initial conditions, writes the method nearly as it would be implemented, sets the performance metric, and lets the package compile the SDP and call a numerical solver (Goujaud et al., 2022). The package follows the Taylor–Hendrickx–Glineur interpolation framework and supports CVXPY with SCS or MOSEK, as well as a direct MOSEK interface (Goujaud et al., 2022).

A characteristic practical feature of PEP is that both primal and dual solutions are informative. The primal Gram matrix yields near-worst-case witnesses, and a Cholesky factorization reconstructs worst-case iterates and gradients (Colla et al., 2022, Colla et al., 2024). The dual variables provide a certifiable proof and can often be translated into an analytic argument by taking appropriate linear combinations of interpolation inequalities (Goujaud et al., 2022). This dual-to-proof interpretation is central in BnB-PEP, where sparse multipliers and low-rank proof matrices are exploited algorithmically and then simplified into analytical potential-function proofs (Gupta et al., 2022).

Computational burden remains an essential limitation. SDP size grows with the number of sampled vectors, which typically grows with the iteration horizon and with the number of auxiliary variables introduced by composite or distributed models (Colla et al., 2022, Rubbens et al., 2023). The literature therefore emphasizes normalization, structure exploitation, symmetry reduction, and small proof systems. In the decentralized context, one paper notes that for $M$0 and DGD spectral PEP, solve times were approximately $M$1 seconds for $M$2 on a laptop (Colla et al., 2022). In another direction, the customized BnB-PEP solver reports runtime reductions from hours to seconds and weeks to minutes relative to off-the-shelf spatial branch-and-bound on selected global-design tasks (Gupta et al., 2022).

6. Decentralized optimization, symmetry reductions, and network-size independence

A major recent extension of PEP concerns decentralized optimization, where $M$3 agents minimize

$M$4

using local gradients and communication through a mixing matrix $M$5 (Colla et al., 2022, Colla et al., 2022). Early decentralized PEP formulations modeled each agent separately and introduced either an exact matrix-specific encoding $M$6 or a spectral-range-based LMI relaxation valid over an entire class of symmetric generalized doubly stochastic matrices with prescribed eigenvalue interval (Colla et al., 2022).

The spectral formulation represents consensus steps through average preservation and LMIs on centered components. In one common symmetric spectral case, the centered outputs satisfy

$M$7

which implies that each consensus step reduces the disagreement by at least a factor $M$8 in the worst case (Colla et al., 2022). This decoupling led to a network-size-independent formulation in which the consensus mode and the orthogonal disagreement mode are separated, and the SDP uses two Gram matrices $M$9 and $K$0 whose sizes depend on the horizon but not on the number of agents (Colla et al., 2022).

The symmetry-based theory of agent equivalence pushes this further. In the fully symmetric setting, if all agents are equivalent, the algorithm, interpolation, and initialization constraints are Gram-representable, and the performance measure together with any non-single-agent initialization constraints are scale-invariant, then the worst-case value $K$1 is computable from a compact SDP whose size is independent of $K$2, and under additional scale-invariance conditions

$K$3

(Colla et al., 2024). The proof uses a symmetrized Gram matrix with repeating diagonal and off-diagonal blocks, together with the reduction

$K$4

which replaces an $K$5 PSD constraint by two small LMIs (Colla et al., 2024).

These formulations have been applied to DGD, DIGing, EXTRA, worst-agent criteria, and percentile criteria (Colla et al., 2022, Colla et al., 2022, Colla et al., 2024). In the EXTRA study, scale-invariant metrics such as objective error at the average iterate and mean iterate error are independent of the number of agents and equal to the two-agent bound, while worst-agent performance is not scale-invariant and requires a multi-class compact PEP (Colla et al., 2024). The same work also states that tight Gram-representable constraints for the exact $K$6 class are impossible, and therefore justifies a convex relaxation based on invariant-subspace interpolation constraints for a larger symmetric class $K$7 (Colla et al., 2024). This is an important qualification: decentralized PEP can be exact for a fixed mixing matrix, or spectral and uniform over a class of matrices, but these are distinct notions of tightness (Colla et al., 2022, Colla et al., 2024).

The modern PEP literature thus presents a unified picture. Worst-case analysis is posed as optimization over sampled information; interpolation constraints turn functional, operator, and network classes into finite constraints; Gram representations make the problems dimension-independent; and SDP or QCQP machinery yields tight numerical bounds, worst-case instances, automated tuning rules, and, in many cases, analytical convergence proofs (Drori et al., 2012, Rubbens et al., 2023, Gupta et al., 2022, Colla et al., 2024).