Performance Estimation Programming

• Performance Estimation Programming (PEP) is an optimization framework that formulates worst-case performance as a semidefinite program over algorithmic trajectories.
• It replaces the infinite-dimensional search over functions with finite interpolation constraints encoded in a Gram matrix, ensuring tight bounds when the interpolation conditions are met.
• PEP has evolved to cover various settings including smooth convex and decentralized optimization, with software tools like PESTO and PEPit automating its worst-case analyses.

Performance Estimation Problem (PEP)—sometimes called “Performance Estimation Programming,” although the community more commonly uses “Problem”—is an optimization-based methodology for deriving tight worst-case performance guarantees for iterative algorithms over prescribed classes of functions. Its central question is finite-iteration and adversarial: what is the largest possible value of a chosen performance metric after a fixed number of algorithmic steps, over all admissible objective functions, initial conditions, and trajectories consistent with the algorithm? PEP answers that question by replacing the infinite-dimensional search over functions or operators with finite interpolation constraints, encoding all relevant inner products through a positive semidefinite Gram matrix, and solving a semidefinite program (SDP). When the interpolation constraints are necessary and sufficient, the resulting bound is tight and the optimizer provides a worst-case instance (Klerk et al., 2017, Rubbens et al., 2023).

1. Historical emergence and terminology

PEP emerged from the observation that the worst-case behavior of a black-box optimization method is itself an optimization problem. The 2012 formulation treated smooth unconstrained convex minimization over $\mathbb{R}^d$ with $L$-Lipschitz gradients, analyzed the gradient method through a convex semidefinite PEP, and extended the same perspective to a broader class of first-order methods that includes the heavy-ball method and fast gradient schemes (Drori et al., 2012). Later work presented PEP as a general methodology for smooth strongly convex optimization, composite problems, monotone inclusions, operator splitting, inexact gradient and Newton methods, and automated Lyapunov-based analyses (Klerk et al., 2017, Rubbens et al., 2023).

The terminological point is not merely stylistic. The software-oriented account in PEPit states that the approach was introduced as “Performance Estimation Problems” and that the community more commonly uses “Problem,” while “Programming” appears as an alternate label (Goujaud et al., 2022). A persistent misconception is therefore to treat “Performance Estimation Programming” as the canonical expansion of the acronym. The literature cited here suggests instead that “Programming” is secondary, whereas “Problem” is the standard historical name.

2. Canonical mathematical formulation

In its generic form, PEP takes as input an algorithm $A$ run for $K$ iterations, a function class $\mathcal{F}$, initial conditions, and a performance metric $\mathcal{P}$, and maximizes $\mathcal{P}$ over all admissible trajectories and all functions in $\mathcal{F}$ consistent with the algorithmic recursion (Rubbens et al., 2023). For centralized first-order methods on $L$-smooth convex functions $f:\mathbb{R}^d\to\mathbb{R}$, a standard worst-case suboptimality formulation is

$L$0

subject to interpolation inequalities for the function class, algorithmic relations linking iterates and gradients, initial conditions such as $L$1, and the Gram constraint $L$2 (Colla et al., 2022).

The Gram reduction is the structural core. One indexes the iterates and the minimizer by $L$3, collects gradients and iterates in a matrix

$L$4

and defines the Gram matrix $L$5. Every pairwise inner product between iterates, gradients, and auxiliary vectors becomes a linear expression in entries of $L$6, while function values remain as scalar decision variables. Algorithmic update rules such as gradient descent,

$L$7

initial conditions, interpolation inequalities, and performance objectives then become linear equalities, linear inequalities, or LMIs in $L$8, turning the original infinite-dimensional worst-case question into an SDP (Colla et al., 2022, Goujaud et al., 2022).

This finite-dimensionalization is exact only to the extent that the underlying interpolation system is exact. That point is decisive: PEP is not simply “SDP-based convergence analysis,” but an SDP whose admissible sample sets are meant to coincide with the traces of genuine functions or operators in the modeled class (Rubbens et al., 2023).

3. Interpolation constraints and modeled classes

The interpolation layer determines what PEP can certify without conservatism. For convex $L$9-smooth functions, one exact discrete interpolation condition is

$A$0

which is necessary and sufficient for interpolability by $A$1 and is tighter than discretizing convexity and Lipschitz-gradient inequalities separately (Rubbens et al., 2023). For $A$2-strongly convex and $A$3-smooth functions, exact necessary-and-sufficient interpolation constraints are also available, for example

$A$4

with a separate exact formula for the case $A$5 (Rubbens et al., 2023).

The same philosophy extends beyond smooth convex first-order settings. PEP formulations have been written for inexact steepest descent, fixed-step inexact gradient descent, and inexact Newton’s method on self-concordant functions. In the self-concordant case, one restricts iterates to a local intrinsic ball and reuses the smooth strongly convex interpolation system under a local inner product, with parameters

$A$6

thereby importing first-order interpolation machinery into short-step Newton analysis (Klerk et al., 2017).

Operator-theoretic and structured variants are also part of the PEP ecosystem. The interpolation survey covers monotone, cocoercive, and Lipschitz operators, exact interpolation for linear maps with bounded singular value, composite convex models, indicator functions, and decentralized consensus operators represented through semidefinite constraints (Rubbens et al., 2023). At the same time, the same survey stresses an important limitation: simply juxtaposing two interpolation constraints does not generally yield an interpolation constraint for the intersection class. This suggests that a substantial portion of current PEP research is not about solving SDPs per se, but about discovering new exact or tight interpolation systems.

4. Decentralized optimization and consensus modeling

A major extension of PEP concerns decentralized optimization, where $A$7 agents cooperatively minimize

$A$8

each agent holding a local variable and communicating through an averaging matrix $A$9. In this setting, $K$0 is typically symmetric and doubly stochastic,

$K$1

with the consensus projector $K$2 and disagreement projector $K$3 (Colla et al., 2022). The methodological challenge is to represent consensus steps in a Gram-compatible way.

Two principal PEP representations have been proposed. The exact formulation fixes a given $K$4 and directly encodes updates such as decentralized gradient descent,

$K$5

as linear equalities in the Gram variables. The relaxed spectral formulation avoids fixing $K$6 and instead assumes spectral bounds

$K$7

which lead to Gram-representable LMIs on centered consensus inputs and outputs. This formulation yields bounds valid uniformly over an entire class of averaging matrices and often allows reconstruction of a worst-case communication matrix a posteriori; in many reported experiments that worst case is

$K$8

with spectrum $K$9 (Colla et al., 2022).

A further development removes explicit dependence on the number of agents from the SDP size. The network-size independent formulation takes a global view of the decentralized problem, decomposes variables into the consensus subspace and its orthogonal complement, and builds separate Gram blocks $\mathcal{F}$0 and $\mathcal{F}$1 whose sizes depend on the iteration horizon rather than on the number of agents. Consensus steps are then represented spectrally on the orthogonal component, while optimality and performance criteria split additively across the two blocks (Colla et al., 2022). This formulation was applied to DGD, DIGing, and EXTRA.

Agent symmetries sharpen the same theme. When all agents are equivalent, the performance criterion is scale-invariant, initial conditions are symmetric, and consensus constraints are enforced through the relaxed operator interpolation, the worst-case value is independent of the number of agents and satisfies

$\mathcal{F}$2

The corresponding compact PEP depends only on symmetry classes of agents rather than on the total agent count, which makes worst-agent and percentile analyses feasible in settings where the full agent-dependent SDP would be impractical (Colla et al., 2024).

5. Automation, software, and computational workflow

PEP has developed a substantial software infrastructure. The PESTO toolbox automates PEP construction and solution, and decentralized extensions added blocks and examples for DGD, DIGing, and Acc-DNGD (Colla et al., 2022). PEPit provides a Python interface for computer-assisted worst-case analyses of first-order methods; it asks the practitioner to write the method nearly as it would be implemented, attaches function-class assumptions and initial conditions, builds the corresponding SDP, calls a solver, and returns the worst-case bound together with a dual certificate interpreted as a potential or Lyapunov function (Goujaud et al., 2022).

The standard workflow is explicit. One declares the function class, defines initial points and initial conditions, implements the update rules through primitive steps or direct formulas, sets a performance metric such as $\mathcal{F}$3, $\mathcal{F}$4, or $\mathcal{F}$5, and solves the resulting SDP. PEPit supports smooth convex and smooth strongly convex classes, convex Lipschitz and bounded-domain models, composite convex objectives, smooth nonconvex and quadratic problems, by-block smooth convex models, and operator classes including cocoercive, monotone, maximally monotone, Lipschitz, and nonexpansive operators (Goujaud et al., 2022).

Automation does not remove the modeling burden entirely. The practical size of the SDP still grows with the number of sampled points, auxiliary variables, and iterations. The decentralized literature therefore emphasizes structure exploitation—average/centered decompositions, consensus constraints only on centered blocks, symmetry reduction, and one-step rate PEPs for infinite-horizon certification—while software accounts recommend minimizing the number of leaf points and reusing variables whenever possible (Colla et al., 2022, Goujaud et al., 2022).

6. Method design, certificates, and limitations

PEP is not restricted to post hoc analysis of a fixed algorithm. Branch-and-Bound Performance Estimation Programming (BnB-PEP) formulates the search for an optimal first-order method as an exact quadratically constrained quadratic program, solves it to certifiable global optimality with a customized spatial branch-and-bound algorithm, and thereby expands automated method design beyond the convexified outer problems used in earlier approaches (Gupta et al., 2022). In this setting, the SDP-based inner PEP is embedded inside a nonconvex outer design problem over step sizes or momentum parameters.

The dual side of PEP is as important as the primal side. Dual variables attached to interpolation, method, and initialization constraints assemble into a potential or Lyapunov function whose one-step decrease proves the certified bound. PEPit reconstructs and prints this certificate, and BnB-PEP uses the same mechanism to translate numerical optimality patterns—such as sparse interpolation weights and low-rank slack matrices—into analytical potential-function proofs (Goujaud et al., 2022, Gupta et al., 2022). This suggests that PEP is simultaneously a worst-case estimator, a proof generator, and, in some settings, a method-construction engine.

Several limitations remain explicit in the literature. SDP size grows with the iteration count and the number of queried points, so multi-step PEPs are practical mainly for modest horizons unless special structure is exploited (Goujaud et al., 2022, Klerk et al., 2017). Exact interpolation systems are still missing for many interesting classes, including blockwise smooth, Hölder smooth, PL-condition, and mixed first-/second-order descriptors, and principled rules for combining interpolation systems remain open (Rubbens et al., 2023). In decentralized optimization, spectral consensus models may be relaxations rather than exact characterizations of admissible averaging matrices; exactness is then assessed a posteriori by recovering a compatible $\mathcal{F}$6 or by matching the relaxed bound with a fixed-$\mathcal{F}$7 PEP value (Colla et al., 2022). These caveats do not diminish the framework’s reach, but they delimit where PEP currently delivers exact certificates and where it yields tight upper bounds instead.