Performance Estimation Problem (PEP) in Optimization

• Performance Estimation Problem (PEP) is a worst-case analysis framework for first-order methods that reformulates performance estimation as an optimization problem over all admissible functions and iterates.
• It leverages interpolation constraints and Gram matrix techniques to reduce infinite-dimensional worst-case problems to finite semidefinite programs, yielding tight numerical bounds.
• The framework extends to decentralized and data-driven settings, enabling automated algorithm design and optimal parameter tuning in various optimization scenarios.

Searching arXiv for recent and foundational papers on the Performance Estimation Problem in optimization. {"query":"Performance Estimation Problem first-order optimization PEPit interpolation constraints decentralized optimization", "max_results": 10} I found several directly relevant arXiv papers, including foundational and recent work on PEP for first-order and decentralized optimization, interpolation constraints, and software support. The Performance Estimation Problem (PEP) is a worst-case analysis framework for first-order optimization methods in which the performance of an algorithm is itself posed as an optimization problem over all admissible functions, iterates, oracle responses, and initial conditions consistent with the method and the assumed problem class. In the formulation introduced for smooth unconstrained convex minimization, one maximizes a final error criterion such as $f(x_N)-f(x_\ast)$ over all trajectories generated by a first-order black-box method; in later work, this viewpoint is expressed more broadly as the search for the worst-case value of a chosen performance metric over all admissible problems and initial conditions, followed by a finite-dimensional semidefinite reformulation that can be solved numerically (Drori et al., 2012, Goujaud et al., 2022).

1. Core formulation of the worst-case problem

A PEP is specified by four ingredients: the algorithm being analyzed, the problem class or assumptions on the objective, a performance measure, and an initial condition (Goujaud et al., 2022). In the classical smooth convex setting, the underlying optimization problem is

$(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$

with $f$ convex, $f\in C^{1,1}_L(\mathbb{R}^d)$, $X_\ast(f)=\Argmin f$ nonempty, and $\|x_0-x_\ast\|\le R$ for some minimizer $x_\ast$ (Drori et al., 2012). The algorithm is a first-order black-box method: it only sees function values and gradients queried sequentially at points it generates (Drori et al., 2012).

The defining PEP construction replaces direct proof by an optimization problem of the form

$\max \ f(x_N)-f(x_\ast)$

subject to the method updates, the function-class assumptions, optimality of $x_\ast$, and the initial condition (Drori et al., 2012). In the more general modern presentation, the performance metric need not be a final function gap. Typical choices include distance $\|x_n-x_\star\|_2^2$, gradient norm $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$0, function gap $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$1, contraction $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$2, and Lyapunov functions (Goujaud et al., 2022).

A standard illustrative case is gradient descent on an $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$3-smooth, $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$4-strongly convex function. The contraction factor $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$5 is defined by

$(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$6

and the corresponding PEP is

$(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$7

subject to the gradient-descent updates, the initial condition $(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$8, and the assumption that the function belongs to the chosen class (Goujaud et al., 2022). Conceptually, this means that a PEP is a worst-case optimization problem over all functions, iterates, and oracle responses consistent with the method and assumptions (Goujaud et al., 2022).

2. Finite-dimensional reduction through interpolation and Gram matrices

The central technical obstacle is that the original worst-case problem is infinite-dimensional because it ranges over functions. The standard reduction replaces the unknown function by finitely many samples

$(M)\qquad \min_{x\in \mathbb{R}^d} f(x),$9

taken at the points queried by the algorithm, where $f$0 and $f$1 (Goujaud et al., 2022). The key question then becomes whether a finite dataset is interpolable by a function in the target class.

For $f$2-smooth convex functions, the exact interpolation condition reviewed in the interpolation survey is

$f$3

Any finite dataset satisfying this condition is interpolable by some $f$4 (Rubbens et al., 2023). For $f$5, exact interpolation with values is given by

$f$6

when $f$7 (Rubbens et al., 2023). In the PEPit presentation, the interpolation theorem for smooth strongly convex functions is equivalently written as

$f$8

for all $f$9 (Goujaud et al., 2022).

This interpolation viewpoint matters because equivalent infinite-dimensional characterizations need not remain equivalent after discretization. The interpolation survey emphasizes that some discretized constraints are only necessary, whereas interpolation constraints are exact for the finite dataset and therefore reduce conservatism (Rubbens et al., 2023).

After interpolation, the remaining variables appear through inner products, so one introduces a Gram matrix. In the gradient-descent contraction example, the vectors $f\in C^{1,1}_L(\mathbb{R}^d)$0, $f\in C^{1,1}_L(\mathbb{R}^d)$1, and $f\in C^{1,1}_L(\mathbb{R}^d)$2 are encoded by

$f\in C^{1,1}_L(\mathbb{R}^d)$3

This removes the explicit dependence on the ambient dimension $f\in C^{1,1}_L(\mathbb{R}^d)$4 and yields a finite SDP (Goujaud et al., 2022).

3. Semidefinite programming formulation and representative guarantees

Once interpolation and algorithmic relations are expressed in Gram variables, the PEP takes a generic SDP form. PEPit gives the template

$f\in C^{1,1}_L(\mathbb{R}^d)$5

Here $f\in C^{1,1}_L(\mathbb{R}^d)$6 is the Gram matrix of points and gradients, $f\in C^{1,1}_L(\mathbb{R}^d)$7 collects function values or other scalar quantities, and the constraints encode initial conditions, interpolation conditions, and any extra semidefinite constraints (Goujaud et al., 2022).

For the one-step gradient-descent contraction example, the objective becomes

$f\in C^{1,1}_L(\mathbb{R}^d)$8

and the resulting worst-case guarantee is

$f\in C^{1,1}_L(\mathbb{R}^d)$9

In the numerical example with $X_\ast(f)=\Argmin f$0, $X_\ast(f)=\Argmin f$1, $X_\ast(f)=\Argmin f$2, PEPit returns about $X_\ast(f)=\Argmin f$3, and for two iterations the reported bound is around $X_\ast(f)=\Argmin f$4, matching the predicted contraction (Goujaud et al., 2022).

The 2012 paper established a complementary foundational result for the classical gradient method with constant step size

$X_\ast(f)=\Argmin f$5

Its main analytical bound is

$X_\ast(f)=\Argmin f$6

which improves the classical $X_\ast(f)=\Argmin f$7 estimate when $X_\ast(f)=\Argmin f$8 (Drori et al., 2012). The same paper also constructs explicit worst-case functions showing that

$X_\ast(f)=\Argmin f$9

and conjectures the exact worst-case bound

$\|x_0-x_\ast\|\le R$0

for $\|x_0-x_\ast\|\le R$1 (Drori et al., 2012).

Beyond gradient descent, the same SDP machinery yields numerical bounds for heavy-ball and fast gradient methods, and later work shows that PEPit reproduces or tightens known bounds for Nesterov’s accelerated method, accelerated Douglas–Rachford splitting, and point-SAGA (Drori et al., 2012, Goujaud et al., 2022).

4. Oracle modeling, automation, and programmable workflows

A major development in the PEP literature is the translation of worst-case analysis into a programmable workflow. PEPit is a Python package built around the performance estimation problem viewpoint; the user writes first-order methods nearly as they would have implemented them, while the package automatically generates interpolation constraints, builds the Gram-matrix SDP, sends it to CVXPY/MOSEK, and reconstructs primal/dual certificates (Goujaud et al., 2022).

This automation is possible because many first-order schemes can be written in a small set of oracle-based primitives. A gradient step is modeled as

$\|x_0-x_\ast\|\le R$2

and a subgradient step uses $\|x_0-x_\ast\|\le R$3 (Goujaud et al., 2022). The proximal map is

$\|x_0-x_\ast\|\le R$4

which can be equivalently encoded by

$\|x_0-x_\ast\|\le R$5

(Goujaud et al., 2022). PEPit also supports inexact proximal steps, linear optimization oracles of Frank–Wolfe or conditional-gradient type,

$\|x_0-x_\ast\|\le R$6

approximate gradient and proximal updates such as

$\|x_0-x_\ast\|\le R$7

and mirror or Bregman variants including

$\|x_0-x_\ast\|\le R$8

and

$\|x_0-x_\ast\|\le R$9

(Goujaud et al., 2022).

The practical significance is that the user does not need to manually write SDP matrices. This bridges the gap between theory-heavy worst-case analysis and a reproducible, programmable workflow for studying and even discovering tight convergence guarantees (Goujaud et al., 2022).

5. Decentralized, network-size-independent, and symmetry-reduced PEPs

Classical PEP was developed for centralized first-order methods, but later work extends it to decentralized optimization by encoding consensus steps in Gram-representable form. One approach introduces two representations of consensus: an exact formulation for a fixed averaging matrix $x_\ast$0, and a relaxed spectral formulation valid over a class of real, symmetric, generalized doubly stochastic matrices with eigenvalues in a prescribed interval (Colla et al., 2022). For centered variables $x_\ast$1 and $x_\ast$2, the spectral formulation uses

$x_\ast$3

together with

$x_\ast$4

(Colla et al., 2022). This yields exact worst-case bounds for a specific network and spectral worst-case bounds valid over an entire class of networks (Colla et al., 2022).

A later development reformulates decentralized PEP in a basis that decouples consensus and disagreement. For

$x_\ast$5

the transformed variable is split into consensus and orthogonal components,

$x_\ast$6

and a consensus step becomes

$x_\ast$7

so consensus leaves $x_\ast$8 unchanged and only acts on $x_\ast$9 (Colla et al., 2022). The resulting SDP size depends on the number of iterations $\max \ f(x_N)-f(x_\ast)$0, the function class, and the spectral bounds of the network, but not on the number of agents $\max \ f(x_N)-f(x_\ast)$1 (Colla et al., 2022).

A further symmetry-based reduction shows that, in many settings, the worst-case performance of a distributed optimization algorithm is independent of the number of agents and can be computed in the fundamental case with two agents (Colla et al., 2024). When all agents are equivalent and the objective and constraints are scale-invariant, the paper states

$\max \ f(x_N)-f(x_\ast)$2

(Colla et al., 2024). The same framework also handles multiple equivalence classes of agents, worst-agent metrics such as

$\max \ f(x_N)-f(x_\ast)$3

and percentile-style criteria (Colla et al., 2024).

6. PEP as an algorithm-design and data-driven framework

PEP is not only an analysis device. Already in the 2012 formulation, the optimization over step sizes is treated as a design problem, and the paper gives an efficient procedure for finding optimal step sizes that results in a first-order black-box method that achieves best performance (Drori et al., 2012). More recent work makes this perspective explicit by minimizing the worst-case SDP bound over algorithm parameters such as the coefficients $\max \ f(x_N)-f(x_\ast)$4 in

$\max \ f(x_N)-f(x_\ast)$5

(Kamri et al., 28 Jul 2025). In that setting, algorithm design becomes a min–max problem,

$\max \ f(x_N)-f(x_\ast)$6

with the inner problem represented by an SDP (Kamri et al., 28 Jul 2025). The same paper reports optimized schedules for memoryless gradient descent, cyclic coordinate descent, inexact gradient descent, and cyclic gradient descent under linear convergence, with accelerated convergence rates compared to classical algorithms (Kamri et al., 28 Jul 2025).

When the outer design problem is nonconvex, a different methodology, Branch-and-Bound Performance Estimation Programming, models it exactly as a quadratically constrained quadratic program and solves it to certifiable global optimality using a customized spatial branch-and-bound solver (Gupta et al., 2022). This extends PEP-based design to smooth nonconvex gradient reduction, weakly convex nonsmooth problems, and proof search through potential-function structures (Gupta et al., 2022).

Recent papers also combine PEP with Wasserstein distributionally robust optimization. One direction minimizes a Wasserstein distributionally robust version of the PEP over algorithm parameters, yielding a framework that interpolates between classical learning to optimize and worst-case optimal algorithm design: $\max \ f(x_N)-f(x_\ast)$7

$\max \ f(x_N)-f(x_\ast)$8

(Ranjan et al., 7 May 2026). Another direction uses sampled trajectories to derive high-confidence bounds on expectation, $\max \ f(x_N)-f(x_\ast)$9, or worst-case performance by combining exact lifted PEP representations with Wasserstein ambiguity sets around the empirical distribution of lifted variables $x_\ast$0 (Park et al., 21 Nov 2025). This suggests a broader role for PEP as a bridge between worst-case analysis, average-case information, and probabilistic robustness.

7. Exactness, limitations, and terminological scope

The strength of a PEP formulation depends on the quality of the interpolation constraints. When interpolation constraints are necessary and sufficient, the resulting SDP can be exact; when they are only necessary, the SDP remains a valid relaxation but need not be exact (Rubbens et al., 2023). The interpolation survey identifies several limitations: interpolation constraints are not known for many classes, combining interpolation constraints is nontrivial, some classes such as network matrices currently have only necessary constraints, and approximate interpolation is not fully understood (Rubbens et al., 2023). In decentralized optimization, the exact formulation for a fixed averaging matrix and the spectral formulation over a class of matrices differ precisely in this sense: the latter is a relaxation, although it is often numerically tight (Colla et al., 2022).

A common misconception is to treat PEP as synonymous with a single SDP recipe for centralized smooth convex analysis. The literature summarized here indicates a broader picture: PEP covers smooth, strongly convex, nonconvex, proximal, Bregman, inexact, decentralized, and data-driven settings, but exactness and tractability depend on whether the function class, oracle model, communication model, and performance metric are linearly or LMI Gram-representable (Goujaud et al., 2022, Rubbens et al., 2023).

The acronym is also overloaded across research areas. In automatic algorithm configuration, “performance estimation problem” denotes a statistical estimation problem: given finite training data and a limited run budget, one asks how to estimate the true expected performance $x_\ast$1 of a configuration and how large the gap $x_\ast$2 can be (Liu et al., 2019). In that setting, the paper proves that the universal best estimator distributes the $x_\ast$3 runs as evenly as possible across the $x_\ast$4 training instances (Liu et al., 2019). In wireless communications, by contrast, PEP commonly denotes pairwise error probability, as in the analysis of NOMA systems with imperfect successive interference cancellation (Bariah et al., 2018). In the optimization literature, however, the term refers to the worst-case estimation framework described above: a reformulation of performance analysis as an optimization problem, typically reduced by interpolation and Gram matrices to semidefinite programming.