---
title: Performance Estimation Framework (PEP)
url: https://www.emergentmind.com/topics/performance-estimation-framework-pep
type: topic
---

# Performance Estimation Framework (PEP)

Searching arXiv for recent and foundational papers on the Performance Estimation Problem framework and related extensions.
The Performance Estimation Framework, usually centered on the Performance Estimation Problem (PEP), is a computer-assisted methodology for computing tight worst-case bounds for iterative optimization algorithms by formulating the analysis itself as an optimization problem. In its standard form, PEP maximizes a chosen performance criterion over all admissible initial conditions and all problem instances in a prescribed class, while replacing infinite-dimensional objects such as functions, operators, or networks by finite samples constrained through interpolation conditions. The resulting finite-dimensional model is typically a semidefinite program (SDP) in a Gram matrix of inner products and a set of scalar function values, and its solution yields both a worst-case bound and an extremal instance [1206.3209][2309.12173][2201.04040].

## 1. Historical formulation and conceptual scope

PEP was introduced from the observation that the worst-case behavior of a first-order black-box method is itself an optimization problem [1206.3209]. In the original smooth convex setting, the objective was to analyze methods for unconstrained minimization over \( \mathbb{R}^d \), under convexity and \(L\)-smoothness, with a first-order oracle returning \(f(x_k)\) and \(\nabla f(x_k)\) at each iterate [1206.3209]. The framework subsequently expanded into a general methodology for fixed-step first-order methods, composite models, proximal and projection oracles, linear minimization oracles, Bregman variants, inexact oracles, and decentralized optimization [2201.04040][2203.05963].

A standard distinction in the literature is between the inner worst-case analysis problem and the outer algorithm-design problem. For a fixed method, PEP computes worst-case performance over all functions and trajectories consistent with the update rules and the class constraints [2203.05963][2201.04040]. In later developments, the outer problem of choosing the method’s parameters is itself optimized. The paper "Branch-and-Bound Performance Estimation Programming: A Unified Methodology for Constructing Optimal Optimization Methods" formulates this joint design-and-analysis task as a nonconvex quadratically constrained quadratic program and solves it to certifiable global optimality with customized branch-and-bound [2203.07305].

The framework is therefore both an analysis formalism and a design formalism. This suggests why the expression “Performance Estimation Framework” is often more informative than the narrower phrase “Performance Estimation Problem”: the same core machinery supports exact worst-case certification, automated parameter tuning, and the extraction of analytical proofs from numerical certificates [2203.07305][2201.04040].

## 2. Generic SDP formulation

For a method \(M\) run for \(K\) steps on a function class \(\mathcal F\) with an initial set \(I^0\), the worst case is written as a finite-sample optimization problem over iterates, gradients, function values, and a minimizer. One generic form is
\[
\sup_{f\in\mathcal F,\;x^0,\dots,x^K,\;x^*\in\arg\min f} \;\mathcal P(f,x^0,\dots,x^K,x^*)
\]
subject to function-class interpolation constraints, algorithmic recursion constraints, and initial constraints such as \(\|x^0-x^*\|^2\le R^2\) [2203.05963]. In decentralized formulations the same structure appears with agent-indexed variables and additional optimality and consensus constraints [2403.11724].

The key modeling device is the Gram matrix. If \(P\) collects all vectors that appear in the description—iterates, gradients, auxiliary points, directions, operator outputs—then
\[
G=P^\top P \succeq 0
\]
encodes all relevant inner products [2403.11724][2201.04040]. Function values are stored in a separate vector. Method constraints, initialization constraints, interpolation inequalities, and many performance measures then become linear or linear-matrix-inequality constraints in the entries of \(G\) and the scalar function values [2201.04040][2309.12173].

This SDP representation is dimension-independent in the standard sense used in the literature: feasibility depends only on the Gram matrix, not on an explicit ambient dimension. One formulation states that the construction is lossless when one does not fix the dimension \(d\) and imposes no rank constraint on \(G\) [2203.05963]. Another states that under the large-scale assumption \(d\ge N+2\), rank constraints on \(G\) can be dropped without relaxation [2203.07305]. These statements clarify an important point: tightness is not a generic consequence of semidefinite relaxation alone, but of exact interpolation and the absence of restrictive rank constraints.

## 3. Interpolation theory and exact class descriptions

Interpolation constraints are the central mechanism by which PEP replaces infinite-dimensional function or operator classes with finite inequalities on sampled data [2309.12173]. For \(L\)-smooth convex functions, a sampled set \(\{(x_i,g_i,f_i)\}\) is interpolable if the pairwise inequalities
\[
f_i \ge f_j + \langle g_j, x_i - x_j\rangle + \frac{1}{2L}\|g_i-g_j\|^2
\]
hold for all \(i,j\) [2201.04040][2309.12173]. Equivalent formulations include the descent lemma
\[
f(y)\le f(x)+\langle \nabla f(x),y-x\rangle+\frac{L}{2}\|y-x\|^2
\]
and Baillon–Haddad cocoercivity,
\[
\langle \nabla f(x)-\nabla f(y),x-y\rangle \ge \frac{1}{L}\|\nabla f(x)-\nabla f(y)\|^2,
\]
both of which are routinely embedded in PEP models [2403.11724][2201.04040].

For \(\mu\)-strongly convex and \(L\)-smooth functions, the exact pairwise interpolation condition can be written as
\[
\begin{aligned}
f_i \ge f_j + \langle g_j,x_i-x_j\rangle
&+ \frac{\|g_i-g_j\|^2}{2(L-\mu)}
+ \frac{\mu L}{2(L-\mu)}\|x_i-x_j\|^2 \\
&- \frac{\mu}{L-\mu}\langle g_i-g_j,x_i-x_j\rangle ,
\end{aligned}
\]
for \(\mu\neq L\) [2309.12173]. Closely related equivalent forms are used internally in software such as PEPit [2201.04040].

The same interpolation viewpoint extends beyond smooth convex classes. The surveyed literature includes strong convexity alone, smooth functions with bounded subgradient, indicator functions of convex sets, monotone operators, cocoercive operators, Lipschitz operators, linear operators with bounded singular value, resolvents, proximal mappings, and consensus operators in decentralized optimization [2309.12173]. Composite models such as \(h(Mx)\) are handled by introducing auxiliary samples \(y_i=Mx_i\), \(u_i=\nabla h(y_i)\), \(v_i=M^\top u_i\), and combining function interpolation with linear-operator interpolation constraints such as
\[
X^\top V = Y^\top U,\qquad Y^\top Y \preceq L^2 X^\top X,\qquad V^\top V \preceq L^2 U^\top U
\]
for \(M\in\mathcal L_L\) [2309.12173].

A recurrent source of confusion is the role of relaxation. The literature emphasizes that using tight interpolable class descriptions is crucial, because weaker necessary conditions can significantly degrade the bounds and mislead parameter selection [2309.12173]. In standard smooth convex and smooth strongly convex settings, the interpolation theorems used in PEP are exact necessary-and-sufficient conditions, which is why the resulting SDP can be tight [2201.04040][2309.12173].

## 4. Performance criteria, algorithm classes, and representative bounds

PEP supports a broad range of performance measures. Canonical examples include final suboptimality \(f(x_N)-f_*\), distance to the solution \(\|x_N-x_*\|^2\), gradient norm \(\|\nabla f(x_N)\|^2\), ergodic criteria, Frank–Wolfe dual gaps, and, in decentralized settings, consensus error and worst-agent metrics [2201.04040][2403.11724]. Because these criteria are linear or convex quadratic in Gram entries and scalar values, they remain compatible with the SDP framework [2203.05963][2403.11724].

The original smooth convex analysis already showed that PEP can sharpen classical constants. For the gradient method with fixed normalized step size \(h\), one tight analytical bound is
\[
f(x_N)-f(x_*) \le \frac{LR^2}{4Nh+2}, \qquad 0<h\le 1,
\]
and explicit worst-case functions attain the bound [1206.3209]. The same work developed semidefinite relaxations for a broader first-order black-box class including the heavy-ball method and fast gradient schemes, and used them to produce new numerical bounds and optimal step-size procedures [1206.3209].

Later formulations generalized this repertoire. PEPit covers gradient, projection, proximal, linear optimization, approximate, and Bregman oracles, and models methods such as Gradient Descent, Projected and Proximal Gradient, Douglas–Rachford and ADMM variants, Frank–Wolfe, Mirror Descent, composite models, and stochastic or randomized examples via expected metrics [2201.04040]. The interpolation-centered survey likewise emphasizes accelerated methods, heavy-ball and momentum variants, primal–dual and splitting methods, coordinate descent, and decentralized algorithms [2309.12173].

A notable later development is the use of PEP for algorithm design. BnB-PEP poses the problem of finding the optimal method as a nonconvex QCQP, replaces the PSD proof matrix \(Z\succeq 0\) by a Cholesky factorization \(PP^\top=Z\), and solves the resulting design-and-analysis problem with a customized three-stage branch-and-bound algorithm [2203.07305]. The method is applied to smooth strongly convex gradient reduction, smooth convex function decrease without momentum, smooth nonconvex gradient reduction, and weakly convex nonsmooth analysis via the Moreau envelope [2203.07305]. An important consequence is that inner-dual solutions can become human-readable proofs: the paper explicitly uses BnB-PEP to find proofs with potential function structures and thereby systematically generate analytical convergence proofs [2203.07305].

## 5. Software, proof extraction, and practical computation

The software ecosystem around PEP aims to make worst-case analysis close to the notation of an algorithmic paper. PEPit is a Python package in which one declares the function class, specifies initial conditions, writes the method nearly as it would be implemented, sets the performance metric, and lets the package compile the SDP and call a numerical solver [2201.04040]. The package follows the Taylor–Hendrickx–Glineur interpolation framework and supports CVXPY with SCS or MOSEK, as well as a direct MOSEK interface [2201.04040].

A characteristic practical feature of PEP is that both primal and dual solutions are informative. The primal Gram matrix yields near-worst-case witnesses, and a Cholesky factorization reconstructs worst-case iterates and gradients [2203.05963][2403.11724]. The dual variables provide a certifiable proof and can often be translated into an analytic argument by taking appropriate linear combinations of interpolation inequalities [2201.04040]. This dual-to-proof interpretation is central in BnB-PEP, where sparse multipliers and low-rank proof matrices are exploited algorithmically and then simplified into analytical potential-function proofs [2203.07305].

Computational burden remains an essential limitation. SDP size grows with the number of sampled vectors, which typically grows with the iteration horizon and with the number of auxiliary variables introduced by composite or distributed models [2203.05963][2309.12173]. The literature therefore emphasizes normalization, structure exploitation, symmetry reduction, and small proof systems. In the decentralized context, one paper notes that for \(N=3\) and DGD spectral PEP, solve times were approximately \(3,12,48,192\) seconds for \(K=5,10,15,20\) on a laptop [2203.05963]. In another direction, the customized BnB-PEP solver reports runtime reductions from hours to seconds and weeks to minutes relative to off-the-shelf spatial branch-and-bound on selected global-design tasks [2203.07305].

## 6. Decentralized optimization, symmetry reductions, and network-size independence

A major recent extension of PEP concerns decentralized optimization, where \(N\) agents minimize
\[
F(x)=\frac{1}{N}\sum_{i=1}^N f_i(x)
\]
using local gradients and communication through a mixing matrix \(W\) [2203.05963][2210.00695]. Early decentralized PEP formulations modeled each agent separately and introduced either an exact matrix-specific encoding \(Y^k=WX^k\) or a spectral-range-based LMI relaxation valid over an entire class of symmetric generalized doubly stochastic matrices with prescribed eigenvalue interval [2203.05963].

The spectral formulation represents consensus steps through average preservation and LMIs on centered components. In one common symmetric spectral case, the centered outputs satisfy
\[
Y_c^\top Y_c \preceq \lambda^2 X_c^\top X_c,
\]
which implies that each consensus step reduces the disagreement by at least a factor \(\lambda^2\) in the worst case [2203.05963]. This decoupling led to a network-size-independent formulation in which the consensus mode and the orthogonal disagreement mode are separated, and the SDP uses two Gram matrices \(G_\parallel\) and \(G_\perp\) whose sizes depend on the horizon but not on the number of agents [2210.00695].

The symmetry-based theory of agent equivalence pushes this further. In the fully symmetric setting, if all agents are equivalent, the algorithm, interpolation, and initialization constraints are Gram-representable, and the performance measure together with any non-single-agent initialization constraints are scale-invariant, then the worst-case value \(w(\mathcal S_n)\) is computable from a compact SDP whose size is independent of \(n\), and under additional scale-invariance conditions
\[
w(\mathcal S_n)=w(\mathcal S_2)\qquad \text{for all } n\ge 2
\]
[2403.11724]. The proof uses a symmetrized Gram matrix with repeating diagonal and off-diagonal blocks, together with the reduction
\[
\Gamma_t\succeq 0,\qquad \Gamma_a-\Gamma_t\succeq 0,
\]
which replaces an \((np)\times(np)\) PSD constraint by two small LMIs [2403.11724].

These formulations have been applied to DGD, DIGing, EXTRA, worst-agent criteria, and percentile criteria [2203.05963][2210.00695][2403.11724]. In the EXTRA study, scale-invariant metrics such as objective error at the average iterate and mean iterate error are independent of the number of agents and equal to the two-agent bound, while worst-agent performance is not scale-invariant and requires a multi-class compact PEP [2403.11724]. The same work also states that tight Gram-representable constraints for the exact \(W\otimes I_d\) class are impossible, and therefore justifies a convex relaxation based on invariant-subspace interpolation constraints for a larger symmetric class \(\mathcal M_{[\lambda_-,\lambda_+]}^\mathcal C\) [2403.11724]. This is an important qualification: decentralized PEP can be exact for a fixed mixing matrix, or spectral and uniform over a class of matrices, but these are distinct notions of tightness [2203.05963][2403.11724].

The modern PEP literature thus presents a unified picture. Worst-case analysis is posed as optimization over sampled information; interpolation constraints turn functional, operator, and network classes into finite constraints; Gram representations make the problems dimension-independent; and SDP or QCQP machinery yields tight numerical bounds, worst-case instances, automated tuning rules, and, in many cases, analytical convergence proofs [1206.3209][2309.12173][2203.07305][2403.11724].

Source: https://www.emergentmind.com/topics/performance-estimation-framework-pep