Proximal Bundle Method: Theory & Extensions

• Proximal bundle methods are convex optimization techniques that build a piecewise-linear under-estimator using historical function values and subgradients, enhanced with quadratic regularization.
• They stabilize traditional cutting-plane methods by integrating a proximal regularization step and using serious/null-step logic to update the solution center efficiently.
• Recent advances extend these methods to stochastic, decentralized, and smooth settings, offering accelerated convergence rates and adaptable complexity bounds.

The proximal bundle method is a class of algorithms for convex optimization, especially nonsmooth convex optimization, that combines two ideas: proximal regularization, as in the proximal point method, and bundle modeling, as in cutting-plane methods. In its standard form, the method builds a piecewise-linear convex under-estimator of the objective from past function values and subgradients, then minimizes that model plus a quadratic proximal term around a stability center. This stabilizes the otherwise erratic behavior of pure cutting-plane schemes and yields a tractable convex subproblem, typically a small quadratic program (Ouorou, 2020). In recent work, the same proximal-bundle template has been analyzed for semi-smooth and composite Hölder settings, connected explicitly to primal-dual and inexact proximal-point frameworks, accelerated in smooth regimes, and adapted to stochastic, decentralized, saddle-point, and application-specific structured optimization problems (Liang et al., 2024).

1. Definition and basic mathematical structure

A standard convex optimization problem considered by proximal bundle methods is

$\min_{x\in\mathbb{R}^n} f(x),$

with $f$ convex and possibly nonsmooth (Ouorou, 2020). In composite variants, the objective is written as

$\phi(x)=f(x)+h(x),$

with $f,h$ proper, lower semicontinuous, and convex, and with $h$ often handled exactly through a proximal term or proximal mapping (Guigues et al., 2024, Liang, 2024). The method is especially natural when subgradients of $f$ are available but direct minimization of $f$ or exact proximal evaluations are difficult.

The defining object is a bundle model assembled from past evaluations. Given points $z^i$ and subgradients $g^i\in\partial f(z^i)$, a classical model is

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$

which satisfies $f$0 for all $f$1 (Ouorou, 2020). The same structure appears in more recent formulations. For example, in the semi-smooth analysis of proximal oracles, the bundle model is

$f$2

with monotonicity

$f$3

(Liang et al., 2024).

The proximal regularization step replaces direct minimization of the model by a strongly convex subproblem. In its basic form,

$f$4

or, equivalently, using a penalty parameter $f$5,

$f$6

(Ouorou, 2020). In proximal-point notation, the exact proximal map is

$f$7

and one may view proximal bundle methods as practical approximations of this map obtained by replacing $f$8 with a bundle model (Liang et al., 2024).

This viewpoint extends beyond plain nonsmooth convex minimization. The same proximal-bundle structure has been used for convex composite optimization (Guigues et al., 2024), convex-concave composite saddle-point problems (Liang, 2024), stochastic convex composite problems with expectation objectives (Liang et al., 2022), and smooth convex problems, where bundle models are no longer necessary for well-posedness but remain useful for richer local approximation and robustness (Zheng et al., 1 Apr 2026, Liao et al., 4 Dec 2025).

2. Bundle models, proximal subproblems, and serious/null-step logic

The classical mechanism of a proximal bundle method has three ingredients: a bundle of cuts, a proximal center, and a step-acceptance test. The bundle stores historical information $f$9, and the model $\phi(x)=f(x)+h(x),$0 is the maximum of the associated affine minorants (Ouorou, 2020). The proximal center $\phi(x)=f(x)+h(x),$1 or $\phi(x)=f(x)+h(x),$2 is the reference point around which the quadratic stabilization is built. The subproblem is then strongly convex and admits a unique minimizer under standard assumptions.

A central practical distinction is between serious steps and null steps. In the standard description, the method computes a candidate point from the proximal subproblem and then compares the actual decrease in the true objective to the predicted decrease from the model. If the decrease is sufficient, the candidate becomes the next center; otherwise the center is retained and the new point is used only to enrich the bundle. In one formulation, a descent test is written as

$\phi(x)=f(x)+h(x),$3

with descent steps updating the center and null steps only updating the model (Liao et al., 4 Dec 2025). This logic is the standard stabilizing mechanism of proximal bundle methods, and it distinguishes them from pure cutting-plane methods such as Kelley’s method, which minimize the model without proximal stabilization and without serious/null-step management (Fersztand et al., 2024).

A closely related stopping quantity is the model gap. In the regularized cutting-plane scheme used as an inner proximal solver, one defines

$\phi(x)=f(x)+h(x),$4

where

$\phi(x)=f(x)+h(x),$5

This gap satisfies

$\phi(x)=f(x)+h(x),$6

and if $\phi(x)=f(x)+h(x),$7, then $\phi(x)=f(x)+h(x),$8 is a $\phi(x)=f(x)+h(x),$9-approximate solution of the exact proximal subproblem (Liang et al., 2024). A similar primal-dual interpretation appears in the 2024 primal-dual bundle framework, where

$f,h$0

is shown to upper bound a primal-dual gap for the proximal subproblem (Liang, 2024).

A common misconception is that proximal bundle methods are synonymous with full-memory multi-cut models. The recent literature makes clear that one-cut, two-cut, and multi-cut constructions all fit a common bundle-management framework (Guigues et al., 2024, Liang, 2024). In particular, one-cut and two-cut variants can preserve the proximal-bundle structure while drastically reducing subproblem complexity (Guigues et al., 2024).

3. Proximal-point, Moreau–Yosida, and primal-dual interpretations

A fundamental interpretation is that proximal bundle methods are inexact proximal point methods. The exact proximal point algorithm iterates

$f,h$1

but these subproblems are usually as hard as the original problem (Ouorou, 2020). Proximal bundle methods replace the exact objective in the prox subproblem by a tractable lower model while preserving enough structure to guarantee progress.

This perspective is especially transparent through the Moreau–Yosida regularization

$f,h$2

whose minimizer is

$f,h$3

The regularization $f,h$4 is finite-valued, convex, and differentiable on $f,h$5, with globally Lipschitz gradient

$f,h$6

and

$f,h$7

(Ouorou, 2020). This identifies the proximal point algorithm as gradient descent on a smooth surrogate of the original nonsmooth objective. In that setting, bundle methods become implementable approximations of the proximal map and hence inexact first-order oracles for $f,h$8 (Ouorou, 2020).

The primal-dual interpretation has been made explicit in recent work. For the convex composite problem $f,h$9, the proximal subproblem

$h$0

has Fenchel dual

$h$1

(Liang, 2024). The inner bundle gap $h$2 in the primal-dual cutting-plane scheme satisfies

$h$3

so the stopping criterion is directly a primal-dual certificate (Liang, 2024).

That same paper establishes a duality between the conditional gradient method and the cutting-plane scheme used in the proximal bundle method. Under a one-cut update, if $h$4 denotes the dual conditional-gradient iterate and $h$5 the primal bundle iterate, then

$h$6

where $h$7 is the conditional-gradient search point (Liang, 2024). This identifies the bundle model update on the primal side with a Frank–Wolfe-type linearization process on the dual side.

A further unifying interpretation appears in the inexact proximal-point framework. The modern proximal bundle method is shown to be an instance of the HPE framework, with

$h$8

and with the bundle stopping rule $h$9 becoming exactly the HPE residual condition

$f$0

(Liang, 7 Jan 2025). This places proximal bundle methods and restarted accelerated gradient methods inside a common inexact proximal-point architecture.

4. Complexity theory and modern variants

Recent work has supplied explicit non-asymptotic complexity bounds for proximal bundle subproblems and outer iterations. In the semi-smooth setting, subgradients satisfy

$f$1

which interpolates between bounded subgradients at $f$2 and standard smoothness at $f$3 (Liang et al., 2024). For the regularized cutting-plane implementation of the proximal map, the semi-smooth inner complexity is

$f$4

while in the composite finite-sum setting the corresponding bound is

$f$5

with

$f$6

(Liang et al., 2024).

On top of this inner solver, the same work proposes an Adaptive Proximal Bundle Method in which the proximal parameter $f$7 is adjusted based on observed inner-gap contraction. The method is described as universal because it does not require $f$8, $f$9, bounds on $f$0, or other problem-specific constants as input (Liang et al., 2024). Its dominant total-inner-iteration term for finding an $f$1-solution is

$f$2

and the method matches the universal lower bound up to logarithmic factors when $f$3, that is, for general Lipschitz nonsmooth convex functions (Liang et al., 2024).

A different adaptive direction appears in the adaptive generic proximal bundle method for hybrid composite convex optimization

$f$4

That method uses a generic bundle-update black box together with a step-size adaptation rule based on a model-error contraction sequence $f$5 and thresholds $f$6 (Guigues et al., 2024). The number of outer cycles to reach $f$7-accuracy is bounded by

$f$8

and a full bound on total inner iterations is also established (Guigues et al., 2024).

The 2024 primal-dual proximal bundle framework proves an $f$9-type iteration complexity in primal-dual gap for convex nonsmooth composite optimization. Specifically, it shows that a primal-dual proximal bundle method produces averages $z^i$0 satisfying

$z^i$1

and with an appropriate choice of $z^i$2, the total iteration complexity is

$z^i$3

(Liang, 2024).

For stochastic convex composite optimization, the single-cut stochastic composite proximal bundle methods SCPB1 and SCPB2 attain optimal $z^i$4 complexity up to logarithmic factors. In the bounded-domain setting, SCPB1 yields

$z^i$5

and with suitable parameter choices the overall iteration complexity is $z^i$6 (Liang et al., 2022).

The following summary table situates several recent variants.

Variant	Setting	Representative guarantee
Adaptive Proximal Bundle Method	Semi-smooth / composite nonsmooth convex optimization	Universal; optimal up to logs at $z^i$7 (Liang et al., 2024)
Adaptive Generic Proximal Bundle	Hybrid composite convex optimization	Explicit total-inner-iteration bound (Guigues et al., 2024)
Primal-dual proximal bundle	Convex composite nonsmooth optimization	$z^i$8 in primal-dual gap (Liang, 2024)
SCPB	Stochastic convex composite optimization	$z^i$9 up to logs (Liang et al., 2022)

A recurring ambiguity in the literature is the acronym “APBM.” In (Liang et al., 2024), it denotes an Adaptive Proximal Bundle Method for nonsmooth optimization, whereas in (Zheng et al., 1 Apr 2026) and (Liao et al., 4 Dec 2025) it denotes an Accelerated Proximal Bundle Method for smooth convex optimization. The shared acronym reflects different extensions of the same proximal-bundle core rather than a single algorithm.

5. Acceleration, smooth regimes, and restart interpretations

Although proximal bundle methods originated in nonsmooth convex optimization, several papers study their acceleration in smooth settings. One line derives bundle algorithms by applying Nesterov-type acceleration to the Moreau–Yosida regularization. In that approach, the exact proximal-point iteration

$g^i\in\partial f(z^i)$0

is embedded in an inertial update

$g^i\in\partial f(z^i)$1

and the exact proximal point $g^i\in\partial f(z^i)$2 is replaced by an approximate bundle-based proximal point (Ouorou, 2020). The resulting algorithms FPBA1 and FPBA2 obtain

$g^i\in\partial f(z^i)$3

for FPBA1 and

$g^i\in\partial f(z^i)$4

for FPBA2 (Ouorou, 2020).

A simpler acceleration mechanism integrates Nesterov momentum directly into the proximal bundle step. In one 2026 formulation, APBM uses the extrapolated point $g^i\in\partial f(z^i)$5, constructs a model $g^i\in\partial f(z^i)$6 satisfying convex minorant conditions, and computes

$g^i\in\partial f(z^i)$7

followed by the standard momentum update

$g^i\in\partial f(z^i)$8

(Zheng et al., 1 Apr 2026). Under Assumptions 1 and 2 of that paper, the algorithm satisfies

$g^i\in\partial f(z^i)$9

and therefore

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$0

matching the optimal $$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$1 rate of accelerated gradient methods (Zheng et al., 1 Apr 2026).

Another 2025 accelerated proximal bundle method preserves the standard bundle test and standard model assumptions while obtaining the optimal smooth-convex complexity

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$2

for producing an $$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$3-accurate solution (Liao et al., 4 Dec 2025). In that formulation, the accelerated method differs from a three-sequence accelerated gradient scheme only by replacing the gradient step

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$4

with

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$5

where $$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$6 is the standard proximal bundle step (Liao et al., 4 Dec 2025).

A nearby development establishes the first variant of PBM for smooth objectives with accelerated convergence rate

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$7

using a smooth lower approximation and a novel null-step test rather than the classical cutting-plane model (Fersztand et al., 29 Apr 2025). That work interprets PBM as an inexact proximal-point algorithm and bases acceleration on accelerated inexact proximal-point theory (Fersztand et al., 29 Apr 2025).

The restart perspective unifies these developments. The 2025 unification paper shows that both restarted accelerated gradient and the modern proximal bundle method are instances of inexact proximal-point frameworks: restarted ACG is an A-HPE implementation for smooth composite problems, while MPB is an HPE implementation for nonsmooth composite problems (Liang, 7 Jan 2025). A plausible implication is that acceleration and bundling are best seen not as competing paradigms but as different inner solvers for proximal subproblems.

6. Extensions, applications, and broader scope

Proximal bundle methods have expanded well beyond deterministic unconstrained nonsmooth minimization. In stochastic convex composite optimization with expectation objectives

$$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$8

the stochastic composite proximal bundle methods SCPB1 and SCPB2 use single-cut models in expectation and handle continuous distributions directly (Liang et al., 2022). The paper states that, to the best of its knowledge, this is the first proximal bundle method for stochastic programming able to deal with continuous distributions (Liang et al., 2022).

In decentralized optimization, historical information is incorporated into a proximal bundle framework adapted to proximal decentralized gradient descent, producing a Decentralized Proximal Bundle Method and asynchronous and stochastic extensions (Zhu et al., 17 Dec 2025). The paper emphasizes that asynchronous DPBM and its stochastic variant can converge with fixed step-sizes that are independent of delays, in contrast with the delay-dependent step-sizes required by most existing asynchronous optimization methods (Zhu et al., 17 Dec 2025).

In convex-concave nonsmooth composite saddle-point problems, proximal bundle methods have been extended through primal-dual proximal-point constructions. The resulting PB-SPP algorithm applies the primal-dual cutting-plane scheme separately to the $$m_j(x)=\max_{(z^i,f(z^i),g^i)\in\mathcal{B}_j} \left\{f(z^i)+\langle g^i,x-z^i\rangle\right\},$$9- and $f$00-side proximal subproblems and yields explicit complexity bounds for finding an approximate saddle point (Liang, 2024).

Application-driven variants also exploit special structure. For piecewise-linear composite objectives, a modified proximal bundle method with fixed absolute accuracy admits a Frank–Wolfe interpretation on the Moreau envelope of the dual and achieves

$f$01

in the analyzed setting, improving over previously available guarantees cited in that work (Fersztand et al., 2024). In multistage adaptive robust optimization, a transformation–proximal bundle algorithm combines a multi-to-two-stage reformulation with bundle stabilization, yielding an average gap of merely $f$02 versus $f$03 for an affine disturbance-feedback control policy in the reported inventory-control application (Ning et al., 2018). In large-scale semidefinite reformulations of three-phase power flow feasibility, a three-cut proximal bundle method on an exact-penalty dual is reported to be numerically over 400 times faster than MOSEK with less than $f$04 of its memory, and approximately 2 times faster with 75% less memory on a decomposed BIM-SDP (Fang et al., 25 May 2026). In structured MAP inference, a proximal bundle method implemented through a multi-plane block-coordinate Frank–Wolfe algorithm is reported to outperform state-of-the-art Lagrangian-decomposition-based algorithms on challenging Markov random field, discrete tomography, and graph matching problems (Swoboda et al., 2018).

Two broader themes emerge from these extensions. First, proximal bundle methods are no longer confined to “classical nonsmooth convex minimization”; they now appear in smooth acceleration, stochastic programming, distributed optimization, saddle-point problems, robust optimization, semidefinite optimization, and discrete inference. Second, the historical information that defines the bundle is being reinterpreted in multiple ways: as a proximal oracle approximation (Liang et al., 2024), as an inexact proximal-point residual (Liang, 7 Jan 2025), as a primal-dual certificate (Liang, 2024), and as a low-memory structure that can be specialized to domain-specific oracles (Swoboda et al., 2018, Fang et al., 25 May 2026).

From a contemporary perspective, the proximal bundle method is best understood as a stabilized model-based approximation of the proximal point method. Its defining operations remain the same—construct a convex lower model from historical first-order information, regularize it proximally, and update the center only when the model is accurate enough—but current research has considerably widened both its theory and its algorithmic realizations (Liang et al., 2024, Liang, 2024, Liang, 7 Jan 2025).