Triangle Scaling Exponent (TSE) in Convex Optimization

• Triangle Scaling Exponent (TSE) is defined via the triangle-scaling property of a Bregman distance and determines an accelerated convergence rate of O(k^(-γ)) in convex optimization.
• In Euclidean settings, the TSE is exactly 2, while in many non-Euclidean cases the uniform exponent may be lower despite the intrinsic local quadratic behavior remaining at 2.
• Adaptive accelerated Bregman proximal-gradient methods exploit the intrinsic TSE to achieve faster empirical convergence, with numerical certificates confirming O(k^(-2)) behavior even under non-Euclidean geometries.

Searching arXiv for the specified paper and closely related work on triangle scaling exponent in accelerated Bregman proximal-gradient methods. The triangle scaling exponent (TSE) is an exponent $\gamma>0$ associated with the triangle-scaling property of a Bregman distance $D_h$, and it governs the accelerated convergence rate attainable by accelerated Bregman proximal-gradient methods in relatively smooth convex optimization. In the setting of minimizing the sum of two convex functions—one differentiable and relatively smooth with respect to a reference convex function, and the other possibly nondifferentiable but simple to optimize—the TSE determines whether the resulting rate is $O(k^{-\gamma})$, with the Euclidean case recovering the classical accelerated rate $O(k^{-2})$ and many non-Euclidean cases exhibiting smaller uniform exponents. At the same time, the intrinsic TSE captures small-$\theta$ asymptotics and is always equal to $2$ under the smoothness assumptions stated below, which motivates adaptive accelerated schemes and posterior numerical certificates for fast empirical rates (Hanzely et al., 2018).

1. Definition through the triangle-scaling property

Let $h:\mathbb{R}^n\to(-\infty,+\infty]$ be a strictly convex function that is differentiable on the relative interior of its domain, $rint\,\dom h$, and essentially smooth (Legendre type). The associated Bregman distance is

$D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$

The triangle-scaling property is defined as follows. The Bregman distance $D_h$ has the triangle-scaling property with exponent $D_h$0 if, for all triples

$D_h$1

one has

$D_h$2

Any such $D_h$3 is called a (uniform) TSE of $D_h$4 (Hanzely et al., 2018).

The smallest $D_h$5 for which $D_h$6 is valid is the uniform TSE:

$D_h$7

In the analysis under discussion, $D_h$8 is used with constant $D_h$9. The more general bound

$O(k^{-\gamma})$0

is acknowledged, but $O(k^{-\gamma})$1 is sufficient in the paper’s development (Hanzely et al., 2018).

A basic remark is that if $O(k^{-\gamma})$2 is jointly convex in $O(k^{-\gamma})$3, then $O(k^{-\gamma})$4 holds with $O(k^{-\gamma})$5. This places $O(k^{-\gamma})$6 as a generic baseline, while larger exponents require additional structure.

2. Uniform TSE in Euclidean and non-Euclidean geometries

For the Euclidean reference function

$O(k^{-\gamma})$7

the triangle-scaling relation is exact:

$O(k^{-\gamma})$8

Accordingly, the uniform TSE is exactly $O(k^{-\gamma})$9 (Hanzely et al., 2018).

For jointly convex Bregman distances, including separable entropies whose second-derivative reciprocals are concave, $O(k^{-2})$0 holds with at least $O(k^{-2})$1. A canonical example is the generalized KL divergence

$O(k^{-2})$2

which has $O(k^{-2})$3 (Hanzely et al., 2018).

By contrast, other Bregman distances can fail joint convexity. The Itakura–Saito distance generated by Burg’s entropy is identified as an example for which the uniform TSE is typically strictly below $O(k^{-2})$4. This establishes a sharp geometric distinction between Euclidean and many non-Euclidean Bregman settings: Euclidean geometry supports a quadratic scaling law globally, whereas non-Euclidean divergences often satisfy only linear or sublinear global triangle scaling.

This distinction is operationally significant because the accelerated rate of the corresponding optimization method depends directly on the available uniform exponent. A plausible implication is that the global worst-case geometry of the Bregman distance, rather than only local curvature, controls the standard non-adaptive rate guarantee.

3. Intrinsic TSE and local quadratic behavior

The intrinsic TSE is introduced to isolate small-$O(k^{-2})$5 asymptotics and disregard global worst-case behavior. For $O(k^{-2})$6 twice continuously differentiable in its arguments, it is defined by

$O(k^{-2})$7

If $O(k^{-2})$8 is twice continuously differentiable on $O(k^{-2})$9, then

$\theta$0

and therefore

$\theta$1

The proof is summarized as a two-step Taylor/L’Hôpital argument showing that the leading term in the numerator is $\theta$2 with nonzero quadratic form coefficient (Hanzely et al., 2018).

The conceptual role of the intrinsic TSE is to separate local second-order structure from global scaling obstructions. A common misconception is to identify the TSE of a Bregman distance with a single universal number without distinguishing whether it is uniform or intrinsic. The paper’s framework makes the distinction explicit: the uniform TSE can be much smaller than $\theta$3, including values at or below $\theta$4 in important non-Euclidean examples, while the intrinsic TSE is still $\theta$5 under the stated differentiability assumptions.

4. Representative examples

The paper gives several illustrative examples of uniform and intrinsic TSE values and bounds (Hanzely et al., 2018).

Example	Uniform TSE	Intrinsic TSE
Euclidean distance $\theta$6	$\theta$7	$\theta$8
Negative entropy $\theta$9	$2$0	$2$1
Burg’s entropy $2$2	$2$3 numerically	$2$4
Strongly convex-smooth $2$5 on a bounded set	$2$6 up to condition-number factor $2$7	$2$8

These examples show that the uniform exponent is sensitive to the global geometry induced by the reference function. In particular, negative entropy and Burg’s entropy behave differently at the uniform level even though both retain intrinsic exponent $2$9. For strongly convex-smooth $h:\mathbb{R}^n\to(-\infty,+\infty]$0 on a bounded set, the statement that $h:\mathbb{R}^n\to(-\infty,+\infty]$1 holds with $h:\mathbb{R}^n\to(-\infty,+\infty]$2 up to a condition-number factor $h:\mathbb{R}^n\to(-\infty,+\infty]$3 indicates that quadratic scaling can re-emerge under additional conditioning and boundedness assumptions.

This suggests that the intrinsic exponent identifies an underlying quadratic local model, whereas the uniform exponent records how much of that local structure survives globally without degradation.

5. Role in accelerated Bregman proximal-gradient methods

The TSE enters the theory through accelerated Bregman proximal-gradient (ABPG) methods for minimizing the sum of two convex functions, where one is differentiable and relatively smooth with respect to $h:\mathbb{R}^n\to(-\infty,+\infty]$4 and the other is possibly nondifferentiable but simple to optimize. One version of ABPG takes a stepsize proportional to $h:\mathbb{R}^n\to(-\infty,+\infty]$5 and requires the uniform TSE $h:\mathbb{R}^n\to(-\infty,+\infty]$6 (Hanzely et al., 2018).

Under this condition, the iterates $h:\mathbb{R}^n\to(-\infty,+\infty]$7 satisfy

$h:\mathbb{R}^n\to(-\infty,+\infty]$8

The dependence on $h:\mathbb{R}^n\to(-\infty,+\infty]$9 is explicit: larger $rint\,\dom h$0 yields a faster worst-case rate. In the Euclidean case, $rint\,\dom h$1 recovers the classical $rint\,\dom h$2 convergence rate of Nesterov’s accelerated gradient methods, while $rint\,\dom h$3 yields only $rint\,\dom h$4 (Hanzely et al., 2018).

The TSE therefore functions as the rate-determining exponent in the non-Euclidean acceleration analysis. This is not merely a technical artifact of proof structure: in the paper’s formulation, the stepsize rule and the complexity guarantee are both parameterized by $rint\,\dom h$5. A plausible implication is that any gap between the uniform and intrinsic exponents creates a gap between the available worst-case theory and the local behavior suggested by the geometry.

6. Adaptive schemes, empirical rates, and numerical certificates

Because the intrinsic TSE is always $rint\,\dom h$6 while the uniform TSE may be smaller, the paper develops adaptive ABPG methods that exploit the intrinsic TSE and converge much faster in practice, even though theoretical guarantees for a fast global convergence rate are stated to be out of reach in general (Hanzely et al., 2018).

In numerical experiments on D-optimal design, Poisson inverse problems, and nonnegatively-constrained regression, several empirical observations are reported. Using the Euclidean-like Bregman with $rint\,\dom h$7 yields drastically faster convergence than plain BPG with $rint\,\dom h$8. For non-Euclidean reference functions such as Burg’s entropy, an adaptive scheme exploiting the intrinsic TSE through gain adaptation achieves empirical $rint\,\dom h$9 convergence, even when the uniform TSE might be less than $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$0 (Hanzely et al., 2018).

The posterior diagnostic is the local triangle-scaling gain

$D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$1

By $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$2, one has $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$3 if $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$4 is valid. Empirically, $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$5 or remains near $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$6 for $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$7, which provides a numerical certificate of $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$8 behavior in practice, even in cases where the a priori uniform TSE could be only $D_h(x,y)=h(x)-h(y)-\langle\nabla h(y),x-y\rangle,\qquad x\in\dom h,\; y\in rint\,\dom h.$9 (Hanzely et al., 2018).

An objective reading of these results requires distinguishing between theory and observation. The guaranteed rate of ABPG is $D_h$0 with $D_h$1 supplied by the uniform TSE. The empirical $D_h$2 behavior obtained by adaptive methods is presented as an observation supported by posterior numerical certificates, not as a general worst-case theorem.

7. Interpretation within relatively smooth convex optimization

Within relatively smooth convex optimization, the TSE provides a bridge between the geometry of the reference function and the attainable acceleration rate. The central tension is that many Bregman divergences have a uniform TSE at most $D_h$3, which limits the standard worst-case guarantee, yet their intrinsic exponent is still $D_h$4, revealing local second-order scaling (Hanzely et al., 2018).

This yields a two-level interpretation of non-Euclidean acceleration. At the global level, the uniform TSE controls provable convergence and may be degraded by lack of joint convexity or other non-Euclidean effects. At the local asymptotic level, the intrinsic TSE restores quadratic scaling universally under twice continuous differentiability of $D_h$5 on $D_h$6. Adaptive ABPG methods are designed precisely around this discrepancy.

A common source of confusion is to assume that non-Euclidean Bregman distances necessarily preclude accelerated $D_h$7 behavior. The paper does not support that conclusion. Instead, it shows that although the uniform TSE can be much smaller than $D_h$8, the intrinsic TSE is always $D_h$9, and adaptive accelerated methods can exploit this fact to attain and empirically certify an $D_h$00 rate in numerical experiments (Hanzely et al., 2018).