ECLipsE-Gen-Local: Local Lipschitz Estimation

• ECLipsE-Gen-Local is a family of algorithms that estimate local Lipschitz constants of deep feedforward networks through a compositional semidefinite programming approach.
• It decomposes a global SDP certificate into efficient, stage-wise subproblems and refines activation slope bounds using local input-region analysis.
• Its Acc, Fast, and CF variants offer a trade-off between computational speed and tightness of bounds, leading to improved robustness certification.

Searching arXiv for the specified paper and a related foundational reference on Lipschitz estimation hardness. ECLipsE-Gen-Local is a family of algorithms for estimating local Lipschitz constants of deep feedforward neural networks through a compositional semidefinite-programming framework that combines heterogeneous slope bounds, layer-wise decomposition, and input-region refinement (Xu et al., 6 Oct 2025). It is designed for the setting in which exact Lipschitz computation is NP-hard and standard SDP relaxations scale poorly with network depth. The method begins from a generalized SDP certificate for global or subnetwork-level Lipschitz estimation, then decomposes that certificate into a sequence of small stage-wise problems, and finally tightens the result by incorporating local information from an input ball. The resulting estimates are strict upper bounds, and for sufficiently small input regions they approach the exact Jacobian norm obtained from autodiff (Xu et al., 6 Oct 2025).

1. Problem setting and scope

The framework considers a feedforward network of the form

$$z^{(0)}=\text{input}\in\mathbb{R}^{d^0},\qquad v^{(i)}=W_i z^{(i-1)}+b_i,\qquad z^{(i)}=\phi(v^{(i)})\ \text{for } i=1,\dots,N-1,$$

with output

$y=z^{(N)}=v^{(N)}.$

The objective is to find the smallest $L\ge 0$ such that

$$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$

This constant is central to robustness certification because it upper-bounds output sensitivity to input perturbations (Xu et al., 6 Oct 2025).

The method is motivated by two difficulties. First, exact computation of the Lipschitz constant is NP-hard, a point attributed to Virmaux and Scaman (2018) in the paper. Second, standard upper-bound methods typically relax the problem to a large SDP whose size grows rapidly with depth. ECLipsE-Gen-Local targets both issues by retaining SDP-based certification while reducing the problem to stage-wise subproblems and by exploiting local information on the input region to tighten the bound (Xu et al., 6 Oct 2025).

A notable aspect of the framework is its generality. The generalized certificate accommodates heterogeneous activation-function slope bounds, arbitrary input-output pairs, and arbitrary choices of sub-networks of consecutive layers. In the terminology of the paper, this generalized global framework is ECLipsE-Gen, while ECLipsE-Gen-Local denotes the local refinement obtained by restricting attention to an input ball and refining per-neuron slope intervals (Xu et al., 6 Oct 2025).

2. Generalized SDP certificate

The starting point is a slope-restricted characterization of the activation function. Element-wise, for any two pre-activations $v,v'$,

$$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$

Equivalently, for any nonnegative diagonal $\Lambda$,

$$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$

with $p=\alpha\beta$ and $m=(\alpha+\beta)/2$ (Xu et al., 6 Oct 2025).

Using layer-wise lower and upper slope vectors $y=z^{(N)}=v^{(N)}.$0 and $y=z^{(N)}=v^{(N)}.$1, the paper introduces diagonal variables $y=z^{(N)}=v^{(N)}.$2 for $y=z^{(N)}=v^{(N)}.$3 and a scalar $y=z^{(N)}=v^{(N)}.$4, and defines a block-tridiagonal LMI $y=z^{(N)}=v^{(N)}.$5. Its diagonal blocks are

$y=z^{(N)}=v^{(N)}.$6

and the off-diagonal blocks are

$y=z^{(N)}=v^{(N)}.$7

Whenever $y=z^{(N)}=v^{(N)}.$8 holds, the certificate yields

$y=z^{(N)}=v^{(N)}.$9

Maximizing $L\ge 0$0 therefore gives the tightest SDP certificate of $L\ge 0$1 (Xu et al., 6 Oct 2025).

The same block-tridiagonal construction also supports arbitrary subnetworks and index sets. To certify the Lipschitz constant $L\ge 0$2 from coordinates indexed by $L\ge 0$3 in layer $L\ge 0$4 to coordinates indexed by $L\ge 0$5 in layer $L\ge 0$6 with $L\ge 0$7, the construction restricts $L\ge 0$8 to columns $L\ge 0$9 and $$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$0 to rows $$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$1, while retaining the same form of the $$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$2 and $$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$3 blocks. This makes the certificate applicable not only to end-to-end input-output bounds but also to intermediate-layer and coordinate-restricted maps (Xu et al., 6 Oct 2025).

3. Compositional decomposition into stage-wise problems

A central contribution is the decomposition of the global block-tridiagonal SDP into a sequence of small subproblems. By a recursive Schur-complement characterization, $$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$4 is equivalent to the positivity conditions

$$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$5

where

$$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$6

and for $$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$7,

$$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$8

The terminal quantity is

$$\|f(z_1)-f(z_2)\|_2 \le L\,\|z_1-z_2\|_2 \qquad \forall z_1,z_2\in Z.$$9

This reformulation converts a single large certificate into a recursive chain in which each stage depends on the preceding one through $v,v'$0 (Xu et al., 6 Oct 2025).

The recursion is summarized through a “messenger matrix” $v,v'$1, defined by

$v,v'$2

and for $v,v'$3,

$v,v'$4

Then

$v,v'$5

This representation supports backward optimization over layer-local certificates rather than direct solution of the full SDP (Xu et al., 6 Oct 2025).

Working backwards to maximize $v,v'$6 produces three variants:

Variant	Layer variable choice	Stage characterization
ECLipsE-Gen-Acc	Full $v,v'$7	Small SDP at each layer
ECLipsE-Gen-Fast	$v,v'$8	Same LMI with scalar $v,v'$9
ECLipsE-Gen-CF	Closed form under $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$0	Spectral-norm formulas, no SDP solver

For ECLipsE-Gen-Acc, stage $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$1 solves

$$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$2

subject to

$$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$3

ECLipsE-Gen-Fast uses the same LMI with scalar $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$4, and ECLipsE-Gen-CF replaces the SDP by closed-form spectral quantities (Xu et al., 6 Oct 2025).

The computational consequence is explicit. The network is processed in a single forward pass to compute centers $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$5 and a backward pass of $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$6 stages. Each stage uses one small SDP of size approximately $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$7 or a closed-form spectral computation. The total cost scales as $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$8 for the SDP variants, while the closed-form variant has the same dominant $$\alpha \le \frac{\phi(v)-\phi(v')}{v-v'} \le \beta.$$9 spectral-norm term but is often far cheaper in practice because the dominant operations reduce to vector-matrix multiplies (Xu et al., 6 Oct 2025).

4. Local input-region refinement

ECLipsE-Gen-Local augments the compositional framework by refining activation slope bounds inside a specific input region. The region is taken to be an input ball

$\Lambda$0

If a Lipschitz bound $\Lambda$1 is known for the map from the input to the $\Lambda$2-th pre-activation at layer $\Lambda$3, then the mean-value theorem gives

$\Lambda$4

Hence

$\Lambda$5

This converts a global slope-restriction problem into neuron-wise interval analysis over a local pre-activation range (Xu et al., 6 Oct 2025).

For each neuron $\Lambda$6 in layer $\Lambda$7, the local slope bounds are then refined as

$\Lambda$8

The paper states that closed-form expressions are available for ReLU, tanh, sigmoid, ELU, Leaky-ReLU, and related activations. Because the refined interval $\Lambda$9 encloses the true local subgradient range on $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$0, the resulting SDP or closed-form certificate remains valid while becoming tighter than a global bound when the input region is sufficiently restricted (Xu et al., 6 Oct 2025).

The local algorithm proceeds as follows. It initializes $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$1 and computes the forward-pass center $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$2. For each layer $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$3, it computes per-neuron Lipschitz values $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$4 via the diagonal entries of $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$5, bounds the range of $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$6, refines $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$7, solves the stage-$$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$8 subproblem in the Acc, Fast, or CF form, and updates $$\begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix}^{\!\top} \begin{bmatrix} p\Lambda & -m\Lambda\ -m\Lambda & \Lambda \end{bmatrix} \begin{bmatrix}\Delta v\ \Delta \phi\end{bmatrix} \le 0,$$9. The final bound is

$p=\alpha\beta$0

This algorithm retains the compositional structure of ECLipsE-Gen while injecting first-order local information at every hidden layer (Xu et al., 6 Oct 2025).

5. Closed-form specialization and theoretical guarantees

The closed-form specialization, ECLipsE-Gen-CF, applies under the mild condition $p=\alpha\beta$1, which the paper states holds for almost all activations and their local subintervals. Under this condition, one may choose adjusted bounds $p=\alpha\beta$2 so that, for each neuron, either $p=\alpha\beta$3 or $p=\alpha\beta$4, thereby forcing $p=\alpha\beta$5 in the slope constraint. The stage-wise optimization then reduces to

$p=\alpha\beta$6

and

$p=\alpha\beta$7

No SDP solver is needed; only largest-singular-value computations are required (Xu et al., 6 Oct 2025).

The paper provides three classes of theoretical guarantees. First, under $p=\alpha\beta$8 and $p=\alpha\beta$9, the Acc SDP and Fast SDP are strictly feasible at each stage, satisfying Slater’s condition, and the closed-form quantities $m=(\alpha+\beta)/2$0 are well-defined and produce $m=(\alpha+\beta)/2$1. Second, the refined local slope bounds enclose the true subgradient ranges of $m=(\alpha+\beta)/2$2 on $m=(\alpha+\beta)/2$3, and all per-neuron and final Lipschitz quantities certify

$m=(\alpha+\beta)/2$4

and

$m=(\alpha+\beta)/2$5

Third, the maximal feasible $m=(\alpha+\beta)/2$6 is monotone with respect to slope tightening: if the interval $m=(\alpha+\beta)/2$7 is reduced for any single neuron, the maximal feasible $m=(\alpha+\beta)/2$8 can only increase, so tighter slope intervals imply tighter Lipschitz bounds (Xu et al., 6 Oct 2025).

These guarantees establish that the local refinement is not merely heuristic. The feasibility theorem ensures that each stage admits a valid certificate, the validity theorem ensures that the resulting $m=(\alpha+\beta)/2$9 is an upper bound on the true Lipschitz constant over the specified region, and the monotonicity theorem formalizes why local interval refinement improves the certificate whenever the input region reduces uncertainty in the activation slopes (Xu et al., 6 Oct 2025).

6. Numerical behavior and relation to robustness

The paper reports that local refinement can drive the certified upper bound toward the exact Jacobian norm as the input region shrinks. In a 5-layer ReLU network with 128 neurons and center $y=z^{(N)}=v^{(N)}.$00, the radius values

$y=z^{(N)}=v^{(N)}.$01

produce ECL-Gen-Local-Acc bounds

$y=z^{(N)}=v^{(N)}.$02

while

$y=z^{(N)}=v^{(N)}.$03

by autodiff. As $y=z^{(N)}=v^{(N)}.$04, the certified $y=z^{(N)}=v^{(N)}.$05 approaches the autodiff value. In a second example, a small random feedforward network with $y=z^{(N)}=v^{(N)}.$06 and $y=z^{(N)}=v^{(N)}.$07 shows local bounds that are often $y=z^{(N)}=v^{(N)}.$08–$y=z^{(N)}=v^{(N)}.$09 tighter than global bounds (Xu et al., 6 Oct 2025).

The global speed-tightness trade-off is quantified on both small and large networks. For a 20-layer ReLU network with 40 neurons per layer, the naive product bound $y=z^{(N)}=v^{(N)}.$10 is approximately $y=z^{(N)}=v^{(N)}.$11 and is described as very loose. LipSDP-Neuron gives $y=z^{(N)}=v^{(N)}.$12 in approximately $y=z^{(N)}=v^{(N)}.$13. ECL-Gen-Acc gives $y=z^{(N)}=v^{(N)}.$14 in approximately $y=z^{(N)}=v^{(N)}.$15, ECL-Gen-Fast gives $y=z^{(N)}=v^{(N)}.$16 in approximately $y=z^{(N)}=v^{(N)}.$17, and ECL-Gen-CF gives $y=z^{(N)}=v^{(N)}.$18 in approximately $y=z^{(N)}=v^{(N)}.$19. For a larger ELU network with 60 layers and 100 neurons per layer, LipSDP-Layer split across subnets takes approximately $y=z^{(N)}=v^{(N)}.$20 with $y=z^{(N)}=v^{(N)}.$21; ECL-Gen-Acc takes approximately $y=z^{(N)}=v^{(N)}.$22 with $y=z^{(N)}=v^{(N)}.$23; ECL-Gen-Fast takes approximately $y=z^{(N)}=v^{(N)}.$24 with $y=z^{(N)}=v^{(N)}.$25; and ECL-Gen-CF takes approximately $y=z^{(N)}=v^{(N)}.$26 with $y=z^{(N)}=v^{(N)}.$27 (Xu et al., 6 Oct 2025).

The reported robustness experiment links the certified local Lipschitz estimate to adversarial behavior. On an 8-layer MNIST MLP, as the adversarial attack radius $y=z^{(N)}=v^{(N)}.$28 decreases, the certified local $y=z^{(N)}=v^{(N)}.$29 from ECL-Gen-Fast correlates tightly with empirical PGD failure rate, and the robustly trained model exhibits both lower $y=z^{(N)}=v^{(N)}.$30 and lower attack-error. Within the scope of the reported experiments, this establishes practical alignment between the local Lipschitz certificate and observed network robustness (Xu et al., 6 Oct 2025).

The paper summarizes these findings by stating that ECLipsE-Gen-Local combines a heterogeneous-slope SDP certificate, layer-wise compositional decomposition, and local slope refinement via first-order bounds; that the Acc variant matches LipSDP-neuron accuracy at approximately $y=z^{(N)}=v^{(N)}.$31 speedup; that the Fast variant matches LipSDP-layer at approximately $y=z^{(N)}=v^{(N)}.$32 speedup; and that the CF variant yields near-instantaneous upper bounds that approach the exact Jacobian norm for small input regions (Xu et al., 6 Oct 2025).