Cascading Robustness Verification (CRV)

• CRV is a framework that certifies adversarial robustness by cascading multiple incomplete verifiers in a fixed, efficiency-aware order.
• It addresses single verifier limitations by combining complementary LP-based and SDP-based relaxations to reduce false negatives and bridge training–verification misalignment.
• The method employs stepwise relaxation and redundant constraint pruning to significantly reduce runtime while maintaining verified robust accuracy.

Cascading Robustness Verification (CRV) is a framework for certifying adversarial robustness of neural networks by combining multiple incomplete verifiers in a fixed, efficiency-aware cascade. In the formulation introduced in "Cascading Robustness Verification: Toward Efficient Model-Agnostic Certification" (Maleki et al., 4 Feb 2026), CRV is model-agnostic in the sense that its correctness and robustness guarantees are independent of the model’s training process, and its central decision rule is that an input is certifiably robust if at least one verifier in the cascade certifies it. The framework was proposed to address two coupled problems in post-hoc robustness certification: the computational cost of tight relaxations, and the underestimation of robustness that arises when a single incomplete verifier yields loose bounds or is misaligned with the training method.

1. Formal certification setting

For a classifier with logits $f(x) \in \mathbb{R}^L$ and predicted label

$y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$

robustness under an $\ell_\infty$ perturbation budget $\varepsilon$ is evaluated on

$$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$

The paper considers a one-hidden-layer ReLU network

$f(x) = W_2\,\mathrm{ReLU}(W_1 x),$

with $W_1 \in \mathbb{R}^{m\times d}$ and $W_2 \in \mathbb{R}^{L\times m}$, and defines, for each incorrect class $y' \neq y$, the worst-case logit margin

$$l^\star(y, y') \triangleq \max_{x'} \big(f(x')_{y'} - f(x')_{y}\big)$$

subject to $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$0 and $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$1. An input $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$2 is certifiably robust if $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$3 for all $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$4 (Maleki et al., 4 Feb 2026).

Because the ReLU constraints make the feasible set non-convex, solving the exact problem is NP-hard. Complete methods such as MILP and SMT can provide exact answers but are typically prohibitively slow for non-toy networks. Incomplete verifiers replace the non-convex constraints with convex relaxations and instead solve

$y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$5

where $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$6 is a convex relaxation of the original constraints. Certification then uses the implication

$y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$7

This construction yields no false positives, because $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$8 upper bounds $y \triangleq \arg\max_{1 \leq i \leq L} f(x)_i,$9, but it can yield false negatives. The paper formalizes the resulting partition of the dataset into true positives $\ell_\infty$0, true negatives $\ell_\infty$1, and false negatives $\ell_\infty$2, with $\ell_\infty$3. In this setting, robustness certification is conservative: failure to certify does not imply non-robustness.

2. Metric interpretation and the single-verifier problem

The paper distinguishes between True Robust Accuracy (TRA),

$\ell_\infty$4

and verified robust accuracy,

$\ell_\infty$5

Thus $\ell_\infty$6 is a lower bound on $\ell_\infty$7, and for attack success rate $\ell_\infty$8 from a strong empirical attack such as PGD,

$\ell_\infty$9

The paper’s stated objective is therefore not merely to increase a reported $\varepsilon$0, but to tighten the interval $\varepsilon$1 so that post-hoc certification better approximates true robustness (Maleki et al., 4 Feb 2026).

Two limitations of single incomplete verifiers are emphasized. The first is loose bounds $\varepsilon$2 false negatives: if a relaxation is coarse, many truly robust inputs remain uncertified, and $\varepsilon$3 can significantly underestimate $\varepsilon$4. The second is training–verification misalignment, also described as “verification-specific behavior”: a model trained with one surrogate, such as LP-based, IBP, or SDP-based training, can verify poorly under a different relaxation even when its true robustness has not changed. The paper gives the concrete example that an SDP-trained model may verify poorly under LP-based certification, while an LP-trained model may verify poorly under SDP relaxations. On this account, a single $\varepsilon$5 value measured with a single verifier is not a verifier-independent robustness metric.

The motivating claim behind CRV is that different verifiers have complementary strengths. Some are cheap and loose, others are tight and expensive; some align better with one training procedure, others with another. CRV addresses this by aggregating certifications across verifiers rather than treating one verifier as definitive.

3. Cascade structure and model-agnostic certification

CRV organizes verifiers $\varepsilon$6 in a fixed order from least to most expensive. For each input, the cascade runs $\varepsilon$7 first and halts immediately if $\varepsilon$8 certifies robustness; otherwise it proceeds to $\varepsilon$9, and continues until some verifier certifies the input or all verifiers fail. The decision rule is therefore a logical OR over verifier outputs: if the true-positive sets of the constituent verifiers are $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$0, then CRV certifies the union $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$1 (Maleki et al., 4 Feb 2026).

This construction is described as model-agnostic because it treats the network as a fixed function and does not assume that the model was trained using any of the constituent verifiers. In the paper’s terminology, CRV is a post-hoc verifier that can aggregate LP-based, SDP-based, and other verification methods without binding evaluation to the model’s training surrogate. The order is fixed a priori rather than input-dependent. Remark 1 guides ordering by relaxation type and typical complexity: leanest relaxations first, richer relaxations later.

The paper concretely instantiates CRV with two verifier families. The first is LP-cert, an LP-based relaxation derived from convex outer approximations of the adversarial polytope. It is fast and scalable but typically looser. The second is SDP-cert, a semidefinite relaxation that encodes ReLU and input constraints as quadratic constraints and lifts them to an SDP. For $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$2 input bounds $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$3 and $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$4, the input constraint is written as

$$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$5

while ReLU is encoded by

$$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$6

The resulting SDP relaxation is tighter but substantially more computationally expensive. In the reported experiments, CRV typically orders these as $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$7 LP-cert and $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$8 SDP-cert.

4. Stepwise Relaxation and Fast Stepwise Relaxation

Within a single verifier $$B_{\varepsilon}(x) \triangleq \{x' \mid \|x' - x\|_{\infty} \leq \varepsilon\}.$$9, the paper introduces the Stepwise Relaxation Algorithm (SR) as an internal cascade over progressively tighter submethods $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$0. The $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$1-th submethod uses feasible set

$f(x) = W_2\,\mathrm{ReLU}(W_1 x),$2

which yields nested feasible sets

$f(x) = W_2\,\mathrm{ReLU}(W_1 x),$3

SR runs the loosest submethod first and halts as soon as one submethod certifies the input; otherwise it continues to tighter variants (Maleki et al., 4 Feb 2026).

The underlying monotonicity argument is elementary but central. If two maximization problems share the same objective and $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$4, then the optimal values satisfy $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$5. Applied to verification, tighter relaxations have fewer false negatives and certify a superset of the robust inputs certified by looser relaxations. The paper states this as Corollary 1: if $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$6, then $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$7 and $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$8.

Theorem 2 gives the key guarantee for SR: the robust accuracy of SR equals the robust accuracy of the tightest submethod alone. In other words, early exits do not change which inputs are ultimately certified relative to always running the tightest relaxation; they only reduce computation. In the reported SDP instantiation, three SDP-based submethods are used: $f(x) = W_2\,\mathrm{ReLU}(W_1 x),$9 is the full SDP-cert, $W_1 \in \mathbb{R}^{m\times d}$0 drops one linear ReLU constraint and the quadratic ReLU constraint, and $W_1 \in \mathbb{R}^{m\times d}$1 drops an additional constraint from $W_1 \in \mathbb{R}^{m\times d}$2. On MNIST/Grad-NN at $W_1 \in \mathbb{R}^{m\times d}$3, $W_1 \in \mathbb{R}^{m\times d}$4 achieves 88% $W_1 \in \mathbb{R}^{m\times d}$5 in 328.98 min, while SR achieves the same 88% $W_1 \in \mathbb{R}^{m\times d}$6 in 231.43 min, approximately 30% faster.

The paper further defines Fast Stepwise Relaxation (FSR), which adds two pragmatic heuristics: redundant-constraint pruning and early termination via thresholding. Before solving, constraints whose feasible region is contained in others, or whose tightening is negligible, are removed. Then, if a submethod’s incremental improvement on a small subset falls below a threshold such as 5%, that submethod is skipped for the full dataset. Empirically, FSR yields up to ~53% speedup over the tightest SDP version, with negligible loss $W_1 \in \mathbb{R}^{m\times d}$7 in certified bounds or $W_1 \in \mathbb{R}^{m\times d}$8.

5. Guarantees, runtime accounting, and reported results

Theorem 1 states the principal certification guarantee of CRV: $W_1 \in \mathbb{R}^{m\times d}$9 Hence CRV certifies at least as many inputs as the best individual verifier in the cascade. Its total verification cost is written as

$W_2 \in \mathbb{R}^{L\times m}$0

so later, more expensive stages are only applied to inputs not already certified by earlier stages. The same logic applies within SR for a single verifier (Maleki et al., 4 Feb 2026).

The empirical evaluation uses MNIST, $W_2 \in \mathbb{R}^{L\times m}$1 perturbations with $W_2 \in \mathbb{R}^{L\times m}$2, and two two-layer fully connected networks with 500 hidden nodes: Grad-NN, trained with an SDP-based regularization, and LP-NN, trained with LP-based robust training. Certification is strictly post hoc and independent of the training procedure.

Setting	SDP-cert	CRV-FSR
Grad-NN, $W_2 \in \mathbb{R}^{L\times m}$3	RA = 88%, 328.98 min	RA = 88%, 111.34 min, 66.16% speedup
LP-NN, $W_2 \in \mathbb{R}^{L\times m}$4	RA = 88%, 328.98 min	RA = 88%, 34.55 min, 89.5% speedup

At $W_2 \in \mathbb{R}^{L\times m}$5, the misalignment phenomenon is explicit. For Grad-NN, LP-cert reports 42% $W_2 \in \mathbb{R}^{L\times m}$6, whereas SDP-cert reports 88% $W_2 \in \mathbb{R}^{L\times m}$7; CRV, CRV-SR, and CRV-FSR all recover 88% $W_2 \in \mathbb{R}^{L\times m}$8 while reducing runtime to 190.8 min, 134.23 min, and 111.34 min, respectively. For LP-NN, LP-cert reports 82% $W_2 \in \mathbb{R}^{L\times m}$9 and SDP-cert 88% $y' \neq y$0; CRV reaches 88% $y' \neq y$1 in 59.22 min, CRV-SR in 41.66 min, and CRV-FSR in 34.55 min. PGD success rates at $y' \neq y$2 are 10% for Grad-NN and 12% for LP-NN, so $y' \neq y$3 lies between $y' \neq y$4 and $y' \neq y$5, which places the interval at $y' \neq y$6 for Grad-NN.

Across $y' \neq y$7, the paper reports that CRV, CRV-SR, and CRV-FSR achieve $y' \neq y$8 close to or higher than SDP-cert, with runtime reductions roughly 40–90% depending on model and perturbation size. At larger $y' \neq y$9, certification becomes harder and $$l^\star(y, y') \triangleq \max_{x'} \big(f(x')_{y'} - f(x')_{y}\big)$$0 drops for all methods, but CRV-FSR still reports substantial verified accuracy, including 36% at $$l^\star(y, y') \triangleq \max_{x'} \big(f(x')_{y'} - f(x')_{y}\big)$$1 for Grad-NN and 78% for LP-NN. Runtime is described as non-monotonic: intermediate budgets can force many inputs into the tightest stages, whereas very large budgets can allow many inputs to be rejected early as clearly non-robust.

6. Scope, distinctions, and terminology

Within robustness verification, CRV is positioned against both complete methods and single-verifier incomplete methods. Complete methods remain exact but expensive; incomplete methods scale but incur false negatives. The stated innovation of CRV is not only an engineering speedup but an explicit critique of the robustness metric obtained from a single incomplete verifier: a verifier-dependent $$l^\star(y, y') \triangleq \max_{x'} \big(f(x')_{y'} - f(x')_{y}\big)$$2 can be heavily biased by bound looseness and by training–verification misalignment. The paper also states several limitations: CRV cannot exceed the joint capability of the included verifiers, ordering is fixed and not input-adaptive, the experiments focus on relatively small networks and MNIST, and scaling to large CNNs or Transformers may require more sophisticated relaxations and verifiers (Maleki et al., 4 Feb 2026).

A distinct cautionary result in the literature concerns cascading ensembles of certifiably robust classifiers. "On the Perils of Cascading Robust Classifiers" shows that such ensembles can be unsound because the routing mechanism that selects among base models is itself non-robust: the ensemble may claim local robustness at $$l^\star(y, y') \triangleq \max_{x'} \big(f(x')_{y'} - f(x')_{y}\big)$$3 even though a nearby $$l^\star(y, y') \triangleq \max_{x'} \big(f(x')_{y'} - f(x')_{y}\big)$$4 causes the cascade to switch models and change its prediction (Mangal et al., 2022). CRV in (Maleki et al., 4 Feb 2026) addresses a different object: it aggregates multiple incomplete verifiers to certify a single fixed model, rather than composing predictions from multiple models. This distinction is central when interpreting the term cascade.

The acronym CRV is also used in unrelated contexts. In control-software verification, it denotes conditionally relevant variables in a formal method for identifying which variables can affect safety under single event upset (Ganesha et al., 10 May 2025). In LLM interpretability, it denotes Circuit-based Reasoning Verification, a white-box classifier over attribution-graph features for step-level chain-of-thought error detection (Zhao et al., 10 Oct 2025). In the adversarial-robustness literature considered here, however, CRV refers specifically to Cascading Robustness Verification as a model-agnostic, verifier-composition framework for certified robustness (Maleki et al., 4 Feb 2026).