Branch and Bound for Relational Verification of Neural Networks

Abstract: Verification of neural networks against relational specifications, such as global robustness, is crucial for safety-critical applications of cyber-physical systems (CPS), given their increasing adoption of AI components. Compared to simple trace properties (e.g., local robustness), verifying relational specifications requires reasoning about the relationship between multiple network inferences, which brings significant technical challenges. Existing research has explored abstraction techniques based on sound and convex over-approximation of neural network outputs; however, since these approaches are inherently incomplete and may raise false alarms, they further underscore the need of effective abstraction refinement. In this paper, we propose a branch-and-bound (BaB) framework to mitigate the issue, which iteratively splits the problem until all sub-problems are verified. Specifically, our BaB framework features splitting of relational neurons rather than individual neurons as prior works do, and as the core of our technique, we devise a relational neuron selection strategy based on the dual formulation of the verification problem, which allows us to efficiently select the (most likely) optimal relational neuron that maximizes the refinement brought by problem splitting. We evaluate SaBRe on 817 verification problems across ACAS Xu, MNIST-F, MNIST-C, CIFAR and GTSRB. The results show that SaBRe outperforms different baseline approaches, in terms of the number of solved instances and verification efficiency, which demonstrates the effectiveness of our proposed techniques.

• The paper introduces SaBRe, a branch-and-bound verifier that splits relational neurons—differences between paired inferences—to reduce over-approximation error in global robustness checks.
• SaBRe’s dual-based relational-neuron selection solves more instances than RaVeN and individual-splitting baselines across most ACAS Xu, MNIST, CIFAR, and GTSRB experiments, including 67 versus 42 solved ACAS Xu cases.
• The results show relational branching is not universally best: it excels with wider perturbations but can underperform individual splitting on models such as CIFAR with many unstable ReLUs, motivating hybrid strategies.

Relational verification of neural networks—certifying properties that relate multiple inferences of the same network, such as global robustness—remains substantially less developed than verification against local robustness. The paper "Branch and Bound for Relational Verification of Neural Networks" (2608.13118) addresses the incompleteness of existing over-approximation-based relational verifiers by introducing SaBRe, a branch-and-bound (BaB) framework whose distinguishing feature is branching on relational neurons (the differences between corresponding neurons across two inferences) rather than on individual neurons. The work is motivated by the observation that relational specifications are hyperproperties: verifying global robustness requires reasoning jointly about two inputs $x, x'$ with $\|x - x'\|_\infty \le \delta$ and bounding $|N_k(x) - N_k(x')| \le \epsilon$, which cannot be handled directly by single-inference verifiers.

Background and motivation

The starting point is the RaVeN verifier (2608.13118), which propagates bounds over a convex abstraction domain covering both individual neurons and relational neurons $\Delta^{(i)}_j = y^{(1,i)}_j - y^{(2,i)}_j$. RaVeN is sound and efficient but incomplete: it may return spurious counterexamples (false alarms) because its linear relaxation of the ReLU applied to a relational neuron introduces unreachable regions. Classic BaB refines such approximations by splitting unstable ReLUs into their positive- and negative-input cases, making each branch exact.

The paper's motivating example shows why individual-neuron splitting is suboptimal in the relational setting. On a small network, splitting an individual neuron refines an output-difference bound to $[-0.15, 0.15]$, whereas splitting the corresponding relational neuron yields $[-0.067, 0.15]$—a strictly tighter bound. Since relational neurons directly determine the quantity being bounded ($|\Delta^{(L)}_k|$), splitting them eliminates relaxation error exactly where it matters. This observation drives the design of SaBRe, which branches exclusively on relational neurons by adding constraints of the form $\Delta^{(i)}_j \le 0$ or $\Delta^{(i)}_j \ge 0$ to form sub-problems.

The SaBRe framework

SaBRe generalizes the verification problem with relational constraint sequences—conjunctions of sign constraints on relational pre-activations—and solves each sub-problem with an LP-based approximation verifier built on RaVeN's propagation rules, including its heuristics that exploit individual-neuron bounds to tighten relational bounds. The BaB loop applies this verifier; if it fails, the returned counterexample is validated against the network (real counterexamples terminate with "unsafe"), and otherwise a relational neuron is selected and split, pushing two new sub-problems onto the queue. Verification succeeds when the queue empties.

The core technical contribution is the relational neuron selection strategy, extending BaBSR's dual-based selection to the relational setting. The authors derive the Lagrangian dual of the relational LP via KKT conditions. Because strong duality holds for the LP (all ReLUs replaced by linear relaxations), the dual objective soundly over-approximates the output difference bound and decomposes into neuron-wise terms. Dual variables propagate backward through three parallel channels—one per inference and one for the difference—with coefficients determined by eleven cases combining the pre-activation states of both individual neurons and the relational neuron. Splitting a candidate relational neuron changes these coefficients, and the resulting change in the dual objective can be computed in constant time from quantities already available during dual construction. This yields an efficient estimate of which split maximizes bound refinement, without solving any additional optimization problems.

Experimental evaluation

The evaluation covers 817 instances across ACAS Xu, MNIST-F, MNIST-C, CIFAR, and GTSRB, comparing against RaVeN, two individual-splitting baselines (ClasIS, mirroring RABBit's strategy adapted to global robustness, and DualIS), and a random-selection ablation (RandRS). Key results:

• Against RaVeN: SaBRe substantially increases solved instances beyond what RaVeN alone certifies—for example, 67 vs. 42 on ACAS Xu and 33 vs. 9 on GTSRB—at the cost of more sub-problems and runtime.
• Relational vs. individual splitting: SaBRe dominates ClasIS and DualIS on four of five benchmarks. On ACAS Xu it solves 67 instances versus 12 (ClasIS) and 9 (DualIS), with roughly half the time ratio (34.24% vs. ~87–89%); on MNIST-F it doubles the solved count while halving sub-problems.
• A notable exception: on CIFAR, individual splitting outperforms SaBRe overall (31/28 vs. 23 solved). A fine-grained analysis shows the effect is perturbation-dependent: at $d_{\text{eps}} = 1/256$, individual splitting wins decisively (16 vs. 5 solved), but at $\|x - x'\|_\infty \le \delta$0 relational splitting wins (8 vs. 4). The authors attribute this to the CIFAR model having many more unstable ReLUs along the reasoning path, which dilutes the local benefit of each relational split when propagated bounds are narrow. This is an honest concession that relational splitting is not uniformly superior.
• Scalability: varying the fraction $\|x - x'\|_\infty \le \delta$1 of perturbed input dimensions, SaBRe is comparable at $\|x - x'\|_\infty \le \delta$2 but clearly superior from $\|x - x'\|_\infty \le \delta$3 onward (e.g., 40 vs. 28 solved at $\|x - x'\|_\infty \le \delta$4), consistent with the finding that wider relational bounds favor relational splitting.
• Selection strategy: against random selection (RandRS), the dual-based selection roughly halves sub-problems and time across benchmarks (e.g., 67 vs. 44 solved on ACAS Xu; 33 vs. 9 on GTSRB).
• Maximum verifiable perturbation $\|x - x'\|_\infty \le \delta$5: via binary search, SaBRe certifies larger regions than all baselines on most instances, with moderate but consistent effect sizes (Cohen's $\|x - x'\|_\infty \le \delta$6–$\|x - x'\|_\infty \le \delta$7 aggregated pairwise).

Limitations and open questions

Several limitations are stated explicitly. First, SaBRe remains incomplete—it is still an approximate method and cannot prove unsafety except through validated counterexamples. Second, the framework branches exclusively on relational neurons; the authors note that jointly considering individual and relational neurons could help (particularly on architectures like CIFAR with many unstable ReLUs), but deciding when to switch between the two is left open, as is a principled selection heuristic spanning both neuron types. Third, the advantage of relational splitting degrades when relational bounds are narrow or unstable-ReLU density is high, so the reported dominance does not extend uniformly to all architectures and perturbation regimes. Finally, the comparison with RABBit is indirect: because RABBit targets universal adversarial perturbations rather than global robustness, only its splitting strategy could be adapted (as ClasIS), not the full system.

Conclusion

This paper makes a focused contribution to an underexplored area: it reformulates BaB refinement for relational verification around relational neurons, and supplies the missing technical piece—a Lagrangian-dual-based selection criterion for those neurons derived from a full KKT analysis of the relational LP. The empirical evidence supports the central claim that relational splitting tightens output-difference bounds more effectively than individual splitting on most benchmarks, while candidly identifying the regime (narrow bounds, high instability) where it does not. The open question of hybrid individual/relational branching strategies is the most concrete direction the paper leaves unresolved.