Exact Certification of Neural Networks and Partition Aggregation Ensembles against Label Poisoning

Abstract: Label-flipping attacks, which corrupt training labels to induce misclassifications at inference, remain a major threat to supervised learning models. This drives the need for robustness certificates that provide formal guarantees about a model's robustness under adversarially corrupted labels. Existing certification frameworks rely on ensemble techniques such as smoothing or partition-aggregation, but treat the corresponding base classifiers as black boxes, yielding overly conservative guarantees. We introduce EnsembleCert, the first certification framework for partition-aggregation ensembles that utilizes white-box knowledge of the base classifiers. Concretely, EnsembleCert yields tighter guarantees than black-box approaches by aggregating per-partition white-box certificates to compute ensemble-level guarantees in polynomial time. To extract white-box knowledge from the base classifiers efficiently, we develop ScaLabelCert, a method that leverages the equivalence between sufficiently wide neural networks and kernel methods using the neural tangent kernel. ScaLabelCert yields the first exact, polynomial-time calculable certificate for neural networks against label-flipping attacks. EnsembleCert is either on par, or significantly outperforms the existing partition-based black box certificates. Exemplary, on CIFAR-10, our method can certify upto +26.5% more label flips in median over the test set compared to the existing black-box approach while requiring 100 times fewer partitions, thus, challenging the prevailing notion that heavy partitioning is a necessity for strong certified robustness.

• The paper proposes EnsembleCert, aggregating white-box certificates across partitions to deliver tight ensemble-level robustness guarantees against label flipping.
• It introduces ScaLabelCert, a polynomial-time method that exploits Neural Tangent Kernel equivalence for exact certification of wide neural networks.
• Experimental evaluations on benchmarks like CIFAR-10 show improved certified robustness with significantly fewer partitions compared to traditional black-box approaches.

Exact Certification of Neural Networks and Partition Aggregation Ensembles Against Label Poisoning

Introduction and Threat Model

Label-flipping attacks present a critical vulnerability in supervised learning, where adversaries perturb training labels to degrade model generalization or induce targeted misclassifications. Most prior certified defenses against such poisoning attacks are ensemble-based, including partition-aggregation (e.g., Semi-Supervised Deep Partition Aggregation/SS-DPA) and randomized smoothing approaches. However, these methods universally treat base learners as black-boxes, resulting in loose (overly conservative) robustness guarantees and failing to capitalize on available internal model structure.

This paper introduces EnsembleCert, the first certification framework for partition-aggregation ensembles that aggregates white-box, per-partition certificates into tight, ensemble-level guarantees. This is accompanied by ScaLabelCert, a polynomial-time exact certification methodology for neural networks with sufficient width, which leverages the equivalence between infinite-width NNs and kernel methods defined by the Neural Tangent Kernel (NTK). The paper establishes both a theoretical regime (via integer program reduction) in which ensemble aggregation can be performed in polynomial time and a scalable pipeline for extracting white-box information from modern networks.

Figure 1: Two-step approach of EnsembleCert, aggregating white-box certificates from base classifiers to achieve ensemble-level robustness guarantees.

Methodological Framework

EnsembleCert: White-box Certification for Partition Aggregation

Partition-aggregation ensembles (SS-DPA) divide the training set into $N_p$ disjoint partitions, each yielding a base classifier. At inference, predictions are made by majority voting. Standard certificates count the minimal set of base learners whose votes must be "flipped" to change the ensemble's decision, pessimistically assuming each can be flipped by only one poisoned label; this is the black-box assumption.

EnsembleCert utilizes access to $\rho_i^c$ for all partition/class pairs—the minimal label flips needed to drive the $i$th base classifier to predict class $c$. The problem of finding the minimal global poisoning budget to change the overall prediction is formulated as an integer program, in which one determines the minimal aggregate cost to reassign sufficient votes away from the true class. A key result is the reduction of this combinatorial problem to a Multiple-Choice Knapsack Problem (MCKP), yielding a certificate computable in polynomial time with respect to the ensemble size and number of classes.

ScaLabelCert: Polynomial-time Exact Certification for Wide NNs

Standard (e.g., MILP- or gradient-bound-based) white-box certificates for neural networks do not scale to large datasets or multi-class tasks—the former due to NP-hardness, the latter due to vacuous bounds in practical regimes.

ScaLabelCert builds on the LabelCert paradigm, exploiting the correspondence between infinite-width neural networks (NTK regime) and kernel SVMs (for hinge loss) or kernel ridge regression (for squared loss). For soft-margin SVMs with a sufficiently small regularization parameter $C$, the certificate problem simplifies—dual variables take on fixed values and the key question becomes "which label assignments cross the decision margin?" This allows for a greedy, provably optimal attack search in the binary setting, and a relaxation to tractable bounds in the multi-class case. The result is a certification method which, for wide NNs trained under an appropriate regime, can compute the exact robustness certificate in polynomial time.

Experimental Evaluation

The experimental section presents an extensive evaluation of EnsembleCert and ScaLabelCert using MNIST, CIFAR-10, and a binary MNIST 1-vs-7 task with both NTK-based SVMs and kernel regression. Feature extraction uses state-of-the-art unsupervised pipelines (RotNet for MNIST, SimCLR for CIFAR-10), and experiments consider varying aggregation partition counts and regularization regimes.

Key Observations

• For kernel SVM base learners (with small $C$), the certified robustness shows invariance or only weak decay with respect to increasing the number of partitions; strong guarantees are realizable with very small ensembles.
• For kernel regression, optimal certified robustness is observed at low to moderate partition counts, particularly under strong regularization; beyond this, excessive partitioning reduces ensemble robustness, even as it increases the number of base learners.
• On CIFAR-10 (using kernel regression, $\lambda=100$), EnsembleCert can certify up to +26.5% more label flips in the median compared to black-box partition aggregation with $10$ partitions—using $100\times$ fewer partitions than the best-performing black-box aggregation.

Figure 3: Performance of EnsembleCert on CIFAR-10 using kernel regression ($\lambda=100$); few partitions suffice for strong robustness, outperforming heavy partitioning.

• Stand-alone models (no partitioning) can often provide stronger certified accuracy than deeply partitioned ensembles, especially with robust base learners.

Figure 4: Certified accuracy comparison for CIFAR-10 SVM—stand-alone models maintain accuracy under higher flip budgets than partitioned ensembles.

• ScaLabelCert substantially outperforms recent gradient-bound-based white-box certificates for finite-width models (see Figure 5 in the supplement), corroborating the utility of exact (rather than loose) certificate regimes in practical settings.

Analysis and Theoretical Implications

The findings of this study carry important implications:

• Partitioning is not universally beneficial: The widely assumed necessity of deep partitioning for certified robustness is contradicted in the presence of robust base classifiers. Excessive partitioning can, in fact, degrade ensemble robustness while incurring prohibitively high computational costs.
• Role of Base Model Robustness: The certified accuracy of the ensemble is determined not only by aggregation but by the underlying robustness of the base classifiers. The observation that singleton (no aggregation) models can outperform partitioned models (in terms of certification) highlights the need to reconsider aggregation strategies.
• Framework Generalizability: While ScaLabelCert is best suited for infinite-width networks, EnsembleCert is compatible with any certification mechanism yielding per-partition, per-class robustness bounds, including randomized smoothing and convex-relaxation certificates for finite-width models.

Practical Considerations and Future Directions

The certificates introduced are polynomial-time computable and scale to common benchmarks. For practitioners, this enables routine deployment of certified training pipelines with realistic (rather than toy) datasets. However, the equivalence to kernel methods for infinite-width NNs means that tightness for finite-width models is not guaranteed; empirical results suggest certificates are tight and practical, but future work may develop more refined certificates for realistic model architectures.

Robustness against other poisoning settings, such as clean-label attacks, is also readily addressed within the EnsembleCert framework—provided white-box injection routines for base classifier certificates can be constructed. Another important direction is a deeper theoretical analysis of the tradeoff between ensemble size, base learner robustness, and system-level certification versus empirical performance.

Conclusion

This paper presents the first certification framework—EnsembleCert—for partition aggregation ensembles that aggregates white-box, per-partition certificates in polynomial time, delivering significantly tighter robustness guarantees against label-flipping attacks. The supporting ScaLabelCert method yields the first exact, scalable certification procedure for infinite-width NNs, providing practical certificates for widely used image benchmarks. These results call into question the conventional wisdom of heavy partitioning and provide new insight into the design of robust learning pipelines. The framework is open for further extension to more general models and attack types.

• "Exact Certification of Neural Networks and Partition Aggregation Ensembles against Label Poisoning" (2604.11416)