Randomized Smoothing: Certified Robustness

• Randomized smoothing is a certifiable defense that converts any base classifier into a robust majority vote classifier by injecting controlled random noise.
• It supports diverse threat models and smoothing distributions, providing concrete certified guarantees for deep neural networks, regression, and structured outputs.
• Scalable techniques such as surrogate estimation and adaptive variance selection improve efficiency while balancing clean accuracy and certified robustness.

Randomized smoothing is a certifiable defense technique that transforms an arbitrary base classifier into a new, smoothed classifier which is provably robust to adversarial input perturbations. By introducing random noise to the input and aggregating the base classifier’s predictions, randomized smoothing yields concrete certified robustness guarantees for high-dimensional models, including deep neural networks, with minimal assumptions on the classifier’s internal structure. Over the past several years, randomized smoothing has expanded to support diverse threat models, smoothing distributions, and rich robustness–accuracy trade-offs in both classification and regression.

1. Foundational Principles of Randomized Smoothing

Randomized smoothing constructs a robust “majority vote” classifier $g$ from a base classifier $f$. For a target input $x\in\mathbb R^d$:

$$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$

where $\mathcal D$ is a smoothing distribution, commonly isotropic Gaussian noise $\mathcal N(0,\sigma^2 I)$. The approach was formalized by Cohen et al. (2019), who proved that with Gaussian smoothing, $g(x)$ is guaranteed to remain unchanged under any adversarial perturbation $\|\delta\|_2 < R$, where

$$R = \frac{\sigma}{2} \left( \Phi^{-1}(p_A) - \Phi^{-1}(p_B) \right)$$

$p_A$ and $f$0 are the probabilities (under noise) of predicting the top and runner-up classes, and $f$1 is the standard normal inverse CDF. The robustness guarantee is information-theoretic and model-agnostic—no assumptions are made about the internals of $f$2 as long as predictions can be computed pointwise (Yang et al., 2020, Kumar et al., 2020). This technique operates as a black-box wrapper and thus is widely applicable to deep networks and other complex predictors.

2. Extensions: Smoothing Distributions and Threat Models

The foundational framework has spurred numerous generalizations along several axes:

• Norms and Distributions: Duan et al. established that for any “nice” norm $f$3 (with a symmetric convex unit ball), the optimal smoothing distributions are uniform over the “Wulff crystal” of the dual norm. This encompasses Gaussian for $f$4, Laplace for $f$5, and exponential distributions for $f$6, yielding closed-form robustness formulas for each case. These results unified prior ad hoc smoothing schemes and enabled strong $f$7 and $f$8 certificates (Yang et al., 2020).
• Multiplicative and Transformation Smoothing: Beyond additive noise, Muravev & Petiushko designed smoothing over multiplicative transformation parameters (e.g., gamma correction) using Rayleigh or log-normal distributions. This enables certified robustness against semantic-preserving transformations—such as intensity scaling in images—and establishes new certificate formulas for such group actions (Muravev et al., 2021).
• Partial or Hierarchical Smoothing: Standard approaches add noise to all input dimensions. Hierarchical randomized smoothing introduces entity-wise subset selection, allowing smoothing over random subsets of input features (pixels, graph nodes), which is strictly stronger under “sparse” or structured threat models (e.g., $f$9-patch attacks). This framework leverages a two-level Neyman–Pearson argument, yielding improved robustness–accuracy Pareto fronts for images and graphs (Scholten et al., 2023).
• Projected and Manifold Smoothing: Projected randomized smoothing, which introduces dimensionality reduction via linear projection (e.g., PCA), smooths in the low-dimensional manifold space and certifies robust “cylindrical” regions in the original input space. This eliminates off-manifold adversarial vulnerabilities and provides tractably larger certified regions in high dimensions (Pfrommer et al., 2023).

3. Input-Dependent and Per-Sample Randomized Smoothing

A central limitation of global-variance randomized smoothing is the inability to optimize the clean accuracy–robustness trade-off at different radii: small $x\in\mathbb R^d$0 improves clean accuracy at the cost of small certified radii, while large $x\in\mathbb R^d$1 boosts radii but degrades accuracy. Several works have overcome this constraint:

• Data-Dependent and Dual RS: Input-dependent variance smoothing selects $x\in\mathbb R^d$2 per sample to maximize the certified radius, yielding strictly better envelopes along the trade-off curve. To ensure soundness, a memory-enhanced scheme enforces non-overlapping balls with different labels (Alfarra et al., 2020). Dual RS further introduces a two-stage process: a variance estimator $x\in\mathbb R^d$3 predicts the optimal $x\in\mathbb R^d$4 per input (itself RS-smoothed to guarantee local constancy), and the classifier is smoothed using the chosen $x\in\mathbb R^d$5. This approach provides strong performance across both small and large radii, outstripping fixed-variance baselines even after accounting for modest computational overhead (Sun et al., 1 Dec 2025).
• Variational and Meta-Learned Smoothing: A variational framework applies a sample-wise noise selector $x\in\mathbb R^d$6 parameterized by a neural network, trained to optimize a KL-regularized ELBO. This enables post-hoc selection of any robustness–accuracy trade-off via a meta-learned selector, and the certified radius per input is $x\in\mathbb R^d$7 (Hase et al., 2024).
• Adaptivity and Instance-wise Search: Instance-wise randomized smoothing systematically searches for the optimal $x\in\mathbb R^d$8 per test example and adapts the training process to augment the base model for instance-wise smoothed robustness, leading to significant improvements in average certified radius and accuracy (Chen et al., 2021).

4. Computational Scalability and Sampling-Efficient Certification

Randomized smoothing is statistically driven—robustness certificates require high-confidence bounds on class probabilities, necessitating large Monte Carlo sample sizes ($x\in\mathbb R^d$9–$$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$0) to achieve tight radii, especially at high dimension or for finely separated classes.

• Surrogate and Accelerated Smoothing: Surrogate neural networks can be trained to mimic the Monte Carlo estimates of the class-count vector, drastically reducing per-input certification from $$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$1 to $$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$2. Empirical studies show that surrogate-based certificates closely track state-of-the-art high-sample randomized smoothing with up to 600$$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$3 speedup (Bhardwaj et al., 2024).
• Double Sampling and Dimension Scalability: Double Sampling Randomized Smoothing (DSRS) exploits multiple smoothing distributions, adding an auxiliary set of samples from an alternative noise family to yield dual constraints. Under concentration assumptions, DSRS can certify $$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$4 radii scaling as $$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$5, overcoming the curse of dimensionality that afflicts single-noise schemes (Li et al., 2022).
• Incremental Certification: When models are slightly modified (e.g., quantized or pruned), the incremental randomized smoothing procedure allows retargeted certification by reusing cached noise samples. Disparity estimation between original and modified models is statistically efficient and enables up to 3–4$$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$6 certification speedup without significant loss in certified accuracy (Ugare et al., 2023).
• Scaling Laws and Finite-Sample Effects: Precise finite-$$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$7 theory quantifies the certified radius and accuracy drop as $$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$8, enabling tight control of certificate loss versus computational effort in both standard and VLM/jailbreak settings (Seferis et al., 19 Sep 2025).

5. Randomized Smoothing Beyond Classification

Randomized smoothing has been extended to additional settings:

• Regression: RS-Reg introduces probabilistic robustness certificates for regression tasks, defining robustness as the probability the (smoothed) output remains in a user-tolerated region under bounded $$g(x) = \arg\max_{c\in\mathcal Y}\; \Pr_{\epsilon\sim\mathcal D} [\, f(x+\epsilon) = c \,]$$9 perturbations. Asymptotic and finite-sample certificates, including incomplete beta-function bounds for bounded outputs, offer formal guarantees for regression pipelines, though only for $\mathcal D$0 attacks (Rekavandi et al., 2024).
• Generative and Vision-LLMs: Smoothing has been generalized from discrete label outputs to sequence or structured outputs by chaining the base model with a discrete oracle classifier. This enables certified robustness for VLMs, content moderation, and discrete action selection, provided oracle error rates can be controlled. New scaling laws tie oracle error, sample size, and certified radii (Seferis et al., 19 Sep 2025).
• Confidence Certificates: Advanced theory leverages the entire distribution of smoothed confidence scores rather than just top-class votes, yielding stronger, margin-based certified radii for both prediction confidence and label stability (Kumar et al., 2020).

6. Fundamental Limits and Future Directions

The robustness of randomized smoothing is fundamentally limited by the amount of information available from the smoothed classifier’s output statistics. No-go theorems (e.g., via Banach space cotypes) rigorously prove that, using only label probabilities, no smoothing-based certification can guarantee nontrivial $\mathcal D$1 radii ($\mathcal D$2), and for $\mathcal D$3 and $\mathcal D$4, the certifiable radius vanishes with dimension (Yang et al., 2020, Ettedgui et al., 2022). However, recent work shows these limits arise not from smoothing per se, but from “single-noise” certification: by probing with a family of noise distributions and exploiting further statistics (e.g., boundary curvature), one can approach the true “perfect” robust radius (Ettedgui et al., 2022).

Ongoing challenges and extensions include:

• Tight, efficient certificates for partial or hierarchical smoothing.
• Richer per-sample or input-adaptive schemes that maximize instance-wise radii without heavy search.
• Smoothed certification in regression, structured outputs, or under discrete, semantic, or composite transformations.
• Efficient and reliable surrogate-based certification under varying data and model regimes.
• Integration with advanced adversarial training and defense protocols.

Randomized smoothing stands as the dominant certifiable defense with broad applicability and a continually expanding theoretical and practical toolkit, though its ultimate limitations and optimal variants are still active research topics.