Output Perturbation Defenses

• Output perturbation defenses are techniques that alter a model’s output (e.g., probability vectors, API responses, parameters) to obstruct adversarial gradient estimation and model theft.
• They employ methods such as deterministic smoothing, stochastic corruption, and noise injection across various settings to balance security and utility.
• These defenses present trade-offs, as their efficacy depends on precise threat models and can be undermined by adaptive attacks despite preserving overall functionality.

Output perturbation defenses are defense mechanisms that modify a model’s emitted signal rather than only its input. Depending on the setting, the perturbed object may be a softmax distribution, a returned probability vector, an API text response, a feature vector, or the final parameter vector of a private learner. The common design objective is to preserve enough utility for ordinary use while degrading an attacker’s ability to estimate gradients, train a substitute, distill a student, or recover latent structure. The literature uses the term in several non-identical ways: some works treat output perturbation as deterministic smoothing of class probabilities, some as optimized or stochastic corruption of returned outputs, some as output-level response rewriting in LLM APIs, and some as adding privacy noise to learned parameters (Yuan et al., 2017, Kilcher et al., 2017, Jiang, 8 Mar 2026, Ma et al., 2019).

1. Taxonomic position and conceptual scope

In the survey taxonomy of adversarial defenses, output perturbation is not presented as a separate top-level family. The closest mechanism is network distillation, which modifies the classifier’s softmax outputs through temperature scaling,

$q_i = \frac{\exp(z_i/T)}{\sum_j \exp(z_j/T)},$

and is placed under proactive defenses alongside adversarial training and classifier robustifying (Yuan et al., 2017). In this usage, the defense acts on the model’s output distribution rather than on the input image.

A later output-level taxonomy for LLM distillation makes the category explicit. DistillGuard divides output-level defenses into perturbation, poisoning, and information throttling. In that framework, output perturbation is defined as modifying the teacher response while preserving approximate meaning,

$\tilde{y} = P(y,\alpha),$

where $y$ is the original teacher response, $\tilde y$ is the returned response, $P$ is a separate paraphrasing model, and $\alpha \in [0,1]$ is perturbation strength (Jiang, 8 Mar 2026).

These formulations imply a broad but coherent family. One branch smooths or distorts probability outputs to reduce sensitivity or stealability. Another perturbs API responses so that downstream learning becomes unreliable. A third branch, in differential privacy, adds noise to the released model itself. A plausible implication is that “output perturbation defense” is best understood functionally—defense by modifying the released output channel—rather than by a single mathematical template.

2. Probability-space defenses in classifiers

The earliest output-side mechanisms in this set operate on classifier probabilities. In network distillation, the softened output distribution is intended to reduce sensitivity to small perturbations, and the survey reports that it reduced the success rate of JSMA by 0.5% on MNIST and 5% on CIFAR-10 (Yuan et al., 2017). The same survey is explicit that this effect is partial rather than universal.

A more aggressive variant is to perturb the returned label distribution adversarially. In “The best defense is a good offense,” the defended output is

$$y_x^* = y_x + \epsilon~\mathrm{sign}(\nabla_y \|\nabla_\theta c(F_\theta, x, y_x)\|_2),$$

so the API returns slightly wrong labels that are optimized to make substitute training infeasible (Kilcher et al., 2017). The perturbation is chosen so that the attacker’s training gradients become large and destabilizing, while the top prediction remains effectively unchanged for ordinary use: the paper reports that the argmax remains unchanged in over 99.9% of cases on MNIST, and essentially 100% on CIFAR10 and SVHN (Kilcher et al., 2017).

A closely related black-box model-stealing defense perturbs each class-probability vector while preserving normalization and ranking. Its Reverse Sigmoid form is

$$\hat{y}^i_j = \alpha^i \left( y^i_j - \beta\left(s(\gamma s^{-1}(y_j^i)) - 1/2\right) \right).$$

The defense aims to make probability inversion ambiguous while keeping top-1 behavior almost intact. Under 19,200 queries, stolen-model agreement drops from 0.82 to 0.40 on CIFAR-10, from 0.93 to 0.68 on MNIST, and from 0.48 to 0.02 on CIFAR-100; the abstract summarizes the effect as degrading the accuracy of the stolen model at least 20%, or requiring up to 64 times more queries while keeping the accuracy of the protected model almost intact (Lee et al., 2018).

Another line adds white noise directly to the DNN output. The released output becomes

$$F({\bf x}) + {\bf v}, \qquad {\bf v}\sim{\cal N}({\bf 0},\sigma^2{\bf I}),$$

with the defense targeting query-based black-box attacks that reconstruct gradients from tiny output differences. The key bound is

${\rm SNR}\le \frac{2L^2\beta^2}{\sigma^2}.$

The paper states that adding slight white noise with a standard deviation less than 0.01 is enough to increase QC by many orders of magnitude without introducing any noticeable classification accuracy reduction, and gives query-ratio estimates at $\tilde{y} = P(y,\alpha),$0 of approximately $\tilde{y} = P(y,\alpha),$1 on MNIST, $\tilde{y} = P(y,\alpha),$2 on CIFAR10, and $\tilde{y} = P(y,\alpha),$3 on ImageNet (Aithal et al., 2021).

3. Output perturbation against model theft and distillation

For black-box extraction attacks, output perturbation attacks the information channel used for imitation. The model-stealing literature emphasizes that probability vectors can reduce the samples needed for stealing by 50–100× compared to labels alone, which makes probability perturbation especially attractive as a defensive lever (Lee et al., 2018). In the same spirit, “slightly wrong labels” corrupt substitute training targets rather than the attacker’s inputs, which reframes defense as adversarial output poisoning rather than adversarial training of the victim model (Kilcher et al., 2017).

The same logic has been extended to LLM APIs. DistillGuard evaluates paraphrasing-based perturbation at three strengths—A02 with $\tilde{y} = P(y,\alpha),$4, A03 with $\tilde{y} = P(y,\alpha),$5, and A04 with $\tilde{y} = P(y,\alpha),$6—using a separate Qwen2.5-7B-Instruct paraphraser. The distilled-student results remain close to baseline: average DE ranges from 0.978 to 1.012, while average DC ranges from 0.030 to 0.070. The strongest perturbation, $\tilde{y} = P(y,\alpha),$7, does not help; it slightly improves MATH and MT-Bench relative to baseline (Jiang, 8 Mar 2026). The paper’s explanation is the perturbation limitation: if a defense preserves correctness and approximately preserves semantics, then it preserves the distillation signal.

The threat-model critique in later LLM work makes the same point from a different angle. “What Does It Mean to Break a Distillation Defense?” argues that output perturbation defenses cannot be meaningfully judged without an explicit threat model $\tilde{y} = P(y,\alpha),$8, consisting of query budget, data budget, and interface profile (Libon et al., 23 Jun 2026). Its case study defense, Antidistillation Sampling (ADS), perturbs the next-token distribution as

$\tilde{y} = P(y,\alpha),$9

Under the original weak attacker setting, ADS appeared strong at $y$0. Under stronger but still realistic assumptions—repetition deletion, max-$y$1 resampling with $y$2, matched-utility budgeting, and a one-token prefill "First"—the defense weakens substantially; the prefill alone improves student accuracy by about 5 percentage points (Libon et al., 23 Jun 2026). This suggests that output perturbation defenses in API settings are inseparable from the exact interface and budget assumptions under which they are evaluated.

4. Differential privacy and parameter-space output perturbation

In differential privacy, output perturbation has a distinct and older meaning. The learner first solves the regularized empirical-risk problem and then adds noise to the final parameters: $y$3 This is the output-perturbation counterpart to objective perturbation, and it is studied as a defense against data poisoning because DP stability limits how much the released model can change when a small number of training items is modified (Ma et al., 2019).

For pure $y$4-DP, the paper proves that for nonnegative attack cost,

$y$5

The corresponding corollary gives a lower bound on the poisoning budget needed to achieve a multiplicative reduction by factor $y$6: $y$7 For $y$8-DP, the guarantee weakens through an additive $y$9-dependent term, and the paper is explicit that the attacker may eventually be able to drive the cost all the way to zero (Ma et al., 2019).

The empirical results make the same point operationally. Output perturbation provides a formal defense against poisoning, but attacks become effective when the attacker can poison sufficiently many training items. On the red wine quality regression task, the attack cost is significantly reduced on output perturbation even when the attacker poisons only 100/1598 \approx 6.3\% of the data (Ma et al., 2019). In this literature, output perturbation is therefore a quantitatively analyzable but incomplete robustness mechanism.

5. Adjacent domains and nonstandard uses

Several later works use the language of output perturbation in adjacent ways. In efficient LLM inference, KeepKV treats output perturbation as the degradation introduced by KV-cache compression. The attention output at step $\tilde y$0 is

$\tilde y$1

and prior merging methods are shown to cause Attention Sag. KeepKV introduces Electoral Votes and Zero Inference-Perturbation Merging, with a theorem stating

$\tilde y$2

The reported result is more than $\tilde y$3 inference throughput improvement while keeping superior generation quality even with 10% KV cache budgets (Tian et al., 14 Apr 2025). This is not an anti-adversarial output perturbation defense in the API sense; it is a defense against compression-induced perturbation of the model’s own output.

In adversarial-example defense, there are neighboring inference-time methods that perturb the attacked input so that the final output recovers. One method observes that the probability of the correct result increases by applying small first-order perturbations generated for non-predicted class labels to adversarial examples. It randomly selects class labels, generates small first-order perturbations, adds them together, clamps them onto a specified space, and adds the obtained perturbation to the adversarial example. The method is applied at inference time and does not require retraining or finetuning the model (Zhang et al., 2021). Another, Hedge Defense, searches within a small defensive radius,

$\tilde y$4

and on CIFAR10 boosts a robust model from 66.16% to 72.66% under AutoAttack, including 71.76% to 83.30% against Square attack (Wu et al., 2021). These are perturbation defenses at inference time, but they act on the attacked example rather than on the released output distribution.

The term also appears in proactive media protection. In voice privacy, perturbation-based defenses produce protected speech

$\tilde y$5

with the intended goals of imperceptibility, verifiability, and inimitability, but VocalBridge shows that a latent diffusion-bridge purifier can recover cloneable voices from protected speech (Abbasihafshejani et al., 5 Jan 2026). In facial-manipulation defense, AEGIS injects adversarial perturbations into the DDIM denoising trajectory rather than clipping pixels under a fixed $\tilde y$6 bound; it reports 100% DSR on StarGAN for CelebA, FFHQ, LFW, 100% DSR on SimSwap for CelebA and LFW, and DSR = 100% with ID sim = 0.070 on Arc2Face (Li et al., 2 Apr 2026). These cases are best read as output- or release-side perturbation of content artifacts rather than perturbation of classifier probabilities.

6. Evaluation difficulties, adaptive attackers, and recurring failure modes

A recurring conclusion across the literature is that output perturbation defenses are highly sensitive to attacker adaptation. The survey on adversarial examples states that most defenses only work for part of attacks and can fail against strong or unseen attacks (Yuan et al., 2017). A broad empirical study of perturbation defenses reports five main results: all input perturbation defenses, whether random or deterministic, are equivalent in their efficacy; attacks transfer between perturbation defenses; a tuned sequence of noise layers across a network provides the best empirical robustness; and perturbation based defenses offer almost no robustness to adaptive attacks unless these perturbations are observed during training (Dziedzic et al., 2020). Although that study concerns input perturbations rather than output perturbations narrowly construed, the warning about adaptive evaluation generalizes.

The model-theft and distillation literature shows analogous failure modes. DistillGuard finds that paraphrasing has low user cost but almost no protective effect, and the threat-model paper on ADS shows that changing $\tilde y$7, $\tilde y$8, or $\tilde y$9 can reverse the apparent evaluation outcome (Jiang, 8 Mar 2026, Libon et al., 23 Jun 2026). In encoder stealing, BESA targets perturbation-based feature defenses such as Top-K, Rounding (RD), and Noise Poisoning (NP), as well as their hybrids. Its detection module achieves generally over 98% accuracy and often nearly 100%, and it improves surrogate encoder accuracy by up to 24.63% under state-of-the-art defenses (Ren et al., 5 Jun 2025). In speech, VocalBridge’s cross-perturbation generalization indicates that purification need not know the exact perturbation family to work well (Abbasihafshejani et al., 5 Jan 2026).

Two misconceptions follow repeatedly from these results. The first is that preserving utility for ordinary users is sufficient for security. DistillGuard directly argues the opposite: if correctness and semantics are preserved, the distillation signal is preserved (Jiang, 8 Mar 2026). The second is that a single evaluation setting can certify robustness. The $P$0 framework rejects that view and treats threat-model specification as a minimum reporting standard for output perturbation defenses (Libon et al., 23 Jun 2026).

Taken together, the literature portrays output perturbation defenses as a technically diverse family with a shared strategic premise: defensive control of the output channel. Their strongest forms are highly targeted, interface-aware, and threat-model-specific; their weakest forms reduce to cosmetic noise. The research trend suggests that future progress depends less on generic perturbation and more on explicit attacker models, adaptive evaluation, and mechanisms that remain effective even when the perturbation process itself becomes learnable or partially invertible.