Poisoned Minima in Neural Networks

• The paper identifies poisoned minima in neural network training, where backdoor triggers are learned rapidly, creating vulnerabilities.
• It explains a dual-task formulation that separates clean and poisoned losses, enabling precise isolation using gradient ascent techniques.
• Empirical results show that Anti-Backdoor Learning drastically lowers attack success rates while maintaining high clean accuracy across datasets.

Poisoned minima refer to local optima in the loss landscape of neural network training that correspond to parameter values where the model perfectly or near-perfectly fits not only the clean data but also embedded backdoor triggers, rendering the model vulnerable to targeted failures. Such minima are a central concern in the context of backdoor attacks, where adversaries introduce “backdoor-poisoned” samples into the training set so that the resulting model behaves maliciously under specific, trigger-activated inputs while exhibiting benign performance on standard data.

1. Backdoor Attacks and the Loss Landscape

Backdoor attacks are a form of data poisoning that modifies a small fraction of the training dataset with an adversarial “trigger” and (optionally) relabeling to a target class. Let the complete dataset be $\mathcal{D} = \mathcal{D}_{c} \cup \mathcal{D}_{b}$, where $\mathcal{D}_{c}$ is the clean subset and $\mathcal{D}_{b}$ denotes the poisoned subset, both of which are unknown to the defender. The empirical risk minimized during standard training is

$$\mathcal{L}_{\text{std}}(\theta) = \mathcal{L}_{\text{clean}}(\theta) + \mathcal{L}_{\text{bd}}(\theta)$$

where $\mathcal{L}_{\text{clean}}$ and $\mathcal{L}_{\text{bd}}$ are the average losses for clean and backdoored samples, respectively.

Poisoned minima arise because backdoor examples present a trivial subtask – all poisoned inputs share a highly regularized mapping (trigger $\to$ target label) – and are thus learned extremely fast by standard deep networks, driving $\mathcal{L}_{\text{bd}}$ to zero within a few epochs. This efficient fitting leads to local minima where the model has memorized both the main task and the backdoor, often with minimal impact on overall clean accuracy (Li et al., 2021).

2. Dual-Task Formulation and “Fast-Learn” Dynamics

This challenge is formalized as a dual-task problem:

• Clean-task loss:

$$\mathcal{L}_{\text{clean}}(\theta) = \mathbb{E}_{(x,y) \sim \mathcal{D}_{c}}\, \ell(f_\theta(x), y)$$

• Backdoor-task loss:

$$\mathcal{L}_{\text{bd}}(\theta) = \mathbb{E}_{(x,y) \sim \mathcal{D}_{b}}\, \ell(f_\theta(x), y)$$

The “fast-learn” property of backdoored samples enables sharp separation in loss dynamics during initial training epochs. The clean-task requires learning complex class boundaries and thus converges relatively slowly, while the backdoor task (direct trigger-to-label mapping) converges much faster, leading to a gap exploitable by defenses (Li et al., 2021).

3. Anti-Backdoor Learning (ABL): Algorithmic Mechanisms

Anti-Backdoor Learning (ABL) provides a defense by restructuring training to avoid convergence to poisoned minima. ABL operates in two main stages:

Stage 1: Local Gradient Ascent (LGA) “Flooding”

• For epochs $\mathcal{D}_{c}$0 and per-example loss threshold $\mathcal{D}_{c}$1:
  • If $\mathcal{D}_{c}$2, apply gradient ascent to increase the loss, else descend.
  • Batch loss:

  $\mathcal{D}_{c}$3 - This maintains a separation between quickly-learned (typically backdoor) and slowly-learned (clean) points in the loss space.

Isolation: After $\mathcal{D}_{c}$4 epochs, all training examples are sorted by loss, and the bottom $\mathcal{D}_{c}$5\% (e.g., 1\%) with lowest loss are isolated as $\mathcal{D}_{c}$6 (suspected poisoned).

Stage 2: Global Gradient Ascent (GGA) to Unlearn

• Epochs $\mathcal{D}_{c}$7 continue training with:

$\mathcal{D}_{c}$8

(with $\mathcal{D}_{c}$9 in practice)

• Descent occurs on $\mathcal{D}_{b}$0 while ascent is applied on $\mathcal{D}_{b}$1, forcing the model to unlearn any memorized backdoor trigger-to-target correlations (Li et al., 2021).

4. Theoretical Intuition for Avoidance of Poisoned Minima

Two core weaknesses of backdoor attacks underpin ABL:

• Fast-learn property: Since backdoor samples quickly achieve near-zero loss, “loss flooding” in LGA keeps their loss above threshold, enabling high-precision separation from slower-learned clean samples.
• Class-tied property: Backdoor samples are tied to a specific target class, and even isolating a small fraction suffices—gradient ascent on this isolated set severs the trigger-target association for the entire poisoned subset.

Empirically, Stage 1 yields high-precision isolation (frequently 100% for small $\mathcal{D}_{b}$2), and Stage 2 reliably “unlearns” the backdoor so the attack success rate approaches random chance, with minimal effect on clean accuracy. No formal convergence guarantees are provided (Li et al., 2021).

5. Empirical Evaluation and Practical Effectiveness

Experiments are conducted on CIFAR-10, GTSRB, and an ImageNet-12 subset with models including WideResNet-16-1 and ResNet-34. The evaluations cover ten attack variants—dirty-label pixel attacks (BadNets, Trojan, Blend, Dynamic), clean-label pixel attacks (SIG, CL), and feature-space attacks (FC, DFST, LBA, CBA). Metrics include Clean Accuracy (CA) and Attack Success Rate (ASR).

A summary of results is shown:

Dataset	No Defense (ASR, CA)	NAD (ASR, CA)	ABL (ASR, CA)
CIFAR-10	97.7%, 83.6%	20.4%, 80.4%	7.7%, 84.8%
GTSRB	99.4%, 96.8%	19.2%, 87.0%	7.3%, 95.3%
ImageNet-12	99.6%, 85.5%	13.4%, 81.8%	6.0%, 86.9%

Under higher poisoning rates (up to 50–70%), isolating as little as 1% of the dataset can substantially reduce ASR from nearly 100% to single or low-double digit percentages, with clean accuracy dropping only a few points.

6. Significance and Implications

The concept of poisoned minima formalizes the risk of overfitting to adversarially crafted patterns in modern DNN training regimes, especially under empirical risk minimization. ABL demonstrates that by exploiting characteristic loss dynamics and the structural simplicity of backdoor triggers, it is possible to systematically avoid convergence to such minima without requiring explicit knowledge of which data are poisoned.

A plausible implication is that similar “flooding” or constrained optimization frameworks could apply to other forms of easy-to-learn spurious correlations or shortcut learning in deep models. The dichotomy between fast-learn triggers and slow-learn class structure, and the utility of even coarse isolation followed by adversarial loss manipulation, offers a concrete strategy for robust learning under contaminated data distributions (Li et al., 2021).