---
title: Adversarial Robustness Toolbox (ART)
url: https://www.emergentmind.com/topics/adversarial-robustness-toolbox
type: topic
---

# Adversarial Robustness Toolbox (ART)

Adversarial Robustness Toolbox (ART) is an open-source Python library that provides a unified, framework-agnostic environment for evaluating and enhancing the adversarial robustness of machine learning (ML) models. ART implements state-of-the-art adversarial attacks, defenses, and certified robustness metrics across diverse ML architectures, including DNNs, GBDTs, SVMs, random forests, logistic regression, Gaussian processes, and tree ensembles. Its modular API supports both research and production deployment of adversarial robustness evaluation, benchmarking, and automated defense strategies [1807.01069].

## 1. Core Principles and Architectural Overview

ART is structured around extensibility, modularity, and framework independence. All attacks, defenses, and robustness metrics are decoupled and interact via standardized abstract base classes (`Classifier`, `Attack`, `Preprocessor`, `Detector`, `Metric`). The core classifier interface wraps models from TensorFlow (v1/v2), Keras, PyTorch, MXNet, Scikit-learn, XGBoost, LightGBM, CatBoost, GPy, and allows arbitrary black-box Python callables. This ensures that any supported adversarial method or defense can operate identically across backends.

Key architectural modules include:
- **classifiers/**: Unified wrappers with methods for prediction, gradient evaluation, and batch operations.
- **attacks/**: Gradient-based (white-box), score-based, and decision-based evasion and poisoning attacks.
- **defences/**: Model hardening (adversarial training), input preprocessing, and runtime detection.
- **metrics/**: Quantitative robustness measures (empirical robustness, CLEVER, loss sensitivity, certified bounds).
- **wrappers/**: For randomized smoothing and black-box gradient estimation.
- **detection/poison_detection/**: Detectors for adversarial and poisoned inputs.
- **data_generators/**: Interfaces and wrappers for batch generation and streaming.

Integration and batching are emphasized for scalability and GPU acceleration, and all modules are licensed under MIT for research and commercial use [1807.01069].

## 2. Implemented Adversarial Attacks

ART provides a comprehensive suite of attacks, all formulated within the "minimal perturbation" paradigm:
- **FGSM (Fast Gradient Sign Method)**: $x_{\mathrm{adv}} = \mathrm{clip}\big(x + \epsilon \cdot \mathrm{sign}(\nabla_x \mathcal{L}(f(x),y)), x_{\min}, x_{\max} \big)$ [1807.01069]
- **BIM (Basic Iterative Method)/PGD (Projected Gradient Descent)**: Iterative FGSM within an $\ell_p$-ball, with or without random initialization [1807.01069].
- **Carlini & Wagner L2/L∞**: Optimizes $\|x'-x\|_2^2 + c \cdot \ell(x')$, with $\ell(\cdot)$ a margin-based loss and binary search on $c$ [1807.01069].
- **DeepFool**: Iteratively approximates minimal perturbation to reach the closest decision boundary via linearization [1807.01069].
- **JSMA (Jacobian-based Saliency Map Attack)**: Iteratively perturbs most "salient" input features to achieve targeted misclassification [1807.01069].
- **Decision-based/Black-box attacks**: Boundary attack, HopSkipJump, ZOO, transfer attacks, Adversarial Patch, and more [1807.01069].
- **Certified/LBFGS/SLSQP attacks**: For tree models and explicit constraint optimization [1807.01069].

Attacks generalize across untargeted/targeted settings, arbitrary $\ell_p$-norms, and complex input domains (e.g., images, text, tabular data).

## 3. Supported Defenses and Robust Training

ART implements defenses across three broad categories:
- **Model Hardening**: Adversarial training augments batches with generated adversarial examples, using methods such as PGD or blended attacks for inner maximization [1807.01069]. Label smoothing is available to control gradient peaks.
- **Preprocessing Defenses**: Quantization (Feature Squeezing), input domain transforms (JPEG compression, spatial smoothing, thermometer encoding, total variation minimization, Gaussian augmentation), and learned generative projections (PixelDefend) [1807.01069].
- **Runtime Detection**: BinaryInputDetector and BinaryActivationDetector enable the training of classifiers on input or activation space for detection, while SubsetScanningDetector facilitates anomaly detection without supervision [1807.01069].

All methods are implemented as composable modules, typically wrapped around the base classifier. Chaining of multiple defenses is supported, and care is taken to ensure consistency of data domain and range (via `clip_values`) throughout the pipeline.

## 4. Evaluation Protocols and Robustness Metrics

ART standardizes reproducible robustness evaluation by providing:
- **EmpiricalRobustness**: Average minimal perturbation (in specified norm) required to cause misclassification for a given attack [1807.01069].
- **CLEVER (Cross Lipschitz Extreme Value for nEtwork Robustness)**: Locally estimates lower bounds on the minimal $\ell_p$-norm adversarial perturbation using extreme value theory [1807.01069].
- **Loss Sensitivity**: Measures the maximum gradient magnitude of the loss with respect to inputs, serving as a proxy for susceptibility to perturbation [1807.01069].
- **Certified Bounds**: For tree models and randomized smoothing-based techniques [1807.01069].

Benchmarks routinely report attack success rate, perturbation magnitude, and model accuracy on adversarially perturbed samples, with full hyperparameter reporting for reproducibility.

## 5. Usage Patterns and Code Examples

Typical ART workflows involve:
1. **Wrapping a pretrained model** with the appropriate classifier wrapper, specifying input normalization and clipping.
2. **Configuring and applying attacks**: Instantiate via the `Attack` class with relevant parameters (e.g., $\epsilon$, step size, number of iterations), and call `.generate(x, y)` on batches.
3. **Composing defenses**: Chain via the `defences` parameter at instantiation or wrap with `Preprocessor`/`AdversarialTrainer` classes.
4. **Calculating metrics**: Call the desired robustness evaluation method, passing in the defended classifier, attack(s), and relevant data [1807.01069].

Sample code for FGSM attack on a Keras model:
```python
from art.classifiers import KerasClassifier
from art.attacks import FastGradientMethod

classifier = KerasClassifier(model=keras_model, clip_values=(0,1))
attack = FastGradientMethod(estimator=classifier, eps=0.2)
x_adv = attack.generate(x=x_test, y=y_test)
```
Defended model with multiple preprocessors:
```python
from art.defences import FeatureSqueezing, JPEGCompression

fs = FeatureSqueezing(bit_depth=4, apply_predict=True)
jpeg = JPEGCompression(clip_values=(0,1), apply_predict=True)
classifier_defended = KerasClassifier(model=keras_model, clip_values=(0,1), defences=[fs, jpeg])
```
Certified bound evaluation (CLEVER):
```python
from art.metrics import clever_t

gamma = clever_t(classifier, x_test[0:1], target_class=3, p=2, nb_batches=10, batch_size=50, radius=0.5)
```
Full documentation and further tutorials accompany the codebase [1807.01069].

## 6. Integration, Limitations, and Best Practices

ART provides first-class support for batch operations and GPU acceleration (when supported by underlying frameworks). It is compatible with major ML and data science toolchains by design.

Best practices highlighted include:
- Always respecting valid input range via `clip_values`.
- Matching attack/defense strengths ($\epsilon$, etc.) to domain-specific perceptual or operational constraints.
- Combining defenses for empirical gains, but validating each step to avoid overfitting to particular attack types.
- Careful benchmarking with full version and hyperparameter documentation for reproducibility.
- Profiling and tuning for computational cost, especially for optimization-based attacks and complex input domains.

ART's fully modular, extensible architecture allows direct addition of new attacks, defenses, metrics, or data generators by subclassing and registering the relevant base classes. This flexibility enables rapid adoption of novel methods and supports research in adversarial ML at scale [1807.01069].

## 7. Relationship to Other Robustness Toolkits

ART positions itself as more production-ready and broad in scope relative to toolkits such as Foolbox and CleverHans, with comparative advantages in:
- Supporting diverse ML architectures (decision trees, tree ensembles, pipelines).
- Providing certified robustness metrics and detection/poison filtering methods.
- Enabling direct integration into ML pipelines, CI/CD workflows, and large-scale experiments.

ART encourages users to combine its components with external libraries (e.g., URET for object-level adversarial inputs [2308.01840]), and to report standardized results for fair, reproducible benchmarking.

---

**References:**

- "Adversarial Robustness Toolbox v1.0.0" [1807.01069]
- Related: "URET: Universal Robustness Evaluation Toolkit (for Evasion)" [2308.01840]

Source: https://www.emergentmind.com/topics/adversarial-robustness-toolbox