Adversarial Random Forests (ARF)

• Adversarial Random Forests (ARF) are tree-based models that combine random forests with adversarial procedures to generate data and estimate densities.
• They employ a minimax training loop using a tree-based generator and discriminator, ensuring robust classification and conditional sample synthesis.
• ARFs offer provable statistical consistency, enhanced adversarial robustness, and efficient imputation through local density estimation in tabular data.

Adversarial Random Forests (ARF) formalize a class of nonparametric, tree-based models for generative modeling, density estimation, and robust classification/regression through adversarial procedures. ARFs leverage the structural strengths of random forests (RFs) while incorporating adversarial principles inspired by generative adversarial networks (GANs), substituting the parametric deep neural components with ensembles of decision trees. This methodology supports both generative learning—synthesizing new, data-manifold-consistent samples for high-dimensional mixed-type tabular data—and robust decision making under adversarial perturbations. ARFs are distinct in their use of adversarial iterative training, explicit density estimation, fast, conditional data synthesis, and provable statistical guarantees for consistency and robustness (Dandl et al., 2024, Golchian et al., 21 Jul 2025, Blesch et al., 19 Jan 2025, Blesch et al., 2023, Watson et al., 2022).

1. Conceptual and Mathematical Foundations

ARF augments standard RF methodology by pairing a generator—implemented via sampling procedures matched to RF leaves’ local empirical marginals—with a discriminator, realized as a random-forest classifier distinguishing real from generated samples. Iterative minimax optimization proceeds analogously to the GAN framework, but is entirely tree-based, avoiding gradient-based parametric learning.

Let $D = \{x^{(i)}\}_{i=1}^n \subset \mathbb{R}^p$ denote real data. The generator maintains parameters $\theta$ (partitioning structure, leaf weights, and intra-leaf univariate density estimates), defining a synthetic distribution $p_\theta$. The discriminator forest $D_\phi$ outputs a realness score $D_\phi(x) \in [0,1]$. The ARF minimax objectives are: $$\begin{aligned} L_D(\phi;\theta) &= - \mathbb{E}_{x \sim p_{\rm data}}[\log D_\phi(x)] - \mathbb{E}_{x \sim p_\theta}[\log(1 - D_\phi(x))] \ L_G(\theta;\phi) &= - \mathbb{E}_{x \sim p_\theta}[\log D_\phi(x)] \end{aligned}$$ ARF alternates: (i) fixing generator parameters, training the RF discriminator to minimize $L_D$; (ii) fixing the discriminator, updating the generator by re-estimating intra-leaf marginals and resampling new synthetic points to minimize $L_G$. The process terminates when the discriminator’s accuracy approaches 50%, indicating indistinguishability between real and synthetic samples (Dandl et al., 2024, Blesch et al., 2023, Watson et al., 2022).

The globally estimated density is a leaf-mixture: $$\hat{p}(x) = \sum_{l: x \in X_l} \pi_l \prod_{j=1}^p p_{l,j}(x_j)$$ where $\pi_l$ is the relative data frequency in leaf $\theta$0, and $\theta$1 is the empirical (for categorical) or smoothed (for continuous) intra-leaf marginal (Golchian et al., 21 Jul 2025, Blesch et al., 19 Jan 2025, Dandl et al., 2024).

2. Algorithmic Architecture and Training Procedure

ARF proceeds through explicit rounds:

1. Initialization: Fit an initial (unsupervised or discriminative) RF to partition data. Compute initial leaf-wise univariate marginals.
2. Generation: Sample synthetic data by: (a) selecting a tree, (b) selecting a leaf proportional to empirical occupancy, (c) sampling each feature from the leaf’s fitted marginal. This defines the current generator distribution $\theta$2.
3. Discrimination: Label synthetic (y = 0) and real (y = 1) samples, retrain an RF classifier to maximize discrimination accuracy.
4. Update generator: Re-estimate univariate marginals in each leaf from the partitioning induced by the latest discriminator, update mixture weights.
5. Stopping: Halt when the discriminator cannot separate real from synthetic beyond chance ($\theta$3 50% accuracy) or after a fixed number of rounds.

This adversarial loop, operating without parametric gradients or explicit manifold regularization, is enforced solely by the recursive RF partition structure and local density updates (Blesch et al., 2023, Golchian et al., 21 Jul 2025, Watson et al., 2022).

Conditional Generation: Conditioning for imputation or counterfactual generation operates by restricting synthesis to leaves and marginals consistent with desired feature values, and reweighting leaf probabilities proportionally to their support for the observed conditioning set (Blesch et al., 19 Jan 2025, Dandl et al., 2024, Golchian et al., 21 Jul 2025).

3. Theoretical Guarantees and Statistical Properties

ARFs provide several provable properties under regularity conditions:

• Consistent Density Estimation: As sample size and tree-depth increase, with appropriate regularization (minimum leaf size, balanced splits), the ARF density estimator converges $\theta$4-consistently to the true underlying distribution (Watson et al., 2022, Golchian et al., 21 Jul 2025).
• Local Independence: Recursive adversarial splitting ensures that, within each leaf, features become approximately independent. This structural property is achieved because the discriminator cannot further exploit cross-feature dependencies at convergence (Watson et al., 2022, Blesch et al., 19 Jan 2025).
• Mixture Model Representation: The global fitted density forms a mixture-of-products of univariate leaf marginals, naturally handling mixed and high-dimensional input spaces (Golchian et al., 21 Jul 2025, Blesch et al., 19 Jan 2025, Watson et al., 2022).
• Provable Robustness (under Adversarial Training): For robust classification settings, adversarial training by pruning or distillation (via soft vote regression) can yield classifiers whose robustness can be analyzed via convex-programming (region-based) attacks and finite-sample optimality theorems (Yang et al., 2019, Apruzzese et al., 2019).

4. Applications of ARF: Generative Modeling, Explanation, Robustness

Synthetic Data Generation and Density Estimation: ARF enables efficient unconditional and conditional sample synthesis for tabular data, including mixed-type features, via the so-called FORDE (density estimation) and FORGE (sample generation) regimes (Blesch et al., 2023, Watson et al., 2022).

Counterfactual Explanations: In the context of XAI, ARFs can generate plausible, on-manifold counterfactual instances by conditioning the generative process on the desired model outcome and fixed subsets of features. ARF-based counterfactual generators achieve naturalism, sparsity, and model-agnosticity, outperforming nearest-neighbor or deep generative approaches in plausibility and efficiency (Dandl et al., 2024).

Conditional Feature Importance: The cARFi method uses ARFs to sample on-manifold interventions for feature importance assessment, providing accurate conditional importances for complex, collinear tabular data (Blesch et al., 19 Jan 2025).

Missing Data Imputation: ARF-driven imputation methods (MissARF) impute missing values by sampling from the conditional distribution estimated via the ARF density, achieving quality and efficiency comparable to state-of-the-art multiple imputation approaches (Golchian et al., 21 Jul 2025).

Adversarial Robustness for Classification: ARFs, either as direct classifiers or in hybrid DNN–RF models, can be structured to block gradient-based attacks (by replacing differentiable heads with non-differentiable forests) or smooth the model via distillation, empirically doubling adversarial detection rates compared to naive RFs (Apruzzese et al., 2019, Ding et al., 2019).

Post-Training Defenses: Augmented RFs, using test-time data augmentations and RFs on DNN logits, enhance adversarial robustness in image classification, synergizing with adversarial training for state-of-the-art defense against both adaptive and non-adaptive adversaries (Cohen et al., 2021).

5. Empirical Performance and Computational Characteristics

ARF methods have demonstrated competitive or superior empirical performance versus deep generative models (CTGAN, TVAE), probabilistic circuits (EiNet, RAT-SPN), and other tree-based alternatives in density estimation (held-out log-likelihood, KL-divergence), generative modeling (F1-score, two-sample statistics), and classification robustness (attack success rate, adversarial recall). ARF training and sampling are typically $\theta$5–$\theta$6 faster than deep model competitors for tabular benchmarks. Available packages (arf in R (Watson et al., 2022), arfpy in Python (Blesch et al., 2023)) provide accessible, well-documented APIs suitable for both research and practice.

For imputation (MissARF), ARF sampling avoids rejection, naturally scales with missingness, and supports both single and multiple imputation without increased runtime (Golchian et al., 21 Jul 2025).

6. Design, Tuning, and Practical Usage

ARF hyperparameters include number of trees, minimum leaf size, adversarial threshold ($\theta$7), maximum depth, chosen univariate intra-leaf density model, and (for conditional generation) conditioning set specification. Tuning balances bias–variance trade-offs: small leaves yield high-variance but local conditionals, larger leaves emphasize smoothing but may lower generative fidelity (Blesch et al., 2023, Golchian et al., 21 Jul 2025, Blesch et al., 19 Jan 2025). For high-dimensional tables, increased ensemble size and moderate leaf sizes are advised.

When integrating with DNNs, forests can be inserted at the layer with maximal adversarial activation divergence for strongest gradient-shattering effect (Ding et al., 2019). In ARF-based XAI, the number and selection strategy for features to perturb in counterfactuals or for cARFi interventions directly affect explanation sparsity and stability (Dandl et al., 2024, Blesch et al., 19 Jan 2025).

7. Relationship to Other Tree-Based and Generative Models

ARF stands apart from classic RFs (purely discriminative, no generative or adversarial loop), tree-based density estimators (which cannot synthesize diverse conditional samples), and deep generative models (parametric, slower, with mode collapse issues). ARF achieves a unique mixture of properties:

• No need for backpropagation or global gradient-based optimization.
• Handles continuous and categorical data natively, without one-hot encoding.
• Provably converges to the true density under regular sampling and tree growing.
• Competitive or superior sample fidelity, likelihood estimation, and adversarial robustness in benchmarks across UCI tabular, synthetic, and real-world application domains (Watson et al., 2022, Blesch et al., 2023, Apruzzese et al., 2019).

In summary, ARFs provide a flexible, scalable, and theoretically grounded platform for generative modeling, conditional sampling, robust classification, explainability, and imputation in complex tabular data environments, filling a distinct gap left by parametric deep models and classic forest approaches.