ROAD: Removing Bias in Machine Learning

• ROAD is a family of methods that remove biased or confounding signals from models, encompassing both attribution evaluation and fairness debiasing.
• In attribution evaluation, ROAD introduces minimally revealing imputation to decouple mask artifacts from predictive information, thereby boosting evaluation consistency.
• For fairness, ROAD employs adversarial debiasing and distributionally robust optimization to enforce local fairness beyond global group averages.

Searching arXiv for the cited ROAD-related papers and nearby references to ground the article. arxiv_search(query="ROAD Remove and Debias fairness adversarial debiasing (Grari et al., 2023) OR (Rong et al., 2022) OR (Song et al., 2023)", max_results=10) arxiv_search(query="On the Fairness ROAD Robust Optimization for Adversarial Debiasing", max_results=5) arxiv_search(query="A Consistent and Efficient Evaluation Strategy for Attribution Methods", max_results=5) Remove and Debias (ROAD) is a recurrent acronym in recent machine-learning research, but it does not denote a single method. In explainability, ROAD names an evaluation framework for attribution methods that seeks to debias pixel-perturbation benchmarks by replacing fixed-value masking with a minimally revealing imputation and by avoiding retraining (Rong et al., 2022). In algorithmic fairness, ROAD also names “Robust Optimization for Adversarial Debiasing,” a distributionally robust adversarial-learning method for enforcing local fairness beyond global group averages (Grari et al., 2023). Related work uses the same removal-and-debiasing motif for representation repair, biased-data removal, inference-time rule erasure, agnostic bias mitigation, and modular attribute-removal subnetworks (Bower et al., 2018, Verma et al., 2021, Zhang et al., 2024, Li et al., 2023, Hauzenberger et al., 2022). The term therefore identifies a methodological pattern—remove information that carries a protected or confounding signal—rather than a single canonical algorithm.

1. Terminological scope and recurring design pattern

The acronym ROAD appears in multiple technically distinct literatures. Some uses are evaluation-oriented, where the objective is to measure attribution quality without confounding from mask shape. Others are intervention-oriented, where the objective is to suppress or remove biased information from predictors, representations, training data, or deployed model outputs.

Usage of ROAD	Core object	Representative paper
Remove and Debias	Attribution evaluation under pixel removal	(Rong et al., 2022)
Remove-And-Debias	Debiased ROAR benchmark with shape-matched random masks	(Song et al., 2023)
Robust Optimization for Adversarial Debiasing	Local fairness via DRO and adversarial debiasing	(Grari et al., 2023)
ROAD approach	Regression/projection removal of protected-attribute variation	(Bower et al., 2018)
ROAD framework	Removal of biased training examples via influence functions	(Verma et al., 2021)

A common misconception is that ROAD refers uniquely to the fairness method of 2023 or uniquely to the attribution benchmark of 2022. The literature instead uses the acronym for several proposals that share a removal-based intuition but operate at different levels: pixels, representations, instances, logits, or fairness constraints. This suggests that “ROAD” is best understood as a family resemblance across methods rather than a single lineage.

2. ROAD in attribution evaluation

In the attribution literature, ROAD was introduced as an information-theoretic alternative to pixel-perturbation evaluations such as ROAR and KAR/LeRF. The central claim is that fixed-value imputation makes the binary removal mask recoverable from the perturbed image, so a classifier can exploit the shape of the removed region rather than the informational content of the remaining pixels. The paper formalizes this using the decomposition

$I(x';C)=I(C;x' \mid M)+I(C;M)-I(C;M \mid x'),$

where $I(C;x' \mid M)$ is “Feature Information,” $I(C;M)$ is “Mask Information,” and $I(C;M \mid x')$ is the “Mitigator.” Under invertible imputation, the mask term leaks into $I(x';C)$ and spuriously boosts evaluation scores (Rong et al., 2022).

ROAD replaces fixed-value masking with a Noisy Linear Imputation. For each removed pixel, the imputed value is a weighted combination of the 4 direct neighbors and the 4 diagonal neighbors, with weights $w_d=1/6$ and $w_i=1/12$, followed by small Gaussian noise $\eta \sim N(0,\sigma^2)$ with $\sigma \approx 0.1$. The stated goal is “Minimally Revealing Imputation,” i.e. to make $M$ and $I(C;x' \mid M)$0 approximately independent so that the evaluation reflects only the information carried by the remaining feature values. The resulting metric evaluates the original model directly on perturbed inputs,

$I(C;x' \mid M)$1

with no retraining step (Rong et al., 2022).

Empirically, the method is reported to increase consistency across removal orders and between retraining and no-retraining protocols. With fixed-value imputation, Spearman rank-correlation between MoRF and LeRF rankings of 8 attribution methods is described as essentially 0; with ROAD it rises to approximately $I(C;x' \mid M)$2 in the retrain setting and approximately $I(C;x' \mid M)$3 in the no-retrain setting. Agreement between retrain and no-retrain evaluations exceeds $I(C;x' \mid M)$4–$I(C;x' \mid M)$5. Runtime is reported as approximately $I(C;x' \mid M)$6 for ROAR versus approximately $I(C;x' \mid M)$7 for ROAD, i.e. $I(C;x' \mid M)$8 of ROAR, and a classifier trained solely on binary masks achieves $I(C;x' \mid M)$9–$I(C;M)$0 accuracy on CIFAR-10 under fixed imputation, supporting the claim that mask information is highly informative (Rong et al., 2022).

3. Debiased ROAR and the blurriness-bias critique

A closely related use of ROAD appears in the literature on ROAR benchmarking. In that setting, ROAD denotes “RemOve-And-Debias,” introduced as a correction for the observation that the shape of a binary drop mask can itself leak class information. The protocol compares the retraining accuracy induced by an explainer-derived mask $I(C;M)$1 to the accuracy induced by a random mask $I(C;M)$2 with the same shape-distribution, and defines

$I(C;M)$3

The intended interpretation is that $I(C;M)$4 captures the extra damage attributable to the explainer’s mask beyond what is caused by an equally shaped random mask (Song et al., 2023).

The later critique “On Pitfalls of RemOve-And-Retrain: Data Processing Inequality Perspective” argues that both ROAR and ROAD can reward blurrier attributions that contain less information about the decision function. The analysis uses the Data Processing Inequality and the Markov chain $I(C;M)$5 to show that post-processing reduces $I(C;M)$6, yet empirical results exhibit cases where the post-processed attribution yields dropped inputs with lower retained label information and therefore a “better” benchmark score. The paper names this effect “blurriness bias” and reports that Gaussian smoothing and max-pool post-processing systematically lower $I(C;M)$7 and increase $I(C;M)$8 relative to the unprocessed map, with a strong positive correlation between drop-accuracy and total variation, with $I(C;M)$9 in the range $I(C;M \mid x')$0 (Song et al., 2023).

This criticism is important because it qualifies what ROAD scores mean in attribution benchmarking. A higher ROAD score need not imply a more faithful attribution map; it may reflect smoother masks that destroy more mutual information between the perturbed input and the label. A plausible implication is that ROAD-style perturbation benchmarks are informative only when paired with auxiliary controls for smoothness, mask statistics, or other information-theoretic confounders.

4. ROAD as robust optimization for adversarial debiasing

In algorithmic fairness, ROAD refers to “Robust Optimization for Adversarial Debiasing,” a method motivated by the distinction between global and local fairness. Global criteria such as Demographic Parity (DP) and Equalized Odds (EO) constrain averages over sensitive groups, for example

$I(C;M \mid x')$1

but may leave substantial disparities within subregions of the feature space. ROAD addresses this by formulating local fairness as a worst-case constraint over an ambiguity set $I(C;M \mid x')$2 of test-time distributions near the training distribution, yielding a DRO-style constraint

$I(C;M \mid x')$3

(Grari et al., 2023).

The operational mechanism couples adversarial debiasing with instance-level reweighting. A predictor $I(C;M \mid x')$4 minimizes task loss, an adversary $I(C;M \mid x')$5 tries to reconstruct the sensitive attribute from the prediction for DP or from $I(C;M \mid x')$6 for EO, and a reweighting function $I(C;M \mid x')$7 defines a worst-case distribution $I(C;M \mid x')$8. ROAD then solves a three-player min–max–max problem in which the reweighter emphasizes points where the adversary has the least difficulty reconstructing the sensitive attribute. The normalization constraints $I(C;M \mid x')$9 for each $I(x';C)$0 preserve the sensitive marginals and prevent concentration on the majority group (Grari et al., 2023).

The nonparametric variant, BROAD, has a closed-form Boltzmann solution,

$I(x';C)$1

while the parametric version learns $I(x';C)$2 by gradient ascent. The training loop alternates adversary updates, reweighter updates, and a predictor update. Reported computational cost scales with $I(x';C)$3, with typical choices $I(x';C)$4–$I(x';C)$5, $I(x';C)$6–$I(x';C)$7, batch size approximately $I(x';C)$8–$I(x';C)$9, and hyperparameter sweeps over $w_d=1/6$0 and $w_d=1/6$1 (Grari et al., 2023).

The empirical evaluation uses COMPAS (race), Law School (race), and German Credit (gender) for local-fairness tests, with held-out subgroups such as age-bins × gender that are unknown at training time, and a drift test that trains on 1994 Adult UCI and tests on 1994 together with 2014 and 2015 Folktables. Global performance is measured against DP or EO, and local fairness is measured by Worst-1-DI, defined as the maximum DP violation across the held-out subgroups. ROAD is reported to Pareto-dominate all baselines with respect to local fairness and accuracy for a given global fairness level, while BROAD is described as more conservative and sometimes over-penalizing. Under distribution shift, ROAD retains a far better accuracy–EO trade-off than CUMA or fairLR (Grari et al., 2023).

5. Earlier remove-and-debias formulations: representations and training data

Before the 2022–2024 ROAD variants, the remove-and-debias idea had already appeared in representation learning. “Debiasing representations by removing unwanted variation due to protected attributes” proposes a linear factor model

$w_d=1/6$2

where $w_d=1/6$3 is the learned representation, $w_d=1/6$4 is the protected attribute, and $w_d=1/6$5 denotes permissible attributes. The debiased representation is obtained by subtracting the protected component,

$w_d=1/6$6

with $w_d=1/6$7 estimated by OLS or ridge within homogeneous subgroups, or by factor-analysis/PCA-style projection when the protected attribute is unobserved. The paper states that this yields a first-order approximation of conditional parity and reports, on COMPAS-style recidivism scores, that the difference in FNR between races at the $w_d=1/6$8th quantile drops from approximately $w_d=1/6$9 to approximately $w_i=1/12$0, with FPR difference eliminated and only a small change in overall accuracy (Bower et al., 2018).

A distinct preprocessing formulation appears in “Removing biased data to improve fairness and accuracy.” There, ROAD is a black-box method that identifies discriminatory similar pairs across sensitive groups, chooses the lower-confidence member of each discriminatory pair, estimates the influence of each training point on those unfair decisions via influence functions,

$w_i=1/12$1

averages these influences to obtain $w_i=1/12$2, and removes the highest-ranked training examples in $w_i=1/12$3 chunks until individual discrimination stops improving (Verma et al., 2021).

The fairness metrics in that work are individual discrimination, statistical parity difference, and test accuracy. Across eight settings covering Adult-Income, German credit, Student exam, Recidivism, Credit default, and Small Salary, the reported averages show that, under exact-match similarity $w_i=1/12$4, the full model has ID approximately $w_i=1/12$5, accuracy approximately $w_i=1/12$6, and SPD approximately $w_i=1/12$7, whereas ROAD attains ID $w_i=1/12$8, accuracy approximately $w_i=1/12$9, and SPD approximately $\eta \sim N(0,\sigma^2)$0. Under $\eta \sim N(0,\sigma^2)$1 tolerance $\eta \sim N(0,\sigma^2)$2, ROAD yields ID approximately $\eta \sim N(0,\sigma^2)$3, accuracy approximately $\eta \sim N(0,\sigma^2)$4, and SPD approximately $\eta \sim N(0,\sigma^2)$5 (Verma et al., 2021). These two earlier formulations show that removal can target either a protected subspace in representation space or a subset of harmful training instances.

6. Inference-time, agnostic, and modular remove-and-debias variants

The broader removal-and-debias motif has also migrated to settings where model weights are frozen or bias types are not explicitly labeled. “Inference-Time Rule Eraser: Fair Recognition via Distilling and Removing Biased Rules” presents a two-stage method that estimates the biased rule $\eta \sim N(0,\sigma^2)$6 with a small patch model and then corrects the deployed model’s outputs by log-space subtraction,

$\eta \sim N(0,\sigma^2)$7

where $\eta \sim N(0,\sigma^2)$8. The paper reports, for example, that on CelebA (Attractive) a vanilla ResNet-34 has EO approximately $\eta \sim N(0,\sigma^2)$9 and worst-group accuracy approximately $\sigma \approx 0.1$0, while adding Eraser yields EO approximately $\sigma \approx 0.1$1 and worst-group accuracy approximately $\sigma \approx 0.1$2; on Colored-MNIST the bias gap falls from $\sigma \approx 0.1$3 to $\sigma \approx 0.1$4, and worst-group accuracy rises from $\sigma \approx 0.1$5 to $\sigma \approx 0.1$6 (Zhang et al., 2024).

When the type and number of biases are unknown, “Partition-and-Debias: Agnostic Biases Mitigation via A Mixture of Biases-Specific Experts” proposes two parallel encoders, multiple bias-specific experts operating at different depths, and a gating module that combines their debiased predictions. The method is trained with classification losses, a diversity regularizer, a gating loss, and a counterfactual contrastive loss. On Biased MNIST with seven biases and bias ratio $\sigma \approx 0.1$7, PnD reports $\sigma \approx 0.1$8 versus $\sigma \approx 0.1$9 for OccamNet and $M$0 for DebiAN; on MIMIC-CXR+NIH with source bias ratio $M$1, it reports $M$2 versus $M$3 for DebiAN (Li et al., 2023). The paper explicitly describes PnD as a practical realization of “Remove and Debias” under agnostic bias conditions.

A modular inference-time variant appears in “Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks.” Each protected attribute is assigned a sparse additive diff subnetwork $M$4 learned under task preservation, debiasing, and $M$5 sparsity objectives. At inference, any subset of attributes can be debiased by evaluating $M$6. On BERT-Base, each subnetwork is constrained to at most $M$7 of parameters, sometimes only $M$8–$M$9, and independently trained subnetworks for gender and age on PAN16 can be summed at inference while recovering nearly the same bias mitigation as joint training (Hauzenberger et al., 2022). These developments show that, beyond its original named instances, remove-and-debias has become a general architectural principle for selective, post hoc, or black-box fairness intervention.