Bias Analysis & Mitigation via Explanation

• BAME is a framework that employs explanations as actionable diagnostics to reveal and mitigate bias in machine learning models.
• The approach integrates techniques like SHAP-based auditing, explanation-quality analysis, and explanation-regularized training to target fairness at both feature and latent levels.
• Empirical studies across vision, hate-speech detection, and tabular data show that explanation-guided interventions can significantly enhance fairness while preserving performance.

Bias Analysis and Mitigation through Explanation (BAME) denotes a family of approaches in which explanations are not treated as a purely post-hoc accessory, but as a technical interface for identifying, quantifying, and reducing unfairness, spurious correlations, and distributional distortions in machine learning systems. In the literature, this idea appears in several forms: explanation-based auditing of predictive unfairness, explanation-quality auditing across protected groups, explanation-regularized training, explanation-guided post-processing, data editing driven by explanation signals, and human-centered protocols that use explanations to counter systematic cognitive biases (Begley et al., 2020, Jain et al., 2020, Balagopalan et al., 2022, Kim et al., 2023). The unifying premise is that bias becomes more actionable when it is attached to interpretable objects—features, keywords, latent explanatory variables, cohorts, or sample-level comparators—and that mitigation can then operate directly on those objects rather than only on aggregate parity constraints (Binkyte et al., 2023, Liu et al., 2024, Franks et al., 1 Apr 2025).

1. Conceptual foundations

BAME is organized around two related questions. The first is whether a model, dataset, or explanation procedure is biased. The second is whether an explanation can be used as the control signal for debiasing. Several papers formalize the first question by defining fairness in explanation space rather than only in prediction space. In "Fairness by Explicability and Adversarial SHAP Learning" (Hickey et al., 2020), a predictor is explicably fair with respect to an external auditor when an auditor-trained model does not detect any difference in the sensitive-attribute attribution between the $Z=\{0,1\}$ groups. In "Explainability for fair machine learning" (Begley et al., 2020), fairness is treated as a scalar functional $U(f)$, such as demographic parity or equalized odds, and Shapley values are used to attribute $U(f)$ to input features; by construction, the attributions satisfy completeness, so the sum of feature attributions equals the model’s unfairness.

A second foundational strand treats explanations themselves as objects that can exhibit disparities. "The Road to Explainability is Paved with Bias" (Balagopalan et al., 2022) defines overall fidelity $F$, group-specific fidelity $F_s$, and subgroup disparity statistics such as the maximum fidelity gap from the average and the mean fidelity gap among subgroups. "Understanding Disparities in Post Hoc Machine Learning Explanation" (Mhasawade et al., 2024) adopts the same perspective for LIME and shows that explanation disparities can arise from limited sample size, covariate shift, concept shift, omitted variable bias, inclusion of the sensitive attribute, and model functional form.

A third strand broadens the meaning of explanation beyond feature attribution. In BaBE, the relevant explanatory object is a latent legitimate variable $E$, estimated from a biased proxy $Z$ and a sensitive attribute $S$, after which decisions are based on $\hat{E}$ rather than directly on $Z$ (Binkyte et al., 2023). In AIM, the explanatory object is a sample-level bias score $U(f)$0, defined through similarity-weighted, credibility-weighted comparisons to other-group samples (Liu et al., 2024). In B2T for vision, the explanatory object is a keyword extracted from captions of mispredicted images and validated by image–text similarity (Kim et al., 2023). Across these formulations, BAME treats explanation as a structured, manipulable representation of why unfairness appears.

2. Bias analysis through explanation

The most direct BAME pattern is explanation-based bias detection. In SHAP-based auditing, the central idea is that if a model’s predictions are discriminatory, then the feature-attribution distribution should also reflect that disparity. "Biased Models Have Biased Explanations" (Jain et al., 2020) operationalizes this by comparing SHAP distributions across protected groups and outcome strata, and by translating demographic parity, equality of opportunity, and equalized odds into explanation-space criteria. For a protected feature $U(f)$1, one can compare quantities such as

$U(f)$2

or Wasserstein distances between attribution distributions. This framework is explicitly black-box compatible via distillation to a mimic model.

A closely related but more general formulation appears in fairness Shapley explanations. For demographic parity, "Explainability for fair machine learning" (Begley et al., 2020) defines a value function $U(f)$3 over feature coalitions and computes Shapley values $U(f)$4 so that

$U(f)$5

The conceptual significance is that unfairness is not reduced to the attribution of the sensitive attribute alone. Proxy variables and interactions inherit part of the decomposition, so explanation analysis can localize hidden pathways even when the model does not directly consume the protected feature.

In unstructured domains, explanation-driven bias analysis often begins with failure cases rather than with attribution vectors. In "Power of Explanations" (Cai et al., 2022), the automatic misuse detector (MiD) identifies potentially misused words in hate-speech detection by using false positive proportion as a proxy and confirming candidates with Sampling and Occlusion (SOC). The formal proxy is

$U(f)$6

and the explanation score for a word $U(f)$7 is

$U(f)$8

Words whose average contribution on misclassified instances exceeds a threshold are treated as wrong reasons for mistakes.

In computer vision, "Discovering and Mitigating Visual Biases through Keyword Explanation" (Kim et al., 2023) reframes visual bias as keywords extracted from captions of mispredicted images. After forming mispredicted and correctly predicted sets $U(f)$9 and $U(f)$0, candidate keywords $U(f)$1 are scored by

$U(f)$2

where similarities are mean image–text cosine similarities under a vision–LLM. This produces human-readable group names such as “man,” “forest,” “illustration,” or “flower,” enabling direct diagnosis of gender bias, background bias, distribution shift, and contextual co-occurrence bias.

A different analysis granularity appears in AIM, where a sample is biased if similar samples from the other group receive different and credible treatments (Liu et al., 2024). The estimated sample bias score is

$U(f)$3

with a separate credibility estimator $U(f)$4. This makes bias analysis local, instance-specific, and auditable through ranked contributing comparators.

3. Mitigation through explanation

Once the explanatory object has been identified, BAME methods use it as the optimization target or intervention handle. Explanation-regularized training is the most direct mechanism. In MiD’s staged correction framework, the correction-stage objective is

$U(f)$5

so the model is penalized for relying on words already identified as wrong-for-wrong-reasons (Cai et al., 2022). In "Fairness by Explicability and Adversarial SHAP Learning" (Hickey et al., 2020), SHAPSqueeze uses a surrogate model $U(f)$6 with SHAP-based regularization,

$U(f)$7

while SHAPEnforce modifies AdaBoost weights using a SHAP-derived penalty on sensitive-attribute attribution.

A second mitigation family turns explanations into group labels or intervention variables. In B2T, validated keywords become group names that can be used for CLIP prompting, reweighting, or Group DRO (Kim et al., 2023). The paper explicitly uses keyword-defined groups for debiased training and reports that fine-grained keywords such as “bamboo” outperform coarse prompts such as “land.” In AIM, explanations support minimal data editing through AIM-Remove and AIM-Augment: the former removes top-biased samples under a budget, while the latter augments low-bias regions via neighborhood mixup (Liu et al., 2024).

A third family is explanation-guided post-processing. In "Explainable post-training bias mitigation with distribution-based fairness metrics" (Franks et al., 1 Apr 2025), a trained model $U(f)$8 is post-processed within an explainable linear family

$U(f)$9

and optimized under a fairness–utility objective based on threshold-distribution discrepancies such as Wasserstein-1 or Kolmogorov–Smirnov. Because the explanation operator is linear in the model, explanations for all post-processed models can be reconstructed from the base model and encoder explanations. The companion regressor-distribution-control framework further uses predictor-level bias explanations to choose a small set of biased predictors, reshapes their distributions via monotone transformations, calibrates the resulting regressor, and uses Bayesian optimization to construct a bias–performance efficient frontier without retraining (Miroshnikov et al., 2021).

A fourth family uses explanatory variables as the fairness target itself. BaBE estimates the group-conditional distribution $F$0 by EM and recovers

$F$1

Decisions are then based on $F$2, which is intended to satisfy conditional statistical parity approximately to the extent that $F$3 approximates $F$4 (Binkyte et al., 2023). This is a pre-processing version of BAME in which explanation is the legitimate decision factor, not merely a diagnostic artifact.

4. Explanation fairness, explanation bias, and human-centered effects

BAME does not assume that explanations are neutral. One major result of the literature is that explanations themselves can be biased or unequally reliable. "The Road to Explainability is Paved with Bias" (Balagopalan et al., 2022) shows that the fidelity of local and global surrogate explanations differs significantly between subgroups in finance, healthcare, admissions, and justice. Reported subgroup gaps include a max accuracy gap from average up to $F$5 for LIME in lsac + NN, mean AUROC gaps up to $F$6, and persistent non-zero gaps even after subgroup balancing. This leads to the notion of explanation fairness: subgroup disparities in $F$7 should themselves be small.

A complementary diagnosis is that explanation disparity often originates in the data-generating process and model choice rather than only in the explainer. "Understanding Disparities in Post Hoc Machine Learning Explanation" (Mhasawade et al., 2024) shows, through simulations and Adult Income experiments, that increased covariate shift, concept shift, and omitted covariates increase LIME fidelity disparities, with the effect often more pronounced for neural networks than for logistic regression. The paper therefore links explanation disparity to coverage, misspecification, and omitted variable bias rather than framing it solely as an explainer artifact.

Other work shows that post-hoc attribution methods can have their own structural preferences. "Explanation Bias is a Product" (Kamp et al., 11 Dec 2025) defines three Jensen–Shannon-based metrics—Bias-cons, Bias-agg, and Bias-attr—to measure lexical bias, position bias, and inter-method divergence. Across controlled artificial tasks and a causal relation detection task, the paper finds that lexical and position biases are structurally unbalanced across models, and that methods producing anomalous explanations are more likely to be biased themselves. The implication for BAME is that explanation-based debiasing cannot ignore the bias profile of the explainer.

Human-centered studies extend this point from explanation artifacts to explanation use. "Mitigating belief projection in explainable artificial intelligence via Bayesian Teaching" (Yang et al., 2021) models the explainee explicitly and selects explanations that maximize the probability of a desired inference. "On the Interaction of Belief Bias and Explanations" (Gonzalez et al., 2021) shows that human rankings of explanation methods can flip once belief bias is controlled through low-quality models or adversarial examples. In both cases, BAME becomes not only a property of model behavior but also a property of how explanations shape human understanding and error detection.

5. Domain-specific implementations and empirical evidence

In computer vision, B2T provides one of the clearest demonstrations that explanation-derived groups can directly support bias mitigation. It identifies known biases such as gender bias in CelebA, background bias in Waterbirds, and distribution shifts in ImageNet-R/C, while also surfacing novel contextual biases such as “ant” $F$8 “bee” in the presence of “flower” (Kim et al., 2023). On worst-group accuracy, DRO-B2T reaches $F$9 on CelebA blond and $F_s$0 on Waterbirds, compared with ERM at $F_s$1 and $F_s$2, and even slightly exceeds DRO with ground-truth groups in both cases.

In hate-speech detection, MiD provides an explanation-centered alternative to static identity-term lexicons. On the Gab Hate Corpus, MiD-Stabilization reaches $F_s$3 accuracy compared with Vanilla at $F_s$4, while feature attribution and false positive proportion for detected words decrease sharply during correction and remain lower through stabilization (Cai et al., 2022). The framework also adds overlooked terms such as “immigrant,” “liberal,” “leftist,” “racist,” “nazi,” “homo,” and “gender,” which illustrates the advantage of runtime, model-behavior-driven detection over fixed debiasing lists.

In tabular fairness, several BAME variants report strong fairness–utility trade-offs. AIM on Adult, in the gender-sensitive setting, raises prediction consistency from $F_s$5 to $F_s$6 under AIM-Augment while reducing EO from $F_s$7 to $F_s$8 and preserving accuracy at $F_s$9 (Liu et al., 2024). Explanation-based post-processing with distribution-based metrics and encoder linearity is reported across synthetic data, Adult, Bank Marketing, and COMPAS, with tree rebalancing and optimal transport projection leading the efficient frontier in several settings (Franks et al., 1 Apr 2025). Regessor-distribution control likewise reports wider fairness–performance frontiers than retraining-based hyperparameter search on synthetic studies (Miroshnikov et al., 2021).

BAME has also entered generative modeling. In "Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations" (Dimgba et al., 3 Sep 2025), explanation-informed prompt engineering improves demographic representation by $E$0 to $E$1, reduces ethnicity TVD in GPT-4 from $E$2 to $E$3, and yields statistically significant intersectional DPR improvements for Claude 3.5 Sonnet, Llama 3.1 70B, and GPT-4 Turbo under Wilcoxon signed-rank tests. The method is notable because it uses only prompt revisions informed by the model’s own explanations rather than parameter updates.

At the same time, mitigation can create new harms. "Explaining Knock-on Effects of Bias Mitigation" (Nizhnichenkov et al., 2023) shows that all tested static mitigation strategies negatively impact a non-trivial fraction of cases, even when fairness metrics improve. Across Utrecht, Adult, and Bank Marketing, the negatively impacted cohort is always present, and explainable meta-classifiers can identify rules describing the affected cohorts. This result is important within BAME because it shifts the objective from aggregate fairness improvement alone to explanation-supported auditing of downstream intervention effects.

6. Limitations, controversies, and future directions

A recurring limitation is dependence on explanation quality. B2T depends on captioners and vision–language scoring models and may fail in specialized domains such as medical or satellite imagery (Kim et al., 2023). MiD depends on SOC quality, FPP thresholds, and the restriction to wrong-for-wrong-reasons rather than right-for-wrong-reasons (Cai et al., 2022). Fairness-by-explicability depends on the chosen auditor model and its SHAP semantics rather than on a universal fairness oracle (Hickey et al., 2020).

A second limitation is that there is no universal remedy for explanation disparities. Robust local training and balanced surrogate fitting can reduce subgroup fidelity gaps, but "The Road to Explainability is Paved with Bias" explicitly reports mixed results and emphasizes that a single solution might not exist across all settings (Balagopalan et al., 2022). "Understanding Disparities in Post Hoc Machine Learning Explanation" similarly indicates that disparities depend on shift, model class, and covariate availability (Mhasawade et al., 2024). This suggests that BAME is best understood as a diagnostic-and-intervention workflow rather than as a single algorithmic principle.

A third concern is that explanation-based mitigation can itself be biased, unstable, or socially contentious. Explanation methods may overemphasize lexical or positional artifacts (Kamp et al., 11 Dec 2025). Human evaluations may overestimate explanation quality because of belief projection or belief bias (Yang et al., 2021, Gonzalez et al., 2021). In sensitive domains, explanation-derived groups or edits may require human oversight; B2T explicitly notes that bias discovery for gender, race, or geography is a decision-support tool rather than an automated arbiter (Kim et al., 2023), and AIM notes that data editing should remain auditable because removing records that reflect true historical harms can obscure realities if not documented (Liu et al., 2024).

Future directions in the literature are correspondingly technical rather than merely presentational. They include synonym clustering, concept linking, and ontology integration for keyword explanations; causal validation through counterfactuals and data ablations; active data collection guided by discovered biases; group-aware objectives for explanation fidelity; representation learning that reduces protected-attribute leakage; and cross-modal extensions to generative models, detection, segmentation, and video (Kim et al., 2023, Balagopalan et al., 2022). The overall trajectory suggests that BAME is moving from a descriptive use of explanations toward a tightly coupled framework in which explanation, fairness metric, optimization objective, and audit protocol are jointly designed.