XAI-BENCH: Benchmarking Frameworks for XAI

• XAI-BENCH is a comprehensive suite of benchmarking frameworks that objectively evaluates explainable AI methods using synthetic datasets and ground-truth comparisons.
• It employs multi-dimensional metrics—including fidelity, robustness, and sparsity—to assess post-hoc attribution techniques across tabular, image, and graph data.
• Interactive tools and standardized evaluation protocols enable robust cross-method comparisons, facilitating deeper insights into method performance and human alignment.

XAI-BENCH denotes a class of benchmarking frameworks, datasets, and evaluation suites for Explainable Artificial Intelligence (XAI) methods, spanning tabular, image, and graph modalities. Across recent research, the term is applied both to specific synthetic-data toolkits, such as XAI-Bench (Liu et al., 2021), Compare-xAI (Belaid et al., 2022), and platforms deriving ground-truth for images, graphs, and Boolean functions (Clark et al., 2023, Brandt et al., 2023, Clark et al., 20 May 2024, Sithakoul et al., 29 Jul 2024, Kazmierczak et al., 4 Nov 2024, Fontanesi et al., 18 May 2025, Armoni-Friedmann et al., 12 Sep 2025), as well as to generalized protocols for objectively and reproducibly quantifying XAI methods' fidelity, faithfulness, robustness, complexity, plausibility, and human-aligned accuracy. The overarching goal of these frameworks is to provide rigorous, reproducible, and empirically grounded evaluations of post-hoc attribution and explanation algorithms for machine learning, under precisely defined ground-truth or well-controlled conditions.

1. Motivation and Context: The Benchmarking Imperative

Benchmarking in XAI emerges from several core methodological and practical challenges:

• Absence of ground truth in real-world scenarios: Most ML tasks lack objective attribution, making it difficult to judge whether an explanation method is "correct" or reliable (Liu et al., 2021, Belaid et al., 2022).
• Conflicting or non-robust metrics: Established faithfulness, robustness, and plausibility metrics often yield contradictory rankings across methods and data regimes (Sithakoul et al., 29 Jul 2024, Kazmierczak et al., 4 Nov 2024).
• Dataset and modality diversity: XAI methods must be evaluated across structured tabular data, high-dimensional images, and complex graphs, each with distinctive attribution semantics (Clark et al., 20 May 2024, Fontanesi et al., 18 May 2025, Zhang et al., 2023).
• Human alignment gap: Quantitative metrics may be poorly correlated with actual human preferences; explainers often pass numerical tests but fail to align with domain-expert intuition or user trust (Kazmierczak et al., 4 Nov 2024).
• Performance variability: Explanations for the same prediction, but from models with matching accuracy, can differ dramatically due to architecture or training randomness (Clark et al., 20 May 2024, Clark et al., 2023).

XAI-BENCH frameworks directly address these issues by synthesizing datasets with known explanation ground truths and/or collecting extensive human annotations, and by implementing multi-dimensional scoring pipelines that capture essential properties for different stakeholder groups (researchers, practitioners, lay users) (Belaid et al., 2022, Kazmierczak et al., 4 Nov 2024).

2. Synthetic Data and Ground-Truth Explanation Construction

Many XAI-BENCH platforms leverage synthetic data generation to ensure objective evaluation:

• Tabular and Linear Models: Synthetic data with canonical equations (linear, polynomial, piecewise) allows for closed-form calculation of feature importances and direct correspondence between explanation methods (LIME, SHAP) and true coefficients (Amiri et al., 2020, Liu et al., 2021).
• Image-based Benchmarks: Synthetic images, such as the tetromino-shape XAI-TRIS suite, inject ground-truth feature importance masks; perturbation protocols can precisely define causal attributions (Clark et al., 2023, Clark et al., 20 May 2024, Brandt et al., 2023).
• Graph Data: The OpenGraphXAI methodology mines real-world molecular datasets using Weisfeiler–Leman coloring to extract sub-graph motifs corresponding to ground-truth explanations for node/graph classification (Fontanesi et al., 18 May 2025).
• Boolean Formulae: Precise causal-variable benchmarks (XAI-BENCH, B-ReX) evaluate how well XAI methods recover Halpern–Pearl actual causality-based responsibilities for predictors of arbitrary Boolean functions (Armoni-Friedmann et al., 12 Sep 2025).

This synthetic paradigm overcomes subjectivity, enabling rigorous distance, rank, and precision-recall comparisons, and supports cross-method, cross-architecture, and cross-modality stress testing (Brandt et al., 2023, Liu et al., 2021, Clark et al., 20 May 2024).

3. Multi-Dimensional Metrics: Fidelity, Faithfulness, Robustness, and More

State-of-the-art XAI-BENCH frameworks instrument evaluation pipelines with multi-metric, multi-level scoring:

• Faithfulness (Fidelity Correlation):

$$\mu_F(f,g;x) = \operatorname{corr}_S \left( \sum_{i \in S} g(f,x)_i, f(x) - f(x_S) \right)$$

Assesses alignment between feature importance and prediction change under masking/intervention (Sithakoul et al., 29 Jul 2024, Liu et al., 2021).

• Robustness (Sensitivity):

$$\mathrm{SENS\_MAX} = \max_{\|\mathbf{y}-\mathbf{x}\| \leq r} \| g(f,\mathbf{x}) - g(f,\mathbf{y}) \|_2$$

Quantifies stability under small input perturbations.

• Complexity/Sparsity:

Gini index, entropy, and related measures favor explanations with minimal, concentrated attribution (Sithakoul et al., 29 Jul 2024).

• Monotonicity, Sufficiency, Comprehensiveness: Diverse measures operationalize whether adding features increases attribution, whether small feature sets suffice, and whether removing top features substantially hurts output (Sithakoul et al., 29 Jul 2024).
• Precision and Recall (Signed Pixel Attribution):

For images and synthetic models, precision and recall are split for positive and negative contributions, enabling nuanced analysis of explainer failure modes (Brandt et al., 2023).

• Alignment Metrics (IoU, Pointing, F1, AUC):

Compare saliency or attribution maps to ground-truth masks (pixel, node, or concept), with ROC-curve AUC for graph node importance (Zhang et al., 2023, Fontanesi et al., 18 May 2025).

• Human-Aligned Metrics:

Learned scoring networks (e.g., PASTA-metric) ingest explanations (saliency, concept) and output predicted human judgment scores, trained on annotated datasets (Kazmierczak et al., 4 Nov 2024).

Notably, Compare-xAI (Belaid et al., 2022) introduces hierarchical scoring: raw per-test scores (functional tests), aggregated category scores (fidelity, stability, simplicity, etc.), and a single comprehensibility index to guide method selection and risk assessment.

4. Evaluation Protocols and Experimental Findings

XAI-BENCH platforms standardize the full pipeline:

1. Data Preparation: Generate or select synthetic or real data; process features (categorical, numerical, time, etc.) (Sithakoul et al., 29 Jul 2024, Liu et al., 2021).
2. Model Training: Benchmark models provided or trained (XGBoost, MLP, CNN, GNN, logistic regression) to controlled accuracy (Sithakoul et al., 29 Jul 2024, Fontanesi et al., 18 May 2025, Clark et al., 2023).
3. Explanation Generation: Apply post-hoc explainers (LIME, SHAP, Saliency, GradCAM, IntegratedGradients, DeepLIFT, PatternNet, KernelSHAP, B-ReX, GNNExplainer, CAM), often via standardized API calls (Zhang et al., 2023, Clark et al., 20 May 2024, Armoni-Friedmann et al., 12 Sep 2025).
4. Metric Computation: Evaluate each (dataset, model, explainer) tuple over chosen metrics; raw, aggregate, and cross-seed statistical summaries are reported (Sithakoul et al., 29 Jul 2024, Liu et al., 2021, Fontanesi et al., 18 May 2025).
5. Visualization and Interpretation: Radar/bar plots, leaderboards, drill-down UI reports highlight per-method performance and failure cases (Sithakoul et al., 29 Jul 2024, Belaid et al., 2022).

Key empirical findings across studies:

• SHAP and Shapley-based methods often score highest on faithfulness/fidelity, but may trade-off sparsity or robustness (Sithakoul et al., 29 Jul 2024).
• Saliency methods (including gradient-based) deliver high robustness but sometimes lower faithfulness for tabular and image data (Sithakoul et al., 29 Jul 2024, Zhang et al., 2023).
• LIME and DeepLIFT excel in sparsity/complexity, but can misorder true feature importances or show instability under sampling (Sithakoul et al., 29 Jul 2024, Amiri et al., 2020).
• On synthetic image benchmarks (XAI-TRIS, EXACT), many popular explainers perform only marginally above random or edge detectors in non-linear/correlated regimes (Clark et al., 2023, Clark et al., 20 May 2024).
• Graph explainers: CAM method yields highest AUC plausibility scores; GNNExplainer and gradient methods are more susceptible to spurious attribution on real-motif datasets (Fontanesi et al., 18 May 2025).
• Human-aligned scoring (PASTA): Annotators strongly prefer standard saliency maps to concept-based explanations; model-independent faithfulness scores are very poorly correlated with human judgments (Kazmierczak et al., 4 Nov 2024).

Representative Benchmarked Results (Tabular/Image/Graph)

Explainer	Faithfulness	Robustness	Sparsity	Alignment (IoU)	Human Score (mean stars)
SHAP	High	Med	Med	Med	2.7 (Q1–Q4), 4.2 (Q5–Q6)
Saliency	Med	High	Low	Med-Low	2.9 (Q1–Q4), 4.3 (Q5–Q6)
LIME	Med	Med	High	Low	2.4 (Q1–Q4), 3.5 (Q5–Q6)
B-ReX	Top (JSD)	Med	Med	N/A	N/A
CAM (Graph)	N/A	N/A	N/A	>0.85 AUC (13/15 tasks)	N/A

Values paraphrased from experimental analyses (Sithakoul et al., 29 Jul 2024, Kazmierczak et al., 4 Nov 2024, Fontanesi et al., 18 May 2025, Armoni-Friedmann et al., 12 Sep 2025).

5. Interactive Tools, Extensibility, and Best Practices

XAI-BENCH platforms emphasize user interaction, method extension, and reproducibility:

• Interactive UI: Web dashboards such as Compare-xAI visualize comprehensibility scores, per-test breakdowns, and Pareto-optimal selection strategies (Belaid et al., 2022).
• Extensibility: Adding a new explainer or metric typically requires subclassing a base class and registering with the evaluation pipeline (Python-centric, scikit-learn/PyTorch ecosystem) (Sithakoul et al., 29 Jul 2024, Clark et al., 20 May 2024).
• Automated Submission: EXACT adopts a Dockerized workflow, enabling code submission and leaderboard placement on predefined benchmarks (Clark et al., 20 May 2024).
• Human-in-the-loop: Integration of human-centric metrics or crowdsourced annotation is encouraged, particularly when numerical metrics diverge from user perception or trust (Kazmierczak et al., 4 Nov 2024).
• Failure Mode Analysis: Practitioners must scrutinize per-test and per-seed variabilities, correlation with random/expert baselines, and check for robustness to adversarial or distributional shifts (Clark et al., 2023, Clark et al., 20 May 2024).
• Open Source and Data Availability: Most platforms (XAI-Bench, BEExAI, OpenGraphXAI, Saliency-Bench, EXACT) are available on GitHub with full code, APIs, and synthetic data generators (Liu et al., 2021, Sithakoul et al., 29 Jul 2024, Fontanesi et al., 18 May 2025).

6. Limitations, Controversies, and Outlook

Several limitations persist in current XAI-BENCH initiatives:

• Synthetic–Real Gap: Synthetic ground-truth benchmarks ensure objectivity but may overstate method capacity for real-world, high-correlation, or distribution-shift scenarios (Clark et al., 2023, Clark et al., 20 May 2024).
• Human Bias and Cultural Specificity: Human-aligned metrics depend on annotator demographics; subjectivity remains even with explicit protocols, limiting global generalization (Kazmierczak et al., 4 Nov 2024).
• Model Architecture Sensitivity: Explanations can differ wildly across equally accurate yet structurally distinct models, challenging transfer and deployment practices (Clark et al., 20 May 2024).
• Metric Non-Redundancy and Aggregation: Certain metrics (faithfulness, robustness, plausibility) are not interchangeable, and single-number scoring may mislead without detailed per-category breakdowns (Belaid et al., 2022, Kazmierczak et al., 4 Nov 2024).
• Adversarial Fragility: Many explainers are vulnerable to minor distributional or adversarial changes, even if they pass the primary benchmark (Belaid et al., 2022, Clark et al., 2023).

Current research aims to integrate graph, text, and multi-modal benchmarks (BEExAI, OpenGraphXAI), develop richer human-in-the-loop protocols, and unite synthetic ground-truth with real-data functional tests. Extensions under paper include rule-based explainers, combined human–machine plausibility metrics, and dynamic/adaptive scoring systems as preferences and domain conventions evolve (Sithakoul et al., 29 Jul 2024, Kazmierczak et al., 4 Nov 2024, Fontanesi et al., 18 May 2025).

---

XAI-BENCH has become the foundational paradigm for empirical, reproducible, and multi-modal evaluation of explainable AI methods, synthesizing rigorous mathematical metrics, scalable software pipelines, and human-aligned behavioral scoring. By illuminating both the strengths and weaknesses of diverse XAI algorithms, it serves as a critical instrument for methodological advance and responsible deployment.