CLEVR-XAI: Benchmark for VQA Explanations

• The paper introduces CLEVR-XAI, a ground-truth framework that uses exact pixel-level relevance masks to benchmark post-hoc visual explanations in VQA.
• It features a fully annotated synthetic dataset with simple and complex splits, ensuring selectivity, control, and realism in evaluating explanation methods.
• CLEVR-XAI employs rigorous metrics such as MassAccuracy and RankAccuracy to quantitatively assess and compare various XAI techniques against objective ground truth.

CLEVR-XAI is a ground-truth–based evaluation framework and dataset designed specifically for benchmarking post-hoc visual explanation methods in neural networks, particularly in the context of visual question answering (VQA). Built on the CLEVR VQA platform, CLEVR-XAI enables rigorous, quantitative, and reproducible assessment of neural heatmap explanations by providing exact pixel-level relevance masks for each question’s target set of objects. Its design uniquely addresses the demands for selectivity, control, and realism in evaluating explainability methods, distinguishing itself as a vital resource for advancing the field of explainable AI (XAI) in vision.

1. Motivations and Distinctive Design Principles

CLEVR-XAI was motivated by the limitations of prior XAI evaluation strategies, which often relied on qualitative human assessment or surrogate proxy tasks (e.g., pixel perturbation, weak localization) that lacked objective ground truth and introduced potential confounds. In contrast, CLEVR-XAI satisfies three critical criteria:

1. Selectivity: Every CLEVR VQA question inherently isolates a subset of objects as relevant, creating a class-discriminative and question-sensitive test case. For example, “What color is the tiny sphere?” restricts semantic relevance to one object, allowing for precise evaluation of explanations tied to this selection.
2. Control: The synthetic, fully annotated generation pipeline produces granular, bias-free, and exhaustively labeled datasets, with scene graphs allowing extraction of exact segmentation masks per question.
3. Realism: Despite its synthetic origin, CLEVR-XAI scenes encompass complex 3D visual attributes (occlusion, shading, perspective), retaining fidelity to real-world image complexity while maintaining annotation accuracy (Arras et al., 2020).

These properties together establish a benchmark environment where the ground truth of relevant pixels is known for every test case, a foundational necessity for objective evaluation of explanation methods.

2. Dataset Construction and Annotation Protocols

CLEVR-XAI extends the original CLEVR dataset by generating 10,000 images (320×480 RGB, each depicting 3–10 objects across a combinatorial range of colors, shapes, materials, and sizes). Two principal evaluation splits are constructed:

• CLEVR-XAI-simple: Comprises ~39,800 questions, each a single-attribute query (query_shape, query_color, query_size, query_material), guaranteed to have a unique, answerable object. Masks generated include:
  • GT_SingleObject: the pixel set corresponding to the target object
  • GT_AllObjects: aggregate pixels for all objects, background suppressed
• CLEVR-XAI-complex: Contains 100,000 questions from the full CLEVR functional program space (Count, Exist, Relational, Attribute, Comparison families) with program lengths 2–24 and multiple possible referents. Four mask types are systematically extracted:
  • GT_Unique: all objects resolved by “unique” operators
  • GT_UniqueFirstNonEmpty: includes the unique outputs and first non-empty targets in program traversal
  • GT_Union: union of all intermediate functional outputs (except root “scene”)
  • GT_AllObjects: analogously to the simple split

Annotated objects have mask sizes ranging from dozens to several thousand pixels, supporting quantitative evaluation across varied selectivity and difficulty (Arras et al., 2020).

3. Evaluation Metrics and Explanation Quality Quantification

CLEVR-XAI introduces two principal metrics for assessing explanation heatmaps $R:\{pixels\}\to\mathbb{R}_{+}$ relative to ground-truth pixel masks $GT$:

• Relevance Mass Accuracy (MassAccuracy) measures the fraction of total explanation mass contained within the ground truth set:

$$MassAccuracy = \frac{\sum_{p\in GT} R(p)}{\sum_{p} R(p)}$$

This metric is sensitive to “leakage,” penalizing methods assigning significant mass to irrelevant regions.

• Relevance Rank Accuracy (RankAccuracy) evaluates whether the K most relevant pixels (where K is the number of ground truth pixels) fall within the ground-truth set:

$$RankAccuracy = \frac{|\{P_1,\ldots, P_K\}\cap GT|}{K}$$

Here, $P_1,\ldots, P_K$ are the top-K pixels by relevance. This penalizes mislocalization in the most salient regions.

RGB heatmaps are pooled using one of ten schemes (e.g., $L_1$, $L_2$, max pooling, sum+ReLU), chosen per method to maximize performance on the simple split’s GT_SingleObject task (Arras et al., 2020).

4. Benchmarked XAI Methods and Model Baseline

CLEVR-XAI provides a comparative suite of ten canonical post-hoc explanation techniques, each applied to a fixed, high-accuracy Relation Network (RN) VQA model. The methods evaluated comprise:

• Gradient (Saliency): single-backward-pass $\partial f_c/\partial x_i$
• Gradient×Input: $x_i \cdot (\partial f_c/\partial x_i)$
• SmoothGrad: averaged gradients over input noise perturbations
• VarGrad: pixelwise relevance variability under input noise
• Deconvnet: modified ReLU backpropagation
• Guided Backpropagation: intersection of standard and Deconvnet backward signals
• Grad-CAM: weighted spatial gradients over featuremaps
• Guided Grad-CAM: elementwise product of Grad-CAM and Guided Backprop outputs
• Integrated Gradients (IG): baseline-integrated input gradients
• Layer-wise Relevance Propagation (LRP): backward conservation of relevance, including $\epsilon$-rule and $GT$0-rule with input-layer “box rule”

The baseline model replicates Santoro et al.’s RN architecture: four ResNet-inspired convolutional layers with stride 2 and batch normalization, LSTM-encoded questions, and a relational core implemented via pairwise spatial feature concatenation and multi-layer perceptron, yielding 28-way logits. Standard training on CLEVR gives ≈ 93.3% test accuracy; only correct predictions are subjected to explanation evaluation (Arras et al., 2020).

5. Experimental Results and Comparative Insights

Empirical results on CLEVR-XAI-simple (GT_SingleObject) indicate that LRP (notably the α₁β₀-rule and box-rule), Integrated Gradients (gray baseline), and Guided Backpropagation achieve the highest performance:

• MassAccuracy in the 0.80–0.85 range
• RankAccuracy in the 0.70–0.75 range

Guided Grad-CAM leads among hybrid methods, but consistently trails LRP- and IG-based techniques. Simple gradient and gradient×input explanations perform at intermediate levels (MassAccuracy ≈ 0.45–0.50, RankAccuracy ≈ 0.30–0.35), while Deconvnet and pure Grad-CAM obtain the lowest scores in both metrics. SmoothGrad and VarGrad marginally improve over raw gradients but entail significant tuning and computational overhead.

On GT_AllObjects as a “sanity check,” LRP and IG localize ≥95% of the relevance within object pixels, while other approaches exhibit greater background leakage. For the more demanding CLEVR-XAI-complex split, all methods exhibit score decay as GT masks become more selective, but their relative rankings persist.

These results contradict some prior proxy-based or qualitative evaluations, especially those positioning Grad-CAM and Deconvnet as state-of-the-art for interpretability; rigorous, pixelwise ground truth identifies these as underperforming relative to LRP, IG, and Guided Backprop (Arras et al., 2020).

6. Contextual Significance and Methodological Implications

CLEVR-XAI highlights several conceptual innovations for XAI benchmarking:

• Natural selectivity: VQA tasks induce context-specific heatmaps, in contrast to image classification where class-insensitive explanations are common.
• Discriminative attribution: The necessity for question-sensitive explanation ensures that evaluated XAI methods must truly reflect decision-relevant features.
• Faithfulness emphasis: Methods such as LRP (especially the all-positive α₁β₀-rule) and IG demonstrate higher faithfulness to known relevant pixels, an outcome advocated for future XAI research.
• Limitations of previous proxies: Randomization or retrain-based “sanity checks” (Adebayo et al., Hooker et al.) can yield conclusions discordant with ground-truth evaluations, possibly because such interventions shift the testing regime away from its natural context.

A plausible implication is that future XAI research should prioritize evaluation on tasks with unambiguous relevance annotation and frameworks like CLEVR-XAI, supplementing or superseding noisier proxy metrics and subjective measures (Arras et al., 2020).

7. Practical Impact and Public Availability

CLEVR-XAI, including all datasets, relevance masks, evaluation scripts, and baseline models, is openly available at https://github.com/ahmedmagdiosman/clevr-xai. It is positioned as a standard benchmark for rigorous, reproducible progress in visual explainability by directly enabling ground-truth–based comparisons and highlighting both the strengths and the limitations of popular explanation techniques. The dataset and protocol have already spurred new insights regarding the faithfulness and efficiency of various heatmapping approaches and are expected to continue shaping the design and assessment of explainability systems in VQA and related visual domains (Arras et al., 2020).