---
title: 'EXP: Explainability Score for AI Transparency'
url: https://www.emergentmind.com/topics/explainability-score-exp
type: topic
---

# EXP: Explainability Score for AI Transparency

Explainability Score (EXP) is a quantitative or structured qualitative metric designed to assess the degree to which artificial intelligence (AI) or machine learning (ML) systems render their internal decision-making transparent and understandable to various stakeholders. EXP scores are motivated by the need for rigorous, repeatable evaluation of explainability across diverse AI models, domains, and explanation modalities.

## 1. Formal Definitions and Variants of EXP

Multiple formulations of EXP have been proposed, each grounded in distinct theoretical and methodological traditions:

### A. Machine-Centric Impact Score and Impact Coverage

Lin et al. [1910.07387] introduce a composite machine-centric EXP metric for image classification systems, constructed from two fundamental measures:

- **Impact Score ($I$):** For input $x$ and classifier $N$, an explainability method $M$ returns a binary mask $c \subseteq x$ identifying “critical factors.” The ablated input $x' = x - c$ is classified as $(y', z') = N(x')$. With threshold $\tau \in (0, 1)$, $I$ is defined as:
  $$
  I = \frac{1}{n} \sum_{i=1}^n \left[ (y'_i \neq y_i) \vee (z'_i \leq \tau z_i) \right]
  $$
  where $y$ is the predicted label and $z$ the confidence. $I_\mathrm{strict}$ counts only label flips.

- **Impact Coverage ($I_\mathrm{coverage}$):** For adversarial perturbations $a \subseteq x$, measures IoU overlap between $a$ and mask $c$:
  $$
  I_\mathrm{coverage} = \frac{1}{n} \sum_{i=1}^n \frac{\left| a_i \cap c_i \right|}{\left| a_i \cup c_i \right|}
  $$

- **Composite EXP:** A weighted sum (after normalization),
  $$
  \mathrm{EXP} = \alpha \, \widehat{I} + (1-\alpha)\, \widehat{I}_\mathrm{coverage}
  $$
  with $\alpha \in [0,1]$ balancing general and adversarial scenarios.

### B. Questionnaire and Factor-Analysis-Based EXP

Chen & Eickhoff [2306.10175] operationalize EXP for information retrieval via a weighted sum over latent explainability factors derived from a 19-item user questionnaire:

$$
\mathrm{EXP} = \mathrm{MinMaxNorm} \left( \sum_{f \in \mathcal{F}} w_f \sum_{i \in I_f} w_i r_i \right)
$$

where $\mathcal{F}$ is the set of factors, $w_f$ and $w_i$ the respective factor and item loadings, $r_i$ the user’s response, and values normalized to $[0,1]$.

### C. Degree of Explainability (DoX/EXP) via Information Pertinence

Sovrano & Vitali [2109.05327] define EXP as the average coverage of archetypal question-aspect pairs by pertinent details extracted from explanation texts, estimated via language model embeddings and cosine similarity. For details $D$, aspects $A$, question templates $Q$, and content $\Phi$:

$$
\mathrm{EXP}(\Phi) = \frac{1}{|Q|\,|A|} \sum_{q \in Q}\sum_{a \in A} \sum_{\substack{d \in D_a\\p(d,q_a)\geq t}}\!p(d,q_a)
$$

where $p(d,q_a)$ is the pertinence of detail $d$ to question $q_a$.

## 2. Factor Structure and Measurement Dimensions

The explainability captured by EXP is inherently multidimensional. Chen & Eickhoff [2306.10175] identify six core factors through exploratory factor analysis:

| Factor                    | Group         | Description                                                     |
|---------------------------|--------------|-----------------------------------------------------------------|
| Global Interpretability   | Roadblocks   | Understanding overall system logic                              |
| Local Interpretability    | Roadblocks   | Grasping why specific results were returned                     |
| Transparency              | Utility      | Visibility of internal decision signals                         |
| Justification             | Utility      | Perceived soundness of provided reasons                         |
| Granularity               | Utility      | Appropriateness of explanation detaillevel                       |
| Sufficiency               | Utility      | Adequacy of information to enable actionable insight            |

Each factor aggregates specific survey items, with empirically determined item and factor weights.

## 3. Algorithmic Implementations

### A. Machine-Centric Procedures (Vision)

- Identify salient region $c$ per explanation method $M$ for each test input $x$.
- Ablate $c$ to produce $x'$.
- Record change in predicted label ($I_\mathrm{strict}$) and/or drop in confidence by $\tau$ ($I$).
- For adversarial inputs, compute IoU of $c$ with patch $a$ ($I_\mathrm{coverage}$).
- Normalize and aggregate as $\mathrm{EXP} = \alpha \widehat{I} + (1-\alpha)\widehat{I}_\mathrm{coverage}$.

### B. Language-Model-Based EXP (DoXpy Pipeline)

- Extract subject–predicate–object “details” $d$ using dependency parsing from explanation text $\Phi$.
- For each aspect $a \in A$ and question archetype $q \in Q$, form $q_a$, embed with a sentence encoder.
- Compute pertinence $p(d,q_a)$; filter details by threshold $t$ and redundancy $r$.
- Sum pertinence scores to yield per-question, per-aspect, and final EXP as described above.

### C. Questionnaire Scoring Pipeline

- Administer structured questionnaire to users post-system interaction.
- Calculate weighted sum of responses per factor; aggregate and normalize for final $\mathrm{EXP} \in [0,1]$.

## 4. Empirical Validation and Interpretation

### A. Model Impact Analysis (Vision)

On ResNet-50/ImageNet, Impact Score $I$ quantifies the causal salience of regions picked out by XAI methods:

- GSInquire: $I \approx 76\%$ (confidence/label strongly affected by ablating mask)
- Expected Gradients: $I \approx 51\%$
- SHAP: $I \approx 44\%$
- LIME: $I \approx 38\%$

GSInquire identified adversarial patches with $I_\mathrm{coverage}$ up to $13$–$20\%$, while LIME was rarely aligned ($0.4$–$1.5\%$).

### B. Human Alignment Studies

In retrieval [2306.10175] and domain tasks [2109.05327], EXP scores calculated post-explanation correlate with human utility and effectiveness:

- In IR, EXP distinguished systems with and without transparent signal visualizations ($0.67$ vs $0.44$).
- In both finance and healthcare, increase in EXP for more comprehensive XAI corresponded to higher user task effectiveness, with statistical significance $p < 0.01$.

### C. Qualitative Checklist Approaches

Winikoff et al. [2502.09861] provide a structured scoresheet covering source code availability, explanation veracity, global/local explanation features, concepts, and automation, functioning as a multidimensional rubric rather than a scalar score.

## 5. Scope, Generalization, and Limitations

### Strengths

- **Machine-centric EXP:** Does not require human annotations or visual inspection; measures model response to perturbation directly [1910.07387].
- **Factor-based and DoX EXP:** Captures breadth and relevance of explanations in terms of archetypal queries and user-elicited dimensions [2306.10175][2109.05327].
- **Empirical alignment:** Correlates with user task success and subjective perception of understanding.

### Limitations

- **Dependence on perturbation operator:** Machine-centric $I$ is sensitive to how regions are ablated; other masking types may alter results [1910.07387].
- **Thresholds and normalization:** Choices such as $\tau$ or item/factor weighting affect sensitivity and comparability.
- **Domain specificity:** Vision-centric metrics may not transfer directly to NLP or recommendation without redefining "deletion" or "mask."
- **Coverage vs. faithfulness:** EXP quantifies the presence and informativeness of explanations but does not guarantee their correctness or faithfulness to model internals [2109.05327].
- **Scoresheet limitations:** The qualitative approach [2502.09861] ensures broad coverage but lacks aggregation or comparability unless a custom scoring rubric is imposed.

## 6. Practical Applications and Extensions

- **Benchmarking XAI algorithms:** Machine-centric and DoX-based EXP can be used to empirically compare fidelity and informativeness across explanation algorithms (e.g., LIME, SHAP, GSInquire, TreeSHAP) [1910.07387][2109.05327].
- **Calibration of explainability requirements:** Scoresheets serve to map stakeholder needs onto system features and support traceable, standardized evaluation [2502.09861].
- **Extension to new domains:** EXP can be adapted to recommender systems, QA, and others by recalibrating questions, aspects, and item/factor structure [2306.10175].
- **Integration with fidelity metrics:** Combining user-centric and machine-centric scores may yield composite measures of both explainability and faithfulness [2306.10175].

## 7. Future Directions and Open Issues

Potential advancements include:

- **Dynamic and human-calibrated weighting:** Learning weighting parameters (e.g., $\alpha$ in machine-centric EXP or factor weights in SSE) from empirical user or task data [1910.07387][2306.10175].
- **Perturbation robustness:** Employing smoother or domain-appropriate perturbations to better assess actual criticality of identified features.
- **Multi-modal, multi-level explainability:** Extending algorithms and rubrics to support sequence models, structured data, and mixed modalities.
- **Faithfulness–explainability tradeoffs:** Investigating relationships between scores measuring coverage and informativeness (EXP) and those quantifying faithfulness to underlying decision logic.

Explainability Score (EXP), therefore, represents a family of metrics and frameworks—quantitative, factor-based, or rubric-driven—that enable standardized, reproducible, and context-sensitive evaluation of AI system explainability, each with distinct methodological foundations and domains of applicability [1910.07387][2306.10175][2109.05327][2502.09861].

Source: https://www.emergentmind.com/topics/explainability-score-exp