Explanation Method Robustness (EMR)

• Explanation Method Robustness (EMR) is a framework that ensures explanation methods produce consistent outputs for similar inputs while changing appropriately for significant variations.
• It employs quantifiable metrics such as symmetry preservation, probabilistic smoothness, and ranking stability to evaluate and certify the reliability of explanation outputs.
• Robust explanations are critical for trusting AI systems, necessitating integration of EMR with faithfulness analyses and tailored regularization techniques to mitigate method-specific artifacts.

Explanation Method Robustness (EMR) denotes the robustness of an individual explanation method: explanations should remain stable under variations that ought not matter for the explanatory goal, and should change when the model, the input-output pair, or the explanatory context changes in ways that are genuinely relevant. In recent work, this requirement has been formalized locally for input-output pairs and globally for models, and instantiated through symmetry preservation, probabilistic local smoothness, retraining robustness, ranking stability, and formal certification of gradient explanations (Boge et al., 18 Aug 2025, Crabbé et al., 2023, Khan et al., 2022, Forel et al., 2022, Wicker et al., 2022).

1. Conceptual scope

A general recent formalization states that a local explanation method is robust if similar input-output pairs receive similar explanations and distinct input-output pairs receive distinct explanations. In the notation of that framework, local EMR requires

$$d'(z_i, z_j) < \varepsilon \Rightarrow d(X(z_i), X(z_j)) < \delta$$

and

$$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$

with analogous global conditions

$$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$

and

$$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$

Probabilistic relaxations replace the second implication by an upper bound on the probability that distinct cases or models still produce nearly identical explanations (Boge et al., 18 Aug 2025).

This formulation separates EMR from explanatory robustness (ER). EMR concerns within-method stability and sensitivity; ER concerns agreement across different XAI methods that pursue the same explanatory goal. The same work argues that EMR is a prior requirement for trust, whereas ER is additionally needed because different methods may otherwise converge on method-specific artifacts; conversely, ER alone is insufficient because multiple methods may agree on the same but still wrong explanation (Boge et al., 18 Aug 2025).

The broader literature operationalizes this general requirement differently according to explanation type and perturbation model. For symmetry-aware models, EMR means that explanations inherit invariance or equivariance from the explained predictor (Crabbé et al., 2023). For randomized ensembles, EMR is the probability that a counterfactual explanation remains valid after retraining on the same data (Forel et al., 2022). For removal-based and attribution-based explainers, EMR is often cast as giving similar explanations for similar inputs, quantified through local smoothness or local Lipschitz-like behavior (Khan et al., 2022, Pala et al., 2024).

Setting	Explanation object	EMR criterion
Symmetry-aware models	saliency maps, example importance, concept indicators	invariant or equivariant explanations under group action $G$ (Crabbé et al., 2023)
Randomized ensembles	counterfactual explanations	validity after retraining with probability at least $1-\alpha$ (Forel et al., 2022)
Similar inputs or models	local or global explanations	EMR-1 and EMR-2 stability/sensitivity inequalities (Boge et al., 18 Aug 2025)
Locally smooth predictors	feature attributions	explainer astuteness within radius $r$ (Khan et al., 2022)

2. Formal models of robustness

One of the most explicit mathematical treatments is the symmetry-based formulation for invariant and equivariant predictors. Data are signals $x \in X(\Omega,C)$ on a finite domain $\Omega$ with channels $C$, and a finite group $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$0 acts on the signal space through a representation $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$1. If a classifier $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$2 is invariant, $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$3 for all $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$4; if $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$5 has structured output, equivariance is written $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$6. Explanations are then maps $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$7, and EMR requires either explanation invariance, $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$8, or explanation equivariance, $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$9, depending on whether the explanation indexing itself should transform. Saliency maps for translation-equivariant CNNs are therefore expected to be equivariant, whereas example-importance vectors and concept-presence indicators are expected to be invariant (Crabbé et al., 2023).

A different formal route links explanation robustness to local smoothness of the predictor. Explainer astuteness is defined as

$$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$0

which quantifies the probability that explanation differences are at most proportional to input differences inside an $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$1-ball. This is connected to probabilistic Lipschitzness of the predictor, yielding lower bounds for SHAP, RISE, and remove-individual explainers such as CXPlain: locally smooth predictors induce locally robust explanations, with explicit constants such as $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$2 for SHAP and remove-individual explainers and $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$3 for RISE under the stated assumptions (Khan et al., 2022).

For counterfactual explanations in randomized ensembles, robustness is probabilistic with respect to retraining randomness. If $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$4 is an ensemble trained on fixed data and $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$5 its average score, a counterfactual is robust when

$$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$6

equivalently

$$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$7

The paper links this ensemble-level chance constraint to the success probability of a base learner, replacing the probabilistic constraint by deterministic thresholds computed from the trained ensemble via sample-average approximation (Forel et al., 2022).

A further formalization emphasizes feature subsets and adversarial robustness. For a classifier $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$8, an input $$D(f,f') < \varepsilon \Rightarrow d(X(f),X(f')) < \delta$$9, and a subset $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$0, the minimum adversarial perturbation restricted to $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$1 is

$$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$2

Large robustness on $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$3 supports the necessity of the selected set $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$4, whereas small robustness on $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$5 supports sufficiency. This makes EMR analyzable without specifying a feature-removal baseline (Hsieh et al., 2020).

3. Metrics and evaluation protocols

The symmetry-based framework introduces two generic scores. For invariant explanations,

$$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$6

and for equivariant explanations,

$$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$7

For real-valued explanations, $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$8 can be cosine similarity; for categorical explanations, it can be per-coordinate accuracy. Large groups such as $$D(f,f') > \varepsilon' \Rightarrow d(X(f),X(f')) > \delta'.$$9 are handled by Monte Carlo estimation. The same framework notes that with $G$0 and $G$1, Hoeffding’s inequality yields a probability at most $G$2 that the estimator deviates by at least $G$3 (Crabbé et al., 2023).

Several works focus on local instability under perturbations. One proposal evaluates ranking stability rather than raw saliency magnitudes by introducing explanation thickness, which measures preservation of the top-$G$4 salient set along paths inside an $G$5 ball; a sufficient condition for full top-$G$6 preservation is that the margin between the $G$7-th and $G$8-th features exceeds $G$9 (Chen et al., 2023). Another line evaluates feature-attribution stability with the average incremental ratio

$1-\alpha$0

where $1-\alpha$1 contains $1-\alpha$2-close inputs whose predicted label is unchanged; lower values indicate more robust explanations (Pala et al., 2024).

Statistical uncertainty estimation has also been incorporated directly into EMR evaluation. MeTFA samples multiple noisy explanations, performs exact binomial median tests for each feature, constructs exact median confidence intervals, and defines a MeTFA-smoothed map by averaging only the central order statistics. It further proposes robust faithfulness metrics such as robust insertion, robust deletion, and robust overall score, obtained by averaging standard faithfulness measures over noisy inputs (Gan et al., 2022).

Text and natural-language settings introduce additional evaluation axes. In adversarial robustness for NLP attributions, explanation instability is measured through cosine similarity and $1-\alpha$3 distance between token-attribution vectors under semantics-preserving perturbations that keep the model prediction fixed (Atmakuri et al., 2022). For LLM-based explanation agents in recommender systems, robustness is decomposed into semantic, keyword, structural, and length consistency, aggregated over five perturbation types and five severity levels:

$1-\alpha$4

with an overall EMR obtained by averaging across perturbations and severities (Zhang et al., 27 Jan 2026).

Clinical explainability work adds agreement-based diagnostics. Global explanation rankings are compared using Feature Agreement,

$1-\alpha$5

and Rank Agreement,

$1-\alpha$6

to quantify how much SHAP, DTD, and logistic-regression coefficients agree with one another and with expert clinical expectations (Brankovic et al., 2023).

4. Improving and certifying EMR

One generic improvement strategy is symmetry aggregation. For any explanation $1-\alpha$7, an invariant version is obtained by

$1-\alpha$8

and an equivariant version by

$1-\alpha$9

This construction is guaranteed to enforce invariance or equivariance, its cost scales linearly with the number of sampled group elements, and Monte Carlo variants are used when $r$0 is large (Crabbé et al., 2023).

Robustness-aware training and regularization constitute a second family. R2ET augments the task loss with a surrogate for explanation thickness that enlarges pairwise margins between top-$r$1 and non-top-$r$2 features, optionally combined with curvature control, thereby anchoring the top salient set under stealthy attacks (Chen et al., 2023). REGEX couples input gradient regularization and virtual adversarial training with explanation-guided training: low-IG tokens are masked, and attention is aligned with attributions through a KL term. Its final loss is

$r$3

with reported default weights $r$4, $r$5, $r$6, and $r$7 (Li et al., 2023).

Counterfactual robustness in ensembles is improved by replacing the naive threshold $r$8 with a deterministic score threshold derived from the chance constraint. Direct SAA uses $r$9, while Robust SAA uses the Agresti–Coull buffered threshold

$x \in X(\Omega,C)$0

The practical consequence is that counterfactuals are forced into regions where base learners agree, rather than into unstable decision-boundary neighborhoods (Forel et al., 2022).

Other methods target explanation robustness through aggregation, smoothing, adversarial training, or certification. MeTFA improves stability by averaging only the central part of the sampled attribution distribution and produces feature-wise significance maps and confidence intervals (Gan et al., 2022). Robust explanation constraints for neural networks propagate intervals through forward and backward passes and train with a differentiable regularizer based on the width of the reachable explanation set, producing certified bounds on worst-case changes in gradient explanations under bounded perturbations of inputs or parameters (Wicker et al., 2022). In recommender systems, adversarial training on the item-feature matrix $x \in X(\Omega,C)$1 uses

$x \in X(\Omega,C)$2

with $x \in X(\Omega,C)$3, to preserve explanation quality under white-box attacks (Vijayaraghavan et al., 2024). For Random Forest explanations, AXOM replaces uniform averaging over all tree-SHAP values by discriminative averaging over only those trees whose prediction agrees with the ensemble prediction, reducing explanation variance from disagreeing weak learners (Pala et al., 2024).

5. Empirical evidence across domains

Empirical studies consistently show that explanation robustness is neither automatic nor uniform across explanation types. In symmetry-aware settings, feature-importance methods with non-invariant baselines frequently violate equivariance, whereas invariant baselines such as $x \in X(\Omega,C)$4 substantially improve $x \in X(\Omega,C)$5. On CINIC-10, evaluated with an STL10-trained model, feature-importance equivariance scores were reported as IG $x \in X(\Omega,C)$6, DeepLift $x \in X(\Omega,C)$7, and GradientShap $x \in X(\Omega,C)$8; example-importance invariance on the same shift was $x \in X(\Omega,C)$9 for TracIn and Influence, but lower for representation-based variants such as SimplEx-Equiv $\Omega$0 and Representation Similarity-Equiv $\Omega$1 (Crabbé et al., 2023).

Randomized-ensemble counterfactuals reveal a different failure mode. Naive validity is below $\Omega$2 on most data sets and can fall to $\Omega$3 on problems with many features. The reported table includes Adult $\Omega$4, Credit Card Default $\Omega$5, German Credit $\Omega$6, Online News $\Omega$7, Spambase $\Omega$8, and Student Performance $\Omega$9. Direct SAA and Robust SAA achieve target robustness on most data sets with only a small increase in distance from the original observation (Forel et al., 2022).

In NLP attribution, semantics-preserving perturbations can strongly disturb explanations even when the predicted label is unchanged. On SST-2 with RoBERTa-base and Integrated Gradients, misspelling attacks achieved an $C$0 success rate, synonym substitution $C$1, word inflection $C$2, and word deletion $C$3. The study reports that the explanation method can be largely disturbed for up to $C$4 of the tested samples with small changes in the input sentence and its semantics (Atmakuri et al., 2022). In text classification training, REGEX reports consistent gains in normalized sufficiency and comprehensiveness across six datasets and stronger Jaccard agreement in Different Initialization Tests; for example, scaled-attention Jaccard@25% rose to $C$5 from $C$6 for one seed pair, while CheckList attack success fell to $C$7 from $C$8 on an IMDB subset (Li et al., 2023).

User-facing explanation systems show analogous patterns. RobustExplain reports aggregate robustness scores of approximately $C$9 for LLaMA 3.1-70B, $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$00 for Qwen2.5-14B, and approximately $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$01 for both LLaMA 3.1-8B and Qwen2.5-7B, with larger models achieving up to approximately $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$02 higher stability. Length consistency was highest at approximately $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$03, structural consistency lowest at approximately $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$04, and robustness degraded only modestly from severity level 1 to 5, with slight recovery at the highest severity because models reverted to more generic explanations (Zhang et al., 27 Jan 2026).

Recommender-system studies and ensemble-specific explanation methods likewise report measurable gains from robustness-aware design. In robust explainable recommendation, attacked explanation F1 for CER on Kindle increased from $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$05 to $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$06, while EFM on CD increased from $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$07 to $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$08 under the defense (Vijayaraghavan et al., 2024). For Random Forest SHAP, AXOM reduced the average incremental ratio by $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$09 on Wine, $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$10 on Glass, $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$11 on Seeds, and $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$12 on Banknote, all with statistically significant RF-versus-AXOM differences below $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$13 (Pala et al., 2024). In clinical decision support using EMR data, FA and RA across SHAP, DTD, and logistic-regression coefficients were variable and generally poor to moderate; for paediatric readmission, SHAP and DTD each included at least three expert-suggested predictors among their top-5, whereas LR-L1 included only one (Brankovic et al., 2023).

Domain	Setting	Representative result
Symmetry-aware feature importance	CINIC-10 with STL10 model	IG $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$14, DeepLift $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$15, GradientShap $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$16 on $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$17 (Crabbé et al., 2023)
Randomized ensembles	robust counterfactuals	naive validity below $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$18 on most data sets; can fall to $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$19 (Forel et al., 2022)
NLP attributions	SST-2, RoBERTa-base, IG	misspelling attack success $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$20; synonym substitution $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$21 (Atmakuri et al., 2022)
LLM explanation agents	four local models from 7B to 70B	overall robustness approximately $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$22–$$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$23; larger models up to $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$24 higher stability (Zhang et al., 27 Jan 2026)
Random Forest explanations	AXOM versus RF SHAP	robustness improvements of $$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$25–$$d'(z_i, z_j) > \varepsilon' \Rightarrow d(X(z_i), X(z_j)) > \delta',$$26 (Pala et al., 2024)

6. Limitations, misconceptions, and open problems

A persistent misconception is that robust explanations are automatically faithful explanations. The literature repeatedly separates these notions. Symmetry work explicitly treats EMR as an additional axis of faithfulness rather than a replacement for infidelity, comprehensiveness, or sufficiency (Crabbé et al., 2023). MeTFA shows that robust faithfulness under noise can differ materially from standard insertion or deletion scores (Gan et al., 2022). The broader philosophical treatment argues that EMR is necessary but not sufficient for trust, and that even perfect cross-method agreement may still be wrong because of the ground-truth problem (Boge et al., 18 Aug 2025).

Another misconception is that model robustness and explanation robustness move together in a simple way. This is contradicted in several settings. Approximate model invariance can coexist with sharp degradation in explanation equivariance, especially for saliency under relaxed symmetry assumptions (Crabbé et al., 2023). Adversarial training for prediction does not necessarily improve ranking robustness of explanations, which is why R2ET targets explanation thickness directly rather than only predictive adversarial loss (Chen et al., 2023). The survey on LIME, SmoothGrad, and SHAP likewise frames robustness as having two sub-dimensions—stability and sample complexity—and stresses that sampling variance, hyperparameters, and distribution shift can undermine seemingly plausible explanations (Galinkin, 2022).

Method-specific assumptions also matter. Symmetry guarantees may require orthogonal or permutation representations, invariant baselines, or invariant internal layers (Crabbé et al., 2023). Counterfactual guarantees for randomized ensembles depend on IID base learners and convexity assumptions for the strongest asymptotic and finite-sample results (Forel et al., 2022). Formal certification of gradient explanations currently relies on interval abstractions and supports a restricted set of architectures and layers; bounds may become loose on deeper or more complex networks (Wicker et al., 2022). Text-robustness studies remain limited by semantic-similarity surrogates such as SBERT cosine and by the absence of broad task coverage beyond classification (Atmakuri et al., 2022).

Taken together, these results suggest that EMR is best understood as a family of context-dependent robustness requirements rather than as a single scalar property. The active research problems are correspondingly heterogeneous: choosing the right invariance group, perturbation family, or model-distance measure; calibrating thresholds and confidence intervals; separating stable spurious explanations from stable causal explanations; and scaling formal or statistical guarantees to contemporary architectures and modalities. The strongest consensus in the current literature is narrower but precise: explanations that fail basic stability or sensitivity tests cannot be considered reliable, and explanations that pass them should still be interpreted together with faithfulness analyses, domain constraints, and, where relevant, cross-method comparison (Boge et al., 18 Aug 2025, Crabbé et al., 2023)