Diagnostic Faithfulness in Explanations

• Diagnostic Faithfulness is a measure of whether a model’s internal rationale, such as explanations or salience maps, accurately reflects the causal computation leading to a prediction.
• It employs techniques like counterfactual interventions, sufficiency and completeness tests, polarity checks, and perturbation benchmarks to assess the explanation-model relationship.
• Its applications span natural language, vision, audio, and software domains, enhancing the trustworthiness and diagnosticity of AI-generated explanations.

Searching arXiv for recent and core papers on diagnostic faithfulness, faithfulness tests, and related benchmarks. Diagnostic faithfulness denotes the evaluation of whether an explanation, rationale, salience map, thinking draft, or internal importance signal genuinely reflects the model behavior that produced a prediction, rather than merely appearing plausible to a human observer. Across recent work, the term is instantiated through counterfactual interventions, sufficiency and completeness tests, polarity-consistency checks, information-flow diagnostics, and perturbation-based benchmarks spanning natural language explanations, attention models, vision transformers, reasoning traces, audio-LLMs, video-LLMs, and log-based anomaly detectors (Atanasova et al., 2023, Liu et al., 2022, Wu et al., 2024, Jia et al., 22 May 2026, Xiong et al., 17 Mar 2026, Yang et al., 12 Mar 2026, He et al., 29 Sep 2025).

1. Conceptual foundations

In the natural-language-explanation setting, an explanation $e$ is faithful if the “reasons” it states for a prediction $y=f_p(x)$ genuinely causally contribute to that prediction under the model $f$. This definition is explicitly causal: if the decisive part of the input changes, a faithful explanation should register that change, and if the explanation states a set of reasons, those reasons should suffice to recover the prediction (Atanasova et al., 2023).

A broader philosophical formulation distinguishes two links. First, a trained model $h^*$ may be “reliable enough” relative to a target phenomenon $f$, written $f \leftrightarrow h^*$, in the sense that the model tracks outputs of the phenomenon to a contextually sufficient degree. Second, an explanation $g^*=E(h^*)$ may be “faithful enough” to the model, written $h^* \leftrightarrow g^*$, in the sense that it captures what the model computes or which inputs or features it uses. On this account, reliability is a justificatory verdict, whereas faithfulness is purely descriptive (Oh et al., 28 Jun 2026).

This distinction fixes a central misconception. Diagnostic faithfulness is not the same as plausibility, fluency, human agreement, BLEU, or even predictive usefulness. A model can generate explanations that look reasonable and still fail counterfactual or sufficiency tests, and an explanation can be fully faithful to a spurious shortcut used by a highly accurate model. The literature therefore treats diagnostic faithfulness as a property of the explanation–model relation, not as a direct warrant for claims about the phenomenon itself (Atanasova et al., 2023, Oh et al., 28 Jun 2026).

2. Recurrent formal criteria

Despite substantial variation in modality and task, several formal criteria recur. For local explanations, faithfulness is decomposed into consistency and sufficiency under the test-time distribution $\mu$. Global consistency is

$$m^c=\Pr_{X,X'\sim\mu}\bigl[f(X')=f(X)\mid e(X')=e(X)\bigr],$$

and global sufficiency is

$y=f_p(x)$0

These quantities depend on the deployment distribution rather than only on the explainer architecture, and the framework provides estimators and sample-complexity bounds for black-box settings (Dasgupta et al., 2022).

For sentence-level rationale generation, faithfulness is formulated through sufficiency and completeness. If $y=f_p(x)$1 is a binary sentence mask, $y=f_p(x)$2 keeps only selected sentences and $y=f_p(x)$3 keeps only unselected sentences, then faithfulness encourages the original prediction to be preserved on $y=f_p(x)$4 and not preserved on $y=f_p(x)$5. The corresponding reward is

$y=f_p(x)$6

where $y=f_p(x)$7 is the sufficiency count, $y=f_p(x)$8 is the completeness count, and $y=f_p(x)$9 is a sparsity penalty toward a target selection fraction $f$0 (Atanasova et al., 2021).

For chain-of-thought (CoT) reasoning, an information-flow framing treats faithfulness as mediation along $f$1, where $f$2 is the prompt, $f$3 the CoT trace, and $f$4 the answer. This yields three complementary properties. Sufficiency is instantiated as $f$5, completeness as a masked-KL direct-effect term

$f$6

and necessity through gradient-based dependence ratios such as $f$7, which measure how much answer sensitivity is concentrated in the CoT rather than in a prompt-to-answer shortcut (Jia et al., 22 May 2026).

For process evaluation by judges, recent work separates causality and coverage. A CoT is causally faithful if each step $f$8 follows from its prior context, and fully covered if it contains every intermediate inference needed to justify the final answer. Coverage is operationalized on a $f$9–$h^*$0 rubric, from “no meaningful steps” to “complete, step-by-step coverage” (Mittal et al., 5 Mar 2026).

3. Natural-language explanations and reasoning traces

The most direct textual instantiation of diagnostic faithfulness is the paired use of counterfactual and reconstruction tests for natural language explanations. Let an intervention $h^*$1 insert a contiguous token block $h^*$2 such that $h^*$3. If the resulting explanation $h^*$4 omits every token in $h^*$5, that is, if $h^*$6, then the explanation is unfaithful under the counterfactual test. In the reconstruction test, reasons $h^*$7 are extracted from the explanation and converted into a reconstructed input $h^*$8; if $h^*$9, the explanation is unfaithful under reconstruction (Atanasova et al., 2023).

This methodology was evaluated on e-SNLI, CoS-E, and ComVE using four T5-Base NLE model configurations and a learned T5-based counterfactual editor. The learned editor found more counterfactual interventions than a random insertion baseline, and the counterfactual test exposed 30–59% total unfaithful rates under the learned editor. The reconstruction test detected 7–10% unfaithfulness on e-SNLI and 25–40% on ComVE. No single model configuration consistently outperformed the others in faithfulness, and models that looked plausible or scored well on BLEU still failed these causal tests (Atanasova et al., 2023).

The same concern reappears in large reasoning models. For “thinking drafts,” faithfulness is split into Intra-Draft Faithfulness, which tests whether inserted counterfactual steps causally alter later steps and the draft conclusion, and Draft-to-Answer Faithfulness, which tests whether the final answer depends on the edited draft conclusion or is recomputed downstream. Across six state-of-the-art LRMs, overall intra-draft faithful rates ranged from roughly 45.5% to 68.2% across GPQA and MMLU, backtrack insertions were substantially more faithful than Continue insertions, and answer-stage behavior often rewrote the draft rather than following it (Xiong et al., 19 May 2025).

Process-level evaluation by LLM judges introduces a different diagnostic object: not the model’s own explanation, but a judge’s ability to assess faithfulness. C2-Faith constructs controlled perturbations from PRM800K to test binary causal detection, causal step localization, and coverage scoring. The results show that model rankings depend strongly on task framing, no single judge dominates all settings, judges exhibit a substantial gap between detecting an error and localizing it, and coverage judgments are systematically inflated for incomplete reasoning (Mittal et al., 5 Mar 2026).

A further generalization treats CoT faithfulness as information flow and uses entropy-based, masked-KL, and gradient-based diagnostics to distinguish mediated reasoning from prompt-to-answer shortcuts. Across hinted arithmetic, reward-hackable code repair, and DAPO-Math wrong-hint injection, update-time interventions such as update-mask, gradient-mask, CoT-only gradients, and FACT shift both behavioral and structural indicators toward stronger CoT mediation. The same work identifies a low-entropy failure mode of KL-based diagnostics, with gradient-based measures remaining more stable (Jia et al., 22 May 2026).

4. Attention, salience, and local explanation diagnostics

Attention-based explanations expose a specific failure mode: weakness in identifying the polarity of feature impact. In the faithfulness violation test, an explanation method assigns weights $f$0 to features, and the true impact polarity is measured by

$f$1

For the feature $f$2, a faithfulness violation occurs when $f$3 and $f$4 disagree. Averaging this indicator yields the violation ratio $f$5 (Liu et al., 2022).

Empirically, this polarity test changes the ranking of explanation methods. Raw attention, despite its prevalence, has violation ratio $f$6 on general/text models and up to $f$7 on transformers. Attention$f$8Gradient yields $f$9 on LSTM/CNN models and $f \leftrightarrow h^*$0 on transformers, while LRP-based attention has high violation comparable to or worse than raw attention. Even methods scoring well on importance-only metrics such as AUC-TP, Sufficiency, and RankCorr can have high violation ratios, which is precisely why the test is diagnostic rather than merely correlational (Liu et al., 2022).

For Vision Transformers, the corresponding concern is whether salience maps reflect the influence of pixels or patches on the prediction. The Salience-guided Faithfulness Coefficient (SaCo) partitions pixels into ordered groups $f \leftrightarrow h^*$1, compares the salience difference $f \leftrightarrow h^*$2 between groups to the confidence-drop difference $f \leftrightarrow h^*$3 induced by perturbing those groups, and aggregates pairwise signed weights into a normalized coefficient $f \leftrightarrow h^*$4. Random Attribution analytically yields $f \leftrightarrow h^*$5, rank correlations between SaCo and AUC/AOPC/Log-Odds/Comprehensiveness are very low (about $f \leftrightarrow h^*$6), and gradient weighting plus multi-layer aggregation markedly improves the faithfulness of attention-based explanations (Wu et al., 2024).

A related line of work argues that traditional perturbation tests suffer from the random dominance problem, especially on complex data. In reduction, synthesis, and augmentation tests, randomly selected features can produce larger output changes than purportedly important features because masking or synthesis pushes inputs off-manifold and high-dimensional models distribute reliance broadly. To address this, three trend-based tests are proposed: EMBT, PTT, and EMT. Each computes a Pearson correlation between a known in-distribution trend of model behavior and the trend of the explanation. Across image, NLP, segmentation, and security tasks, these tests identify Integrated Gradients and SG-SQ-IG as the most faithful methods and avoid the OOD artifacts of traditional tests (He et al., 2023).

These results reinforce a general point already formalized for local explanations: faithfulness is not a property of visual appeal, and the relevant probabilities are distribution-dependent. A method can be locally coherent, globally inconsistent, or sufficient only on a narrow deployment distribution, which is why empirical estimators are defined directly with respect to $f \leftrightarrow h^*$7 (Dasgupta et al., 2022).

5. Diagnosticity of faithfulness metrics

Diagnostic faithfulness has generated a second-order problem: how to evaluate the evaluators. In a comparative study of feature-attribution metrics, diagnosticity is defined as

$f \leftrightarrow h^*$8

the probability that metric $f \leftrightarrow h^*$9 agrees with the ground-truth ordering when one interpretation is in fact more faithful than another. Because true faithfulness is usually unknown, the work introduces an $g^*=E(h^*)$0-approximation $g^*=E(h^*)$1 and estimates it using an $g^*=E(h^*)$2-faithfulness golden set of interpretation pairs (Chan et al., 2022).

On SST, IMDB, and AG News with BERT and CNN models, the golden set contains $g^*=E(h^*)$3 pairs per model–dataset setting, with each pair formed from one model-based attribution and one uniformly random attribution. Under this protocol, sufficiency (SUFF) and comprehensiveness (COMP) achieve the highest diagnosticities, averaging about 73–77% on BERT and CNN across all datasets; MONO is a close third at about 73–75%; CORR and DFFOT lie in the mid-60% range; and DFMIT never exceeds about 17%. COMP and SUFF also require only five model forward passes per attribution, placing them in the high-diagnosticity, low-time-complexity “sweet spot” (Chan et al., 2022).

A causal extension of this idea targets natural-language explanations and CoT metrics directly. Causal Diagnosticity constructs faithful–unfaithful explanation pairs by knowledge editing, using In-Context Editing and MEMIT to create post-edit models whose prediction remains invariant but whose internal reason changes. The framework spans fact-checking, analogy, object counting, and multi-hop reasoning. Across seven metrics, including CC-SHAP, Simulatability, Counterfactual Edits, Early Answering, Filler Tokens, Adding Mistakes, and Paraphrasing, no metric consistently exceeds random chance across all tasks. Simulatability and Counterfactual Edits are near zero in several settings, whereas Paraphrasing and, in some tasks, CC-SHAP attain the highest diagnosticity (Zaman et al., 26 Feb 2025).

This line of work reframes metric design itself as a diagnostic problem. A faithfulness metric is useful only if it reliably prefers a genuinely model-driven explanation over a spurious alternative. When that ranking fails, the metric is not merely noisy; it ceases to be informative about faithfulness in the first place (Chan et al., 2022, Zaman et al., 26 Feb 2025).

6. Multimodal and operational extensions

The diagnostic-faithfulness perspective has been generalized to settings in which the model’s internal reasoning is mediated by audio, video, logs, or summarization preferences rather than standard token-level rationales.

Domain	Diagnostic target	Metrics
Audio MLLMs	acoustic grounding under semantic conflict and misleading prompts	Acc, ASS, ARS, EDI
Video-LLMs	reliability under visual degradation, evidence corruption, and temporal intervention	RR, TSS
Log anomaly detection	diagnostic causality and diagnostic consistency	HR@k, PR@k, MAP@k, MRR, SR
Dialogue summarization	model preference for faithful over errorful summaries	GS, FS

For Audio MLLMs, DEAF defines acoustic faithfulness as grounding predictions in the actual sound signal rather than in transcript semantics or prompt text. It builds over 2,700 conflict stimuli across emotional prosody, background sounds, and speaker identity, and evaluates models at three interference levels. Across seven Audio MLLMs, ASS is often high at 70–95%, indicating that encoders do register acoustic changes, but ARS remains low, especially for emotional prosody and background sounds, yielding what the authors call a “perception–trust gap.” Under dual interference, nearly all ESC/BSC ARS values collapse below 7% for six of seven models, which the benchmark interprets as overwhelming text dominance (Xiong et al., 17 Mar 2026).

For Video-LLMs, INFACT diagnoses faithfulness and factuality hallucinations under four modes: Base, Visual Degradation, Evidence Corruption, and Temporal Intervention. Reliability is quantified by Resist Rate for label-preserving perturbations and Temporal Sensitivity Score for order-sensitive interventions. Across 14 models, evidence corruption hurts more than visual degradation, temporal intervention yields the largest degradation, and many open-source baselines exhibit near-zero TSS on factuality, indicating pronounced temporal inertia (Yang et al., 12 Mar 2026).

For log-based software reliability maintenance, diagnostic faithfulness is defined through diagnostic causality and diagnostic consistency. The first asks whether attention highlights ground-truth root-cause events; the second asks whether anomaly confidence depends on the events receiving the highest attention. Evaluation uses attention-based root cause localization and event perturbation, with HR@k and SR as primary scores. Existing methods perform poorly on both tasks, and FaithLog improves diagnostic faithfulness via a causality-guided attention mechanism and adversarial consistency learning, reaching HR@1 = 70.0% and SR = 88.6% on BGL, with similarly strong gains on Thunderbird and an industrial dataset (He et al., 29 Sep 2025).

For dialogue summarization, faithfulness is probed at the model level rather than only by post-hoc annotation. FacEval constructs multiple-choice diagnostic questions from rule-based transformations such as Speaker Swap, Entity/Pronoun/Date/Number Swap, and Negation. A summarization model is scored by how often it assigns a higher length-normalized generation score $g^*=E(h^*)$4 to a faithful paraphrase than to a corrupted variant, producing the model-level score $g^*=E(h^*)$5. On SAMSum, FacEval achieves an average Spearman correlation of about $g^*=E(h^*)$6 with underlying model rankings, substantially outperforming ROUGE, BLEU, BERTScore, FactCC$g^*=E(h^*)$7, FEQA, and NLI in this setting (Wang et al., 2022).

7. Limits, misconceptions, and open directions

A recurrent empirical finding is that plausible or high-performing systems can still be diagnostically unfaithful. NLEs that score well on BLEU or human judgments routinely fail causal and sufficiency tests; judge models can detect causal errors better than they localize them; audio and video systems can score well on standard benchmarks while defaulting to text or temporal inertia under conflict; and clean accuracy does not reliably proxy reliability under induced perturbations (Atanasova et al., 2023, Mittal et al., 5 Mar 2026, Xiong et al., 17 Mar 2026, Yang et al., 12 Mar 2026).

Another misconception is that passing one or two tests establishes “full faithfulness.” The NLE literature explicitly treats counterfactual and reconstruction tests as necessary but not sufficient, and the philosophy of scientific XAI makes the same point at a higher level: even perfect reliability and perfect explanation–model faithfulness do not show that the model reproduces the true causal or mechanistic structure of the phenomenon. External corroboration, targeted interventions on the phenomenon, architectural constraints, and alignment with established theory remain necessary for mechanistic claims (Atanasova et al., 2023, Oh et al., 28 Jun 2026).

A further limitation concerns annotation and verification itself. In summarization, the boundary between faithful content and acceptable external knowledge is often ambiguous. VeriGray addresses this by introducing Out-Dependent as an intermediate category for cases where $g^*=E(h^*)$8 but $g^*=E(h^*)$9, alongside Explicitly, Implicitly, Contradicting, Fabricated, Ambiguous, and No-Fact labels. In its 2,044-sentence benchmark, about 8% of generated sentences fall into the Out-Dependent category, and even strong zero-shot detectors perform poorly on Out-Dependent and Ambiguous cases (Ding et al., 24 Oct 2025).

Open directions in the literature are correspondingly diagnostic. Proposed next steps include richer, semantics-aware counterfactual insertion generators; learned reconstruction models to replace hand-crafted heuristics; adaptive grouping and alternate weighting schemes for SaCo; regularizers that optimize faithfulness during training; structured checks for judge coverage inflation; architectural changes that more tightly couple thinking and answering stages; and localization diagnostics that identify the frames, tokens, or hidden representations driving hallucinations (Atanasova et al., 2023, Wu et al., 2024, Mittal et al., 5 Mar 2026, Xiong et al., 19 May 2025, Yang et al., 12 Mar 2026).

Taken together, these works position diagnostic faithfulness not as a single metric but as a family of causal, perturbational, and distribution-sensitive tests for whether visible explanations actually track model computation. The unifying idea is stable across domains: explanations are trustworthy only when they preserve the right prediction under the right interventions, fail when decisive reasons are removed or contradicted, and outperform random or spurious alternatives under a metric that is itself diagnostically sound.