Faithful Self-Explanations

Updated 15 December 2025

Faithful self-explanations are model outputs that accurately reflect the true computational processes behind each prediction.
Methodologies include direct architectural designs, hypercube search, and iterative post-hoc refinement to ensure explanation fidelity.
Empirical studies highlight that while formal metrics like SEV and NeuroFaith provide guarantees, challenges remain in scaling and balancing interpretability.

Faithful self-explanations are explanation outputs from machine learning models that verifiably capture the internal factors responsible for each specific prediction, without introducing discrepancies or artifacts from external surrogates, plausible but spurious rationales, or opaque post-hoc interpreters. In technical terms, a self-explanation is “faithful” if it reflects the true computational pathway by which the model arrived at its answer, such that concrete interventions—guided by the explanation—cause the model's behavior to consistently change as predicted. Research across model families, including deep neural networks, LLMs, graphical models, and logical rule engines, establishes that faithful self-explanation is a distinct and challenging goal, spanning both architectural and post-hoc paradigms.

1. Formal Definitions and Faithfulness Metrics

Faithfulness in self-explanation is defined as the alignment between the information in the explanation and the model's true decision-making process. Formally, for a model $M$ and input $x$ , an explanation $E(M, x)$ is faithful if its content causally or counterfactually influences $M$ 's prediction (Lyu et al., 2022). Common formalizations include:

Self-consistency check: If $E$ claims certain features or changes are responsible for prediction $y$ , then modifying $x$ according to $E$ should flip or alter $M(x)$ accordingly. For a dataset $D$ , faithfulness is

$F_\mathrm{avg}(M) = \frac{1}{|D|} \sum_{x \in D} \mathbf{1}\{ M(\operatorname{Modify}(x, E(M, x))) \sim y' \}$

where $y'$ is the predicted or intended new label (Madsen et al., 15 Jan 2024, Madsen, 27 Nov 2024, Doi et al., 8 Dec 2025).

Sufficiency/necessity: Explanations identify a subset of features/rationales such that (a) the prediction holds using only these features (sufficiency) and (b) changing them flips the prediction (necessity) (Lyu et al., 2022, Christiansen et al., 2023, Azzolin et al., 21 Jun 2024).
Volume-based and local gradient metrics: For counterfactual explanations, faithfulness can be measured by the alignment of input changes with the model’s own gradient or energy landscape, e.g.,

$\operatorname{faith}(x, x', y') = \frac{(x' - x)^\top \operatorname{sign}( \nabla_x s_{y'}(x) )}{\|x' - x\|_1}$

where $s_{y'}$ is the score for class $y'$ (Altmeyer et al., 2023).

Phi-CCT: For natural LLMs, the phi-coefficient of correlation between prediction impact ( $I_D$ ) and mention in the output explanation ( $M$ ) is used to measure explanation–intervention alignment (Siegel et al., 17 Mar 2025).

2. Model-Agnostic and Model-Specific Methodologies

Faithful self-explanations can be obtained by:

Direct architectural designs: Models are built to produce explanations alongside predictions, using mechanisms such as Neural Module Networks (NMNs), constrained structure outputs, or hard-masked decision components (Lyu et al., 2022). Construction ensures that for each prediction, the explanation object (e.g., rule, subgraph, feature set) is exactly what the model relied on.
BFS/hypercube search (Sparse Explanation Value – SEV): For tabular or simple input domains, explainability is realized by enumerating minimal feature subsets whose alteration suffices to flip a prediction. The SEV framework formalizes

$\mathrm{SEV}^+(x) = \min_{b \in \{0,1\}^p} \|b\|_0 \quad \mathrm{s.t.}\; f( x_b ) = 1$

and

$\mathrm{SEV}^-(x) = \min_{b \in \{0,1\}^p} \| 1 - b \|_0 \quad \mathrm{s.t.}\; f( x_b ) = 0$

These values are computed without any surrogate model, guaranteeing faithfulness (Sun et al., 15 Feb 2024).

Rule-based methods (DISCRET): In settings such as ITE estimation, explanations are synthesized as rules or clauses selected by a deep RL policy, ensuring that all samples sharing a rule explanation receive the same predicted effect—maximizing “consistency”

$\hat{\mathrm{consistency}}(x) = \frac{|\{ x' \in C_x \,:\, \hat{\mathit{ITE}}(x)=\hat{\mathit{ITE}}(x') \}| - 1}{|C_x| - 1}$

(Wu et al., 2 Jun 2024).

Iterative post-hoc refinement (e.g., FaithLM, SR-NLE): Starting from initial (often inaccurate) self-explanations, black-box LLMs are prompted to critique and refine their outputs based on explicit perturbations, counterexamples, or feature attributions, resulting in substantially improved faithfulness (Chuang et al., 7 Feb 2024, Wang et al., 28 May 2025).
Mechanistic alignment (NeuroFaith): Hidden state interpretability tools are used to extract internal circuit-level evidence (e.g., attention to bridge entities during reasoning), and explanation consistency is measured by overlap between what is represented in neural activations and what is claimed in the explanation (Bhan et al., 10 Jun 2025).

3. Strengths, Weaknesses, and Common Pitfalls

Strengths

Exactness and directness: For models such as decision sets, SEV, or modular architectures, explaining by direct interrogation of their computation or by minimal input changes ensures explanations are exact certificates of boundary-crossing or reasoning steps (Sun et al., 15 Feb 2024, Chuang et al., 7 Feb 2024).
Formal guarantees: In some frameworks (e.g., DISCRET), faithfulness is theoretically guaranteed in the rule language and by design every explanation determines the corresponding model prediction (Wu et al., 2 Jun 2024).
Quantitative, testable metrics: Self-consistency checks, intervention-based tests, and aggregatable statistics allow systematic and robust evaluation of faithfulness across tasks, domains, and explanation styles (Madsen et al., 15 Jan 2024, Siegel et al., 17 Mar 2025).

Weaknesses and Limitations

Faithfulness–plausibility trade-off: High-quality, human-appealing explanations (plausibility) often diverge from true model logic, especially in unrestricted free-form natural language (Agarwal et al., 7 Feb 2024). Models can generate fluent and convincing rationales that do not match any real internal feature use.
Dependence on architecture and supervision: Predict-then-explain and loosely coupled joint models cannot guarantee faithfulness; only explain-then-predict or tightly supervised chains achieve this property (Lyu et al., 2022).
Scaling issues: Faithful hypercube enumeration becomes intractable for high-dimensional domains without further structural constraints (Sun et al., 15 Feb 2024).
Uninformative explanations: For maximally expressive architectures (e.g., injective GNNs), strict faithfulness can be trivial: the only faithful explanation is the full model input, which is not informative (Azzolin et al., 21 Jun 2024).
Unreliable faithfulness: For large LLMs, self-explanation faithfulness varies dramatically with task, explanation type, and prompt template. Even large models often fail basic faithfulness checks, and task-level and model-level decisions cannot reliably be inferred from their explanations without intervention-based audits (Madsen et al., 15 Jan 2024, Doi et al., 8 Dec 2025).

4. Empirical Characterization and Results

Empirical studies document both the promise and limits of faithful self-explanation.

Sparse explanations in tabular/classification: Off-the-shelf models (L1/L2-Logistic, MLP, GBDT) yield SEV values of 1–2 per prediction; thus most decisions can be explained by 1–2 changing features. Optimizing for SEV can drive decision sparsity to the theoretical minimum with negligible drop in accuracy (Sun et al., 15 Feb 2024).
LLM self-explanation faithfulness: Over 62 models, larger LLMs improve φ-CCT faithfulness significantly, but instruction-tuning shifts verbosity along a TPR/FPR Pareto frontier rather than improving the best attainable faithfulness (Siegel et al., 17 Mar 2025). SR-NLE self-critique reduces unfaithfulness rates from ~55% to ~36% on NLE tasks (Wang et al., 28 May 2025).
Self-explainable GNNs: Even with custom faithfulness-driven architectures, true faithfulness is not achieved in practice—prototype-based and bottleneck approaches fail to yield explanations superior to random subgraphs on hard metrics, and performance varies widely across datasets (Christiansen et al., 2023, Azzolin et al., 21 Jun 2024).
Recommendation and ITE systems: FIRE combines SHAP attributions with language generation to produce explanations whose sentiment matches model prediction with high faithfulness, and DISCRET achieves consistency rates near 100%, outperforming LIME/SHAP/Anchor (<20%) (Sani et al., 7 Aug 2025, Wu et al., 2 Jun 2024).

5. Recent Advances: Faithfulness Optimization and Generalization

Research is turning toward systematically improving the faithfulness of self-explanations:

Fidelity-optimized training: Training models using “pseudo-faithful” one-word explanations generated by feature attribution can substantially improve faithfulness across styles and even generalize to unseen tasks, with cross-style transfer observed among attribution, redaction, and counterfactual explanation formats (Doi et al., 8 Dec 2025).
Prompt optimization and iterative refinement: FaithLM style systems iteratively optimize both explanations and triggers, achieving significant gains in explanation fidelity by constructing contrary statements and measuring output flips (Chuang et al., 7 Feb 2024).
Self-critique and feedback: Allowing the model to use natural-language or attribution-based critique and refinement in a zero-shot, post-hoc regime produces refined explanations with much lower unfaithfulness rates, especially when attention- or IG-based feedback highlights implicit token influences (Wang et al., 28 May 2025).

6. Open Challenges and Future Directions

Several open directions and controversies are identified:

Faithfulness metric non-equivalence: Different intervention sets or divergence choices for sufficiency/necessity scores yield non-comparable faithfulness metrics. Care must be taken when comparing or optimizing for any particular faithfulness quantity (Azzolin et al., 21 Jun 2024).
Triviality in highly expressive models: In injective GNNs and other universally expressive architectures, the only strictly faithful subgraph or feature set is the entire input, rendering “faithful” explanations vacuously uninformative (Azzolin et al., 21 Jun 2024).
Faithfulness in the presence of model bias or shortcut learning: LLMs and deep models may encode spurious correlations or use features undetectable via surface explanations; mechanistic audits (e.g., NeuroFaith's comparison of neural activity and explanation content) are crucial (Bhan et al., 10 Jun 2025).
Reconciling faithfulness and interpretability: Current faithfulness metrics are often binary and do not address human relevance or usefulness. Balancing strict process-fidelity with simulatability or human usability is a persistent challenge (Lyu et al., 2022, Agarwal et al., 7 Feb 2024).
Standardization and meta-evaluation: There is a need for unified benchmarks and a meta-evaluation of faithfulness metrics, especially for open-domain and multi-hop reasoning tasks in natural language (Lyu et al., 2022, Agarwal et al., 7 Feb 2024).

7. Exemplars and Impactful Architectures

The following table summarizes key faithful self-explanation methodologies and their defining features:

Method	Faithfulness Guarantee	Explanation Type
SEV (BFS)	100% (w.r.t. true $f$ )	Minimal feature set
NMNs/Program	By construction (module chain)	Program/steps
DISCRET	Provable consistency	Rule/query
SR-NLE (IWF)	Empirical via self-consistency	NL explanation
FIRE	SHAP-to-prediction agreement	NL explanation
FaithLM	Causal intervention on output	NL explanation

Faithful self-explanation stands as both a foundational technical challenge and a practical requirement for trustworthy ML systems. The contemporary landscape demonstrates that perfect faithfulness is achievable in some settings but remains elusive, ambiguous, or even vacuous in others. Ongoing research continues to refine the quantification, optimization, and interpretation of faithful self-explanations, guiding progress toward systems that are both genuinely transparent and practically useful (Sun et al., 15 Feb 2024, Lyu et al., 2022, Madsen et al., 15 Jan 2024, Siegel et al., 17 Mar 2025, Wang et al., 28 May 2025, Doi et al., 8 Dec 2025, Azzolin et al., 21 Jun 2024).