Functional Faithfulness in Computational Models

• Functional faithfulness is defined as the measure by which an explanation or subnetwork accurately reflects the actual computational process or reasoning pathway of a model.
• It employs criteria like necessity and sufficiency, using perturbation methods and salience metrics to validate the true causal components involved in model predictions.
• This concept bridges AI interpretability with rigorous mathematical formulations, enabling robust diagnostics in GNNs, vision models, and categorical frameworks.

Functional faithfulness denotes the extent to which an explanation, subnetwork, or functor accurately captures and conveys the actual computational function or reasoning pathway used by a system (e.g., a neural network, a LLM, a GNN, or a functor between categories). Across interpretability and mathematical contexts, functionally faithful explanations and mappings are those that preserve or recapitulate the genuine mechanisms, substructures, or morphisms that are causally or logically responsible for the output or behavior of interest.

1. Core Definitions and Principles

In model interpretability, functional faithfulness evaluates whether the structures or attributions surfaced by an explanation (e.g., a salience map, a pruned circuit, or a detected subgraph) are not merely correlated with output, but are precisely those required for the model to produce that output. The guiding desiderata can be formalized as follows:

• Necessity: Only components present in the explanation are genuinely utilized by the model’s computation.
• Sufficiency: All and only the components in the explanation are enough, such that alterations outside the explanation leave the prediction unchanged, but alterations inside do alter it.

More precisely, in GNNs, sufficiency and necessity are expressed by comparing the model's output given perturbations inside or outside the candidate explanation subgraph $R_A$, aggregated via chosen divergences and intervention distributions (Azzolin et al., 2024). In circuit discovery for LLMs, faithfulness is measured by whether a discovered subcircuit can, under hard knockout (weight/edge mask), reproduce the full model's task performance (Yu et al., 2024).

In categorical homological contexts, a functor such as the family Floer functor is faithful if it induces injective maps on morphism spaces, thus preserving structural information from source to target category (Abouzaid, 2014).

2. Mathematical Formalization Across Domains

Graph Neural Networks (GNNs)

Faithfulness is parameterized by a pair of notions—sufficiency and necessity—using general divergence metrics and perturbation distributions:

$$\mathrm{Suff}_{d,p_R}(R_A) = \mathbb{E}_{G'_A\sim p_R} d\!\left(p_\theta(\cdot| G_A),\,p_\theta(\cdot|G'_A)\right),\qquad \mathrm{Nec}_{d,p_C}(R_A) = \mathbb{E}_{G'_A\sim p_C} d\!\left(p_\theta(\cdot| G_A),\,p_\theta(\cdot|G'_A)\right)$$

where $p_R$ samples interventions on the complement of $R_A$ and $p_C$ samples interventions within $R_A$. Functional faithfulness uses the harmonic mean of normalized sufficiency and necessity (Azzolin et al., 2024).

For injective, aggregation-based GNNs, the only perfectly faithful subgraph is the entire computation graph (full $L$-hop neighborhood); only modular architectures decouple faithfulness from triviality.

Vision Models

In vision transformers, faithfulness formally requires ordering consistency (if group $G_i$ has higher total salience, its removal should drop confidence more) and salience difference sensitivity (violations are punished proportionally to the salience gap). The Salience-guided Faithfulness Coefficient (SaCo) expresses this via weighted pairwise comparisons over pixel groups, normalized to $[-1,1]$ (Wu et al., 2024):

$$F = \frac{\sum_{i<j}\text{sign}(\nabla \text{pred}(G_i) - \nabla \text{pred}(G_j))(s(G_i)-s(G_j))}{\sum_{i<j} |s(G_i)-s(G_j)|}$$

where $s(G_i)$ is total salience of group $G_i$, and $\nabla \text{pred}(G_i)$ is the model’s output drop upon removing $G_i$.

Circuit Discovery in Neural LLMs

Functional faithfulness is defined by the ability of a pruned subnetwork (mask $\mathbf{m}$) to match the full model’s output distribution:

$$\mathcal{L}_{\text{faith}} = -\sum_{i} \log p_{c_{\mathbf{m}}} \bigl( \hat y^{(i)} \,\bigm|\, x^{(i)} \bigr )$$

with functional faithfulness signified by high accuracy of the pruned circuit, evaluated on the full model’s own predictions or the true task (Yu et al., 2024). Completeness and sparsity are used as auxiliary requirements.

Homological Mirror Symmetry

In symplectic geometry, faithfulness of the family Floer functor means injectivity on morphisms: $$\operatorname{Hom}_{\mathrm{Fuk}(X)}(L, L') \to \operatorname{Hom}_{D^{\mathrm{perf}}(Y)}(\mathcal{F}(L), \mathcal{F}(L'))$$ is injective for all $L, L'$ (Abouzaid, 2014).

3. Evaluation Metrics and Methodologies

Existing Metrics

• AUC (Area Under Curve): Measures change in model output as salient regions are sequentially masked.
• AOPC (Average Output Probability Change): Probability drop averaged over progressively larger masks.
• Comprehensiveness/Log-Odds: Compare output confidence with vs. without content highlighted by explanation.

Advances

• SaCo (Salience-guided Faithfulness Coefficient): Overcomes two key limitations—cumulative perturbation and neglect of magnitude—by evaluating isolated groups and weighting by salience difference, allowing proper discrimination between random and structured explanations (Wu et al., 2024).
• Ensemble Perturbation (FEI): Replaces hard fractile masking with smooth, optimizable masks over multiple area fractions. Incorporates hidden-layer faithfulness via clipped gradients, ensuring that perturbations do not activate spurious features deep in the model (Zhang et al., 4 Sep 2025).

Circuit Discovery

• DiscoGP: Evaluates functional faithfulness via accuracy on pruned circuits, augmented by completeness (complement circuit performs randomly) and sparsity (Yu et al., 2024).
• Direct Pruning vs. Activation Patching: Only hard knockout (pruning) suffices for evaluating functional faithfulness, as patching retains shortcut flows and fails both faithfulness and completeness tests.

4. Empirical Findings and Trade-Offs

Model Interpretability

• Vision Transformers: Only gradient-weighted, cross-layer aggregated attention maps approach high faithfulness (SaCo ≥ 0.45), while simpler or random attributions perform poorly (SaCo ≈ 0) (Wu et al., 2024).
• Circuit Extraction: DiscoGP circuits attain nearly full-model performance at edge densities of <3%, with the complement failing, demonstrating both functional faithfulness and parsimony; prior methods based on patching yield circuits that cannot function in isolation (Yu et al., 2024).
• GNNs: For high-expressivity GNNs, perfect faithfulness implies triviality; modular architectures with properly designed subgraph selectors yield informative and functionally faithful explanations (Azzolin et al., 2024).

Text Summarization

Decoding strategy strongly affects faithfulness: Large-beam search and faithfulness ranking or lookahead consistently yield more factually faithful summaries (DAE errors 3–20 percentage points lower) than greedy or sampling, with minor impact on ROUGE-L. Distilled models can inherit these gains with faster inference (Wan et al., 2023).

Quantitative Results Example

Model / Explanation	Faithfulness Metric (SaCo / Accuracy)	Sparsity (Edge %)
ViT + Transformer-MM	SaCo ≈ 0.456	—
Random Attribution (ViT)	SaCo ≈ 0	—
GPT-2 (DiscoGP, IOI, Joint)	Accuracy 100% (pruned)	Edge 2.03%
GPT-2 (ACDC patching, IOI)	Accuracy 51.6%	—

5. Architectural and Algorithmic Determinants

Cross-Domain Insights

• GNNs: Strict injectivity leads to trivial explanations; modularity and stability in the detector-classifier split enable non-trivial, faithful explanations (Azzolin et al., 2024).
• ViTs: Gradients inject class specificity; cross-layer aggregation captures distributed computation; their combination maximizes faithfulness (Wu et al., 2024).
• Ensemble-interpretation (FEI): Jointly optimized soft masks across fractiles (area constraints) raise deletion/preservation AUC; internal faithfulness is improved by enforcing gradient clipping at hidden layers, avoiding “adversarial” attributions (Zhang et al., 4 Sep 2025).
• DiscoGP: Unified differentiable masking across weights and edges, straight-through binarization, and post-training graph-reachability yield circuits that are provably both functionally faithful and sparse (Yu et al., 2024).

6. Practical Implications, Limitations, and Open Directions

Robustness and Generalization

• OOD Generalization: In GNNs, functionally faithful, invariant subgraphs directly bound the difference in OOD and ID performance; empirical correlation between faithfulness + invariance and reduced generalization gap is strong (Pearson −0.83) (Azzolin et al., 2024).

Limitations and Recommendations

• Alignment with human intuitions requires further study combining faithfulness metrics with human-centered evaluations (e.g., OpenXAI).
• Existing metrics can be insensitive to random attributions or heavily dependent on hyperparameters; SaCo and FEI address some but not all issues.
• Partition granularity and aggregation strategies (e.g., cross-head, nonlinear) in transformer explanations remain under-explored.

7. Functional Faithfulness in Mathematical Contexts

Beyond model interpretability, functional faithfulness arises in homological mirror symmetry. The family Floer functor, mapping the Fukaya category of a symplectic manifold to perfect complexes over its mirror, is proven to be faithful via degeneration arguments involving annuli. This theorem ensures injectivity on morphisms, preserving the computational and algebraic “function” between categories (Abouzaid, 2014).

The proof constructs universal moduli spaces interpolating between identity maps (discs/caps) and continuation maps (degenerate annuli), with the Cardy relation yielding a chain-homotopy between the composite and the identity, thus demonstrating faithfulness.

---

The unifying theme is that functional faithfulness, rigorously operationalized in diverse settings, measures to what extent an explanation, substructure, or mapping preserves, exposes, or matches the core computational pathway or logical relation responsible for the object or outcome of interest. Advances in evaluation (e.g., SaCo, FEI, pruning-based indices) and architecture (modularity, aggregation, differentiable masking) furnish practical paths toward more faithful, non-trivial explanations, robust subnetwork extraction, and structurally informative functors. Further research is required to synchronize such mathematical and mechanistic faithfulness with human-aligned, semantically meaningful explanation.