Faithful Vision Transformers Overview

• Faithful Vision Transformers are ViT variants designed to maintain explanation and prediction stability under bounded perturbations.
• The term FViT covers diverse approaches, including faithful attention modules, foundation backbones, and Gabor-based focal architectures.
• Robust methods like denoised diffusion smoothing and inherent masking improve causal interpretability, though they may incur high computational costs.

Searching arXiv for papers on Faithful Vision Transformers and closely related faithful ViT interpretation methods. Faithful Vision Transformers (FViTs) is an umbrella label with multiple distinct meanings in the literature rather than a single standardized model family. In one line of work, “FViT” explicitly denotes Faithful Vision Transformers, defined as ViTs whose explanations and predictions remain stable under bounded perturbations (Hu et al., 2023). In another, the same acronym denotes foundation vision transformers, referring to large pretrained ViT backbones adapted to downstream tasks (Yu et al., 2023), or Focal Vision Transformers, a Gabor-based pyramid backbone unrelated to explanation faithfulness (Shi et al., 2024). A broader and increasingly influential usage concerns methods for faithful mechanistic or post-hoc interpretation of standard Vision Transformers, where faithfulness means preserving, or closely approximating, the causal and computational structure actually used by the original model (Kim et al., 22 Sep 2025). Across these strands, the common theme is a demand that explanation, feature reconstruction, concept attribution, or dense prediction remain tightly aligned with what a ViT actually computes, rather than merely producing plausible visualizations (Wu et al., 2024).

1. Terminological scope and competing meanings

The term “Faithful Vision Transformers” is most precisely associated with the formulation introduced in “Improving Interpretation Faithfulness for Vision Transformers” (Hu et al., 2023). There, an FViT is a ViT equipped with a faithful attention module whose explanation is stable in the sense of preserving top-$k$ attended indices under perturbation, and whose prediction distribution is also robust under the same perturbation. The paper formalizes this with two conditions: Top-$k$ Robustness and Prediction Robustness. For a function $f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$, the top-$k$ set is

$$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$

the top-$k$ overlap ratio is

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$

and an $(R, D, \gamma, \beta, k, \|\cdot\|)$-faithful attention module must satisfy, for all $\|x-x'\|\le R$,

$V_k(f(x'), f(x))\geq \beta$

and

$k$0

(Hu et al., 2023).

The same acronym, however, is used differently elsewhere. In “Hint-Aug: Drawing Hints from Foundation Vision Transformers Towards Boosted Few-Shot Parameter-Efficient Tuning,” FViTs means foundation vision transformers: large pretrained vision transformer backbones such as ViT, DeiT, Swin, and CvT, used in a pretraining-then-tuning regime (Yu et al., 2023). That paper links “faithfulness” only indirectly to preservation of pretrained representations under frozen-backbone parameter-efficient tuning. By contrast, “FViT: A Focal Vision Transformer with Gabor Filter” uses FViT to mean Focal Vision Transformer, not faithful or foundation (Shi et al., 2024).

A further distinction is necessary for recent interpretability work. “Interpreting vision transformers via residual replacement model” does not introduce a new architecture literally called “Faithful Vision Transformer” or “FViT.” Instead, it proposes an interpretability framework for standard ViTs that is explicitly designed to make their explanations more faithful (Kim et al., 22 Sep 2025). Likewise, “Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers” focuses on faithful mechanistic interpretability via edge-based circuits, not on a new model family (Żukowska et al., 15 Apr 2026). This suggests that “Faithful Vision Transformers” is best treated as a cross-cutting research theme spanning robust attention explanations, faithful post-hoc attribution, mechanistic circuit discovery, inherently faithful masking architectures, and faithful feature reconstruction.

2. Formal definitions of faithfulness in ViTs

The most explicit formal definition is the perturbation-stability definition of FViTs in (Hu et al., 2023). In that framework, faithfulness is not equated with visually plausible attention maps. Instead, explanation faithfulness requires that the top-$k$1 attention indices remain mostly unchanged under perturbation, while prediction faithfulness requires that the output distribution remain close as measured by a divergence $k$2, with the paper focusing on Rényi divergence:

$k$3

The paper also states a prediction-certification theorem: if a function is an $k$4-faithful attention module and

$k$5

then the predicted class is invariant within radius $k$6 (Hu et al., 2023).

A different formalization appears in post-hoc explanation work. “Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer” argues that faithful ViT attribution cannot be obtained from attention weights alone, because transformer blocks both route information and transform token content (Wu et al., 2024). The paper rewrites MHSA as

$k$7

with transformed tokens $k$8, and FFN as

$k$9

Faithfulness then requires attribution to account for both attention and token transformation effects. TokenTM quantifies token transformation through relative length changes

$f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$0

and directional consistency using cosine similarity and the positive normalized weighting

$f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$1

The resulting transformation weight matrix is

$f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$2

(Wu et al., 2024).

Mechanistic-interpretability work uses a further notion of faithfulness. In (Kim et al., 22 Sep 2025), faithfulness means that an explanation should preserve, or at least closely approximate, the causal and computational structure used by the original model. The extracted circuit is evaluated by

$f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$3

with completeness defined as $f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$4, reported as $f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$5, and causality measured by earlier-layer node ablation and downstream activation reduction (Kim et al., 22 Sep 2025). “Seeing Through Circuits” gives a related sufficiency-style definition: a circuit $f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$6 is faithful if

$f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$7

where $f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$8 is a task fidelity metric (Żukowska et al., 15 Apr 2026).

A concept-based formulation appears in medical imaging. “Stable Vision Concept Transformers for Medical Diagnosis” defines a stable concept module $f: \mathbb{R}^{q \times n}\mapsto \mathbb{R}^n$9 by top-$k$0 concept overlap stability

$k$1

and prediction robustness

$k$2

for all $k$3 (Hu et al., 5 Jun 2025). This suggests that faithful ViTs can also be defined at the level of concept bottlenecks rather than attention maps or token attributions.

3. Robust-attention FViTs and Denoised Diffusion Smoothing

The most direct architecture-level proposal for FViTs is Denoised Diffusion Smoothing (DDS) (Hu et al., 2023). DDS combines randomized smoothing and diffusion denoising so that the model sees denoised noisy inputs rather than raw perturbed inputs. The smoothed attention module is defined as

$k$4

where $k$5 is the original self-attention module, $k$6, and $k$7 is the diffusion denoising operator (Hu et al., 2023).

The paper aligns Gaussian smoothing with the diffusion forward process via

$k$8

and

$k$9

using

$$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$0

At timestep $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$1, DDS computes

$$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$2

denoises to obtain $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$3, and computes self-attention on $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$4 (Hu et al., 2023). The resulting theory states that if $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$5 is sufficiently large relative to $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$6, $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$7, and the top-$$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$8 overlap criterion, then $$T_k(x)=\{i: i\in [d] \text{ and } \{|\{x_j\geq x_i: j\in [n]\}|\leq k\} \},$$9 is an $k$0-faithful attention module. The paper also gives an $k$1 counterpart with an extra factor $k$2 (Hu et al., 2023).

Empirically, DDS improves robustness and explanation quality across ViT, DeiT, and Swin. Under the default attack, the ViT backbone with DDS achieves ImageNet classification accuracy 0.85, pixel accuracy 0.76, mIoU 0.65, and mAP 0.93; DeiT reaches ImageNet classification accuracy 0.86 and mIoU 0.66; Swin reaches ImageNet classification accuracy 0.87 and mIoU 0.67 (Hu et al., 2023). The ablation study shows that both smoothing and denoising matter: the full method yields classification robust accuracy 99.5, segmentation robust accuracy 97.8, mIoU 0.985, positive test 15.29, and negative test 63.23, while removing both degrades to 92.1, 90.5, 0.947, 38.13, and 48.58, respectively (Hu et al., 2023).

A reproduction study broadly supports these findings while introducing important caveats (Kurek et al., 18 Sep 2025). It confirms that DDS generally improves robustness of explanation under attack, but also notes that the gains are not universally large, are not always consistent across methods and models, and come at very large computational and environmental cost. In segmentation on ImageNet-Segmentation, Transformer Attribution improves from pixel accuracy 0.73, mIoU 0.52, mAP 0.82 to 0.77, 0.58, 0.85 with DDS, while Attribution Rollout with DDS reaches 0.78, 0.60, 0.86 on ViT and 0.79, 0.61, 0.85 on DeiT (Kurek et al., 18 Sep 2025). The reproduction further argues that the FViT definition “seems to emphasize stability alone with its priority on robustness criteria,” suggesting that the framework may be closer to a robust ViT than to a fully general notion of faithful interpretability (Kurek et al., 18 Sep 2025).

4. Faithful post-hoc and mechanistic interpretation

A major line of research treats faithful ViTs as a problem of explanation methodology rather than architecture design. The central critique is that raw attention maps, attention rollout, or qualitative feature visualizations often indicate where a model looks but not what internal representations it uses or how those are composed over depth (Kim et al., 22 Sep 2025).

TokenTM is the clearest formalization of this critique at the post-hoc level. Its update map for MHSA is

$k$3

with

$k$4

and for FFN,

$k$5

Token contributions are propagated from an initialization

$k$6

through

$k$7

This yields a transformation-aware rollout over all layers (Wu et al., 2024). On ImageNet-Segmentation with DeiT, TokenTM achieves 81.79 / 86.45 / 64.22 for pixel-wise accuracy, mAP, and mIoU, outperforming Transformer Attribution (79.17 / 85.81 / 61.02) and GAE (79.05 / 85.71 / 61.56) (Wu et al., 2024). Perturbation tests on CIFAR-10, CIFAR-100, and ImageNet consistently favor TokenTM over GAE, with lower positive-perturbation AUC and higher negative-perturbation AUC (Wu et al., 2024). This supports the narrower claim that faithful post-hoc explanation for ViTs should incorporate token transformation, not attention alone.

Mechanistic-interpretability work pushes the faithfulness requirement further by extracting sparse computational graphs. In (Kim et al., 22 Sep 2025), the residual replacement model begins from the observation that the residual stream is the main communication channel of the transformer. A TopK sparse autoencoder maps residual activations $k$8 to sparse feature activations $k$9:

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$0

with reconstruction

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$1

and reconstruction error

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$2

SAEs are trained layerwise on residual-stream activations from supervised ViT-B/16, CLIP ViT-B/16, and DINOv2 ViT-B/14, with the objective

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$3

where

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$4

The residual replacement model then constructs a directed acyclic graph over SAE features and error terms, averaging feature activations across tokens for tractability (Kim et al., 22 Sep 2025).

Edge importance is estimated using attribution patching:

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$5

with a token-aggregated form accelerated by a Jacobian-vector-product trick that yields roughly a 200$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$6 speed-up (Kim et al., 22 Sep 2025). The method reports AUC faithfulness 94.1% on ViT, 85.1% on DINOv2, and 82.3% on CLIP for the best feature-circuit method, compared with 64.9%, 58.9%, and 50.7% for naive feature circuits and only 61.4%, 42.8%, and 32.4% for the best neuron-circuit variants (Kim et al., 22 Sep 2025). Completeness is 99.6%, 99.8%, and 99.7%, while causality is lower at 54.5%, 54.8%, and 53.8% (Kim et al., 22 Sep 2025). This supports the specific claim that sparse feature nodes are better units than raw neurons and that edge-based discovery materially improves circuit fidelity, while also indicating that output preservation and recovery of causal pathways are not identical achievements.

“Seeing Through Circuits” generalizes this mechanistic perspective to edge-based computational graphs over attention heads, MLP blocks, and attention-input abstractions (Żukowska et al., 15 Apr 2026). Vi-CD builds clean/corrupted image pairs via segmentation and inpainting, uses activation patching over residual-stream edges, and greedily prunes edges using target logit difference. On ViT-B, it recovers near-perfect class accuracy with fewer than 10% of edges, and compared to EAP-IG, it finds circuits approximately $V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$7 sparser at comparable accuracy (Żukowska et al., 15 Apr 2026). The same paper uses these circuits for typographic-attack steering in CLIP: for big text, clean Top-1 accuracy changes from 57.0% to 55.8%, corrupted Top-1 improves from 34.7% to 50.0%, and attack success rate Top-1 drops from 39.1% to 2.8% (Żukowska et al., 15 Apr 2026). This suggests that faithful ViT interpretation increasingly means recoverable, sparse, behavior-preserving subgraphs that support intervention, not only visualization.

5. Architectures that impose faithfulness by construction

A distinct response to the faithfulness problem is to redesign the computational graph so that the explanation mechanism is itself causal. The clearest example is iFAM, introduced in “Inherently Faithful Attention Maps for Vision Transformers” (Aniraj et al., 10 Jun 2025). Its central principle is that attention maps are only genuinely faithful if only attended image regions can influence the prediction. Rather than relying on soft, late feature masking, iFAM uses a two-stage framework. Stage 1 processes the full image to discover object parts and identify task-relevant regions; stage 2 is a ViT-based classifier explicitly prevented from seeing anything else (Aniraj et al., 10 Jun 2025).

The paper contrasts standard late masking,

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$8

with early masking at the input of the predictor,

$V_k(x, x')=\frac{1}{k} | T_k(x) \cap T_k(x')|,$9

For a ViT predictor, masking is implemented directly in self-attention:

$(R, D, \gamma, \beta, k, \|\cdot\|)$0

where

$(R, D, \gamma, \beta, k, \|\cdot\|)$1

Thus a token masked out by stage 1 can neither send nor receive attention (Aniraj et al., 10 Jun 2025). The paper explicitly argues that only a truly discrete attribution map can provide faithfulness guarantees by fully preventing information leakage.

This architectural notion of faithfulness is supported by out-of-distribution and spurious-correlation benchmarks. On MetaShift, iFAM improves worst-group accuracy from 81.0% or 75.5% for PDiscoFormer variants to 88.6% at $(R, D, \gamma, \beta, k, \|\cdot\|)$2 while maintaining average accuracy 88.7% (Aniraj et al., 10 Jun 2025). On Waterbirds, iFAM reaches 97.0% worst-group accuracy at $(R, D, \gamma, \beta, k, \|\cdot\|)$3; on Waterbird200, it reaches 86.2% OOD accuracy compared with roughly 76% for late-masking PDiscoFormer; on SIIM-ACR, worst-group AUC improves from 46.7 to 65.9; and on ImageNet-9, iFAM achieves BG-GAP = 2.4, better than 5.3 for PDiscoFormer (Aniraj et al., 10 Jun 2025). The soft-mask ablation is especially revealing: soft masks slightly improve in-distribution CUB accuracy but reduce OOD robustness relative to hard masks, supporting the claim that soft relevance weights leak nuisance information (Aniraj et al., 10 Jun 2025).

Another post-hoc but structurally stronger method is Vision DiffMask (Nalmpantis et al., 2023). It learns a minimal subset of patches whose retention preserves the model’s full output distribution. Given gate activations

$(R, D, \gamma, \beta, k, \|\cdot\|)$4

training-time masks are sampled using Hard Concrete,

$(R, D, \gamma, \beta, k, \|\cdot\|)$5

aggregated across layers by

$(R, D, \gamma, \beta, k, \|\cdot\|)$6

and applied to the input with a learned baseline:

$(R, D, \gamma, \beta, k, \|\cdot\|)$7

The optimization objective is

$(R, D, \gamma, \beta, k, \|\cdot\|)$8

relaxed with a Lagrangian (Nalmpantis et al., 2023). This operationalizes faithfulness as prediction-preserving patch erasure rather than visual plausibility alone. The paper reports that Vision DiffMask is especially strong on negative perturbation, indicating that low-attribution patches can actually be removed with little effect on the model’s prediction (Nalmpantis et al., 2023).

6. Faithful feature spaces, concepts, and representations

Not all work on faithful ViTs focuses on explanation maps. Some define faithfulness as preservation of the semantic geometry of the backbone feature space. ViT-Up is the clearest case (Wandel et al., 12 Jun 2026). It is proposed as a faithful feature upsampling method: rather than sharpening coarse ViT features using an external image encoder, it predicts dense features from the ViT’s own hidden states so that they remain aligned with the original backbone feature space. The target is to upsample

$(R, D, \gamma, \beta, k, \|\cdot\|)$9

to

$\|x-x'\|\le R$0

while preserving semantic geometry (Wandel et al., 12 Jun 2026).

ViT-Up performs coordinate-conditioned implicit feature prediction. For continuous image coordinate $\|x-x'\|\le R$1, it constructs an initial query embedding $\|x-x'\|\le R$2 from the ViT’s own patch embedding at higher internal resolution, then refines it through blocks

$\|x-x'\|\le R$3

using $\|x-x'\|\le R$4 and $\|x-x'\|\le R$5 in the main setup (Wandel et al., 12 Jun 2026). Each block combines a transition MLP, cross-window multi-head attention over intermediate hidden states, and a local sub-token extractor FeatX based on the nearest patch token:

$\|x-x'\|\le R$6

encoded and used to FiLM-modulate the nearest token feature (Wandel et al., 12 Jun 2026). Training uses multi-scale teacher supervision over $\|x-x'\|\le R$7, with losses

$\|x-x'\|\le R$8

and

$\|x-x'\|\le R$9

with all $V_k(f(x'), f(x))\geq \beta$0 equal to 1 (Wandel et al., 12 Jun 2026).

The empirical case for feature-space faithfulness is strongest on dense prediction and correspondence. On DINOv3-S+, ViT-Up achieves 64.09 mIoU on COCO segmentation, 87.47 on VOC, 44.73 on ADE20K, 65.41 on Cityscapes, and 62.72 / 59.82 for $V_k(f(x'), f(x))\geq \beta$1 / RMSE on COCO depth, with gains up to +2.07 mIoU on Cityscapes (Wandel et al., 12 Jun 2026). On SPair-71k correspondence, it reaches 55.44 / 39.07 / 7.30 at [email protected] / 0.05 / 0.01, outperforming prior upsamplers by +4.17, +5.11, and +3.47 (Wandel et al., 12 Jun 2026). On DINOv3-B, gains increase to +3.36 mIoU on Cityscapes and +8.09 [email protected] on SPair-71k (Wandel et al., 12 Jun 2026). This supports a broader use of “faithful ViT” in which the goal is not faithful explanation but faithful dense reconstruction of pretrained backbone semantics.

Concept-based medical architectures provide a further variant. In SVCT, a ViT backbone produces $V_k(f(x'), f(x))\geq \beta$2, a concept projection

$V_k(f(x'), f(x))\geq \beta$3

is learned, and the final representation is fused as

$V_k(f(x'), f(x))\geq \beta$4

The concept layer is trained by aligning concept-neuron activations to CLIP concept activations:

$V_k(f(x'), f(x))\geq \beta$5

and stability is then imposed with DDS on token embeddings (Hu et al., 5 Jun 2025). Under perturbation radius $V_k(f(x'), f(x))\geq \beta$6, SVCT improves over VCT and label-free CBMs on both accuracy and concept stability. On HAM10000, accuracy rises from 95.28% for VCT to 97.24% for SVCT, while the Concept Faithfulness Score drops from 0.4637 to 0.1725 and Concept Perturbation Cosine Similarity rises from 0.8844 to 0.9836 (Hu et al., 5 Jun 2025). This suggests that faithful ViTs can also be framed as perturbation-stable concept-bottleneck ViTs, though the model’s final decision remains partly mediated by raw ViT features rather than concepts alone.

7. Broader context, theoretical grounding, and persistent limitations

A theoretical precursor to faithful-ViT thinking is “Vision Transformers provably learn spatial structure” (Jelassi et al., 2022). That work does not study explanation faithfulness directly, but it proves that in a stylized one-layer, one-head ViT with positional attention, gradient descent can learn patch association, meaning that the learned positional similarities recover the latent patch groups underlying the data-generating function. The key formal definition is:

$V_k(f(x'), f(x))\geq \beta$7

for every $V_k(f(x'), f(x))\geq \beta$8 (Jelassi et al., 2022). The paper explicitly proves that there exist generalizing solutions without patch association, so accuracy alone does not imply structurally faithful internal organization. But under the specified optimization dynamics, gradient descent selects a structure-aligned solution and enables sample-efficient transfer with frozen attention (Jelassi et al., 2022). This suggests that faithful internal structure in ViTs may arise as an optimization-selected inductive bias rather than as a post hoc artifact.

A separate but related usage appears in few-shot adaptation of foundation vision transformers (Yu et al., 2023). There, the frozen-backbone property of parameter-efficient tuning is treated as preserving pretrained representations, enabling Hint-Aug to compare tuned and pretrained attention-score maps and to use pretrained features as augmentation hints. The input image is tokenized into patches

$V_k(f(x'), f(x))\geq \beta$9

with self-attention

$k$00

An Attentive Over-fitting Detector compares pretrained and tuned attention-score maps using

$k$01

and selects the most changed patch

$k$02

Confusion-based Feature Infusion then perturbs that patch toward features of confusable classes using a target distribution derived from a confusion matrix (Yu et al., 2023). This suggests a looser notion of faithfulness as preservation of pretrained semantic structure during adaptation, rather than faithful explanation.

Several persistent limitations recur across the faithful-ViT literature. First, stability is not equivalent to causal correctness. The reproduction of DDS-based FViTs explicitly notes that the formal definition is heavily robustness-centered (Kurek et al., 18 Sep 2025). Second, output preservation does not guarantee recovery of the true internal mechanism. The residual replacement model reports mid-50s causality despite 80–94% faithfulness and near-100% completeness (Kim et al., 22 Sep 2025). Third, many methods trade granularity for tractability: token aggregation in residual replacement models omits token-specific interactions, while graph simplifications in Vi-CD abstract away position-level edges (Kim et al., 22 Sep 2025, Żukowska et al., 15 Apr 2026). Fourth, architectural faithfulness often incurs significant cost: iFAM requires two forward passes (Aniraj et al., 10 Jun 2025), DDS incurs roughly tenfold runtime increases in reproduction (Kurek et al., 18 Sep 2025), and Vision DiffMask requires training an auxiliary interpretation network (Nalmpantis et al., 2023). Fifth, acronym ambiguity remains substantial. “FViT” can mean faithful, foundation, or focal, and only some of these usages concern explanation faithfulness at all (Hu et al., 2023, Yu et al., 2023, Shi et al., 2024).

Taken together, these strands suggest that Faithful Vision Transformers are best understood not as a single architecture but as a research program. In the narrowest sense, an FViT is a ViT whose attention explanations and predictions are certifiably stable under perturbation (Hu et al., 2023). In a broader and increasingly influential sense, faithful ViTs are standard or modified Vision Transformers for which explanations, concepts, circuits, or dense features are forced to remain aligned with the model’s actual computation, feature space, or causal support (Wu et al., 2024, Kim et al., 22 Sep 2025, Żukowska et al., 15 Apr 2026, Aniraj et al., 10 Jun 2025, Wandel et al., 12 Jun 2026). A plausible implication is that future progress will continue to move away from visually appealing but weakly grounded attention maps and toward interventions, reconstruction objectives, discrete masking, edge-based circuits, and representation-space preservation as the main criteria of faithfulness.