TEXTER: Zero-Shot Textual Explanations for Classifiers

• TEXTER is a post-hoc, zero-shot method that produces natural-language explanations by isolating decision-critical features from image classifiers.
• The approach leverages Integrated Gradients and MACO-based concept image generation to focus on the most influential neurons driving a specific prediction.
• For Transformer architectures, a TopK Sparse Autoencoder is employed to enhance explanation fidelity and counter global feature bias in feature representations.

TEXTER, short for Textual EXplanations via Translating dEcision-cRitical features, is a post-hoc, zero-shot method for producing short natural-language explanations of image-classifier predictions from the classifier’s internal evidence rather than from generic image content (Yamauchi et al., 8 Dec 2025). It is designed for the setting in which a pretrained classifier $\mathcal{F}=g\circ f:\mathcal{X}\to\mathcal{Y}$ has no language component, no explanation supervision is available, and the classifier itself must remain unchanged. TEXTER addresses a specific failure mode of prior zero-shot textual explanation methods: when global image features are aligned directly to language, retrieved phrases often describe what is visible in the image rather than what actually drove the classifier’s prediction. Its central claim is that textual explanations become more faithful when decision-critical neurons are isolated before any language alignment is attempted (Yamauchi et al., 8 Dec 2025).

1. Problem formulation and explanatory objective

TEXTER operates on a pretrained image classifier $\mathcal{F}=g\circ f$, where $f:\mathcal{X}\to Z_f$ is a vision encoder and $g:Z_f\to\mathbb{R}^C$ is a classification head over $C$ classes (Yamauchi et al., 8 Dec 2025). Given an image $x\in\mathcal{X}$ and a predicted class

$c=\arg\max_k[g(f(x))]_k,$

the objective is to return natural-language phrases that describe the visual evidence the classifier uses for that specific prediction, rather than a generic caption of the image.

This objective places TEXTER within zero-shot textual explainability rather than captioning or multimodal generation. Large vision-LLMs such as BLIP, LLaVA, and GPT-4V can produce rich descriptions, but they are optimized for general visual understanding rather than for exposing the reasoning of a fixed image classifier (Yamauchi et al., 8 Dec 2025). Likewise, methods such as Text-To-Concept and ZSNLE align classifier features to language but do so using global image features, which tends to produce what the paper calls a form of “global feature bias”: dominant visual elements, such as “cushions,” may be surfaced even when the classifier’s decision was driven by a more discriminative feature such as “whisker spots” (Yamauchi et al., 8 Dec 2025).

A common misconception is that any semantically plausible text about an image constitutes an explanation of a classifier. TEXTER explicitly rejects that equivalence. In its formulation, an explanation is intended to name features that are decision-critical for a particular prediction, not merely features that are present or easy for a vision-LLM to verbalize (Yamauchi et al., 8 Dec 2025).

2. Core pipeline: isolating and visualizing decision-critical features

TEXTER consists of three stages: concept image generation, vision-language space alignment, and textual explanation retrieval (Yamauchi et al., 8 Dec 2025). The method begins by selecting a representation layer $\ell$ and extracting a feature vector

$z=f^{(\ell)}(x)\in\mathbb{R}^d.$

To quantify which neurons contribute most to the class-$c$ logit, it applies Integrated Gradients in feature space:

$\mathcal{F}=g\circ f$0

with baseline $\mathcal{F}=g\circ f$1 and step count $\mathcal{F}=g\circ f$2 (Yamauchi et al., 8 Dec 2025). The top $\mathcal{F}=g\circ f$3 neurons are then selected, with $\mathcal{F}=g\circ f$4 in the experiments:

$\mathcal{F}=g\circ f$5

These top neurons are treated as encoding the decision-critical concept for class $\mathcal{F}=g\circ f$6 on the given image. TEXTER does not translate these activations directly into text. Instead, it first generates a concept image $\mathcal{F}=g\circ f$7 by feature visualization. The optimization objective is

$\mathcal{F}=g\circ f$8

with

$\mathcal{F}=g\circ f$9

The paper uses MACO (Magnitude-Constrained Optimization), which fixes the magnitude of the Fourier spectrum and optimizes only the phase, an empirical design choice intended to keep images natural-looking and to avoid adversarial noise patterns (Yamauchi et al., 8 Dec 2025).

The resulting concept image is not treated merely as a visualization artifact. TEXTER defines the classifier feature of this concept image,

$f:\mathcal{X}\to Z_f$0

as the decision-critical feature to be translated into language (Yamauchi et al., 8 Dec 2025). This design is central to the method: it replaces the original global image representation with one synthesized explicitly from the neurons most responsible for the predicted class.

This suggests a shift in explanatory granularity. Rather than asking which words best match the original image, TEXTER asks which words best match a representation constructed to excite the model’s most important units for the target prediction.

3. Alignment to CLIP space and language retrieval

Because the classifier feature space $f:\mathcal{X}\to Z_f$1 is not natively aligned with language, TEXTER learns an affine aligner

$f:\mathcal{X}\to Z_f$2

where $f:\mathcal{X}\to Z_f$3 and $f:\mathcal{X}\to Z_f$4 are trainable (Yamauchi et al., 8 Dec 2025). Training uses a subset of the classifier’s original training data and minimizes an $f:\mathcal{X}\to Z_f$5 regression loss against the frozen CLIP ViT-B/16 image encoder:

$f:\mathcal{X}\to Z_f$6

This stage requires no labels or explanation annotations; it uses only image pairs of the form $f:\mathcal{X}\to Z_f$7 (Yamauchi et al., 8 Dec 2025).

At inference time, TEXTER applies the aligner not to the original image feature but to the concept-image feature:

$f:\mathcal{X}\to Z_f$8

Candidate explanation phrases are drawn from a concept bank $f:\mathcal{X}\to Z_f$9. For each candidate phrase $g:Z_f\to\mathbb{R}^C$0, TEXTER computes the CLIP text embedding $g:Z_f\to\mathbb{R}^C$1 and ranks candidates by cosine similarity:

$g:Z_f\to\mathbb{R}^C$2

The top $g:Z_f\to\mathbb{R}^C$3 phrases are returned, with $g:Z_f\to\mathbb{R}^C$4 in the experiments (Yamauchi et al., 8 Dec 2025).

TEXTER also introduces a patch-based robustness procedure because MACO concept images can contain repetitive patterns. It randomly crops 6 patches from $g:Z_f\to\mathbb{R}^C$5, each with side length 25–30% of the image, maps each patch through $g:Z_f\to\mathbb{R}^C$6, computes cosine similarities for each candidate phrase, and averages the 6 scores for ranking (Yamauchi et al., 8 Dec 2025). This is an explicit attempt to stabilize retrieval against artifacts introduced by the visualization stage.

The explanatory logic is therefore indirect but structured: first isolate influential neurons, then synthesize a visual object that jointly activates them, then align that object to a vision-language space, and only then retrieve language.

4. Sparse autoencoder factorization for Transformer architectures

A major architectural distinction in TEXTER concerns the difference between CNN and Transformer feature spaces. The paper states that CNN feature spaces already have somewhat localized, interpretable units, whereas Transformer representations are more entangled and compositional; applying Integrated Gradients directly on raw Transformer features yields diffuse, non-specific neuron subsets and poor concept images (Yamauchi et al., 8 Dec 2025).

To address this, TEXTER uses a TopK Sparse Autoencoder (TopK SAE), especially for ViT-based models. The encoder is

$g:Z_f\to\mathbb{R}^C$7

where $g:Z_f\to\mathbb{R}^C$8 retains only the $g:Z_f\to\mathbb{R}^C$9 largest-magnitude entries, and the decoder reconstructs the original feature by

$C$0

The training loss is

$C$1

No explicit sparsity penalty is required because TopK is a hard sparsity operator (Yamauchi et al., 8 Dec 2025).

The training details are specific: the SAE embedding is 8 times larger than the original feature dimension, the TopK ratio is 10%, and training is performed for 10 epochs with batch size 1024, learning rate $C$2, and Adam, while the classifier remains frozen (Yamauchi et al., 8 Dec 2025). Once trained, TEXTER applies Integrated Gradients and feature visualization in the sparse latent space rather than directly in the original Transformer feature space.

The empirical effect is pronounced. For Transformers, raw-feature-based concept images are reported as very poor, with top-1 accuracy of the classifier on concept images around $C$3–$C$4; with SAE, top-1 accuracy rises to $C$5–$C$6 (Yamauchi et al., 8 Dec 2025). For CNNs, SAE is optional: concept images from raw features are already strong, and SAE may slightly reduce some quantitative validity scores while largely preserving behavior (Yamauchi et al., 8 Dec 2025).

This difference is significant because it marks the method as not merely architecture-agnostic in practice but architecture-adaptive. A plausible implication is that TEXTER’s interpretability claims depend strongly on whether the intermediate representation can be decomposed into units that admit meaningful axis-aligned manipulation.

5. Concept bank construction and zero-shot explanation generation

TEXTER does not search over arbitrary strings. It constructs a finite concept bank $C$7 from two sources (Yamauchi et al., 8 Dec 2025).

First, an LLM-derived bank is created using GPT-3.5-turbo. For each class $C$8, the model is prompted to generate generic visual features of that class, independent of the specific image. The prompts constrain the outputs to short phrases of 1–3 words, include both object and context features, and avoid repeating the class name. After deduplication, this yields approximately 100 phrases per class (Yamauchi et al., 8 Dec 2025).

Second, a VLM-derived bank is created using Qwen2.5-VL-7B-Instruct. Given the image $C$9 and class name $x\in\mathcal{X}$0, the model is asked for visual features that distinguish the image as an instance of that class, again as short, specific phrases. This yields approximately 30 image-specific phrases (Yamauchi et al., 8 Dec 2025).

The final concept bank therefore has cardinality

$x\in\mathcal{X}$1

For multi-label or class-wise explanations, TEXTER can take the union of concept banks from several classes (Yamauchi et al., 8 Dec 2025).

The method is zero-shot with respect to explanations because it never sees paired $x\in\mathcal{X}$2 data and never uses concept labels or human explanation labels at any stage (Yamauchi et al., 8 Dec 2025). The trainable components are only the sparse autoencoder and the affine aligner; the classifier, CLIP encoders, LLM, VLM, and MACO machinery remain frozen (Yamauchi et al., 8 Dec 2025).

This zero-shot designation should not be confused with training-free operation. TEXTER does train auxiliary modules, but the supervision excludes explanation annotations and does not alter the original classifier. That distinction is essential to the method’s positioning.

6. Evaluation, qualitative behavior, and common points of contention

TEXTER is evaluated first through concept image validity. On 1000 random ImageNet test images, using 200 classes with 5 images each, the paper measures top-1 accuracy, top-5 accuracy, a confidence ratio

$x\in\mathcal{X}$3

and cosine similarity between the original and concept-image logit vectors (Yamauchi et al., 8 Dec 2025). For CNNs without SAE, Acc$x\in\mathcal{X}$4 is reported around $x\in\mathcal{X}$5, Acc$x\in\mathcal{X}$6, and cosine similarity around $x\in\mathcal{X}$7–$x\in\mathcal{X}$8; for Transformers without SAE, Acc$x\in\mathcal{X}$9 is only about $c=\arg\max_k[g(f(x))]_k,$0–$c=\arg\max_k[g(f(x))]_k,$1 and Acc$c=\arg\max_k[g(f(x))]_k,$2, but with SAE, Acc$c=\arg\max_k[g(f(x))]_k,$3 rises to $c=\arg\max_k[g(f(x))]_k,$4–$c=\arg\max_k[g(f(x))]_k,$5, Acc$c=\arg\max_k[g(f(x))]_k,$6 to approximately $c=\arg\max_k[g(f(x))]_k,$7, and cosine similarity to about $c=\arg\max_k[g(f(x))]_k,$8–$c=\arg\max_k[g(f(x))]_k,$9 (Yamauchi et al., 8 Dec 2025).

A second evaluation uses a multi-label setting on PASCAL VOC, where predicted classes are those with sigmoid probability greater than $\ell$0, and the concept bank is the union over predicted classes (Yamauchi et al., 8 Dec 2025). Explanations are assessed by how well they semantically match the concept image rather than the original image, using CLIP-Score, LPIPS with AlexNet- and SqueezeNet-based models, and feature similarity

$\ell$1

where $\ell$2 is an image generated by Stable Diffusion from the explanation prompt (Yamauchi et al., 8 Dec 2025). Across all five tested models, TEXTER achieves higher CLIP-Score, lower LPIPS, and higher feature similarity than both Random and Text-To-Concept. For DINO ResNet-50, for example, CLIP-Score increases from $\ell$3 for Random and $\ell$4 for Text-To-Concept to $\ell$5 for TEXTER, while feature similarity rises from $\ell$6 and $\ell$7 to $\ell$8 (Yamauchi et al., 8 Dec 2025).

On ImageNet-1K, the paper reports a revealing contrast. When explanations are evaluated against concept images, TEXTER is best across metrics for all models; when evaluated against original images, Text-To-Concept performs best (Yamauchi et al., 8 Dec 2025). The paper interprets this not as a contradiction but as evidence that the two methods optimize different explanatory targets: Text-To-Concept is better at describing global image content, while TEXTER is better at describing the model’s decision-critical features.

Qualitative examples support this interpretation. In multi-object scenes, Text-To-Concept can attach a prediction to background or context, such as “sedan shape” for “person” or “light sensor” for “TV monitor,” whereas TEXTER returns phrases such as “hairstyle,” “hair,” and “facial expression” for “person,” and “high-definition display,” “flat screen,” and “tv stand” for “TV monitor” (Yamauchi et al., 8 Dec 2025). TEXTER is also used for class-conditioned analysis of misclassification: for an image with ground truth “water snake” but prediction “stick insect,” the method produces “slender antennae” and “slender body segments” for the predicted class, suggesting that twig-shaped background lines were interpreted as insect limbs, while returning “slithering through vegetation” and “venomous look” for the true class (Yamauchi et al., 8 Dec 2025).

The main point of contention concerns faithfulness. TEXTER’s own evaluation grounds explanations in concept images, not directly in logit sensitivity to text-induced directions. A later method, FaithTrace, explicitly criticizes similarity-based approaches including TEXTER for relying on correlational signals in CLIP space rather than measuring whether a text concept actually changes the class logit (Yamauchi et al., 16 May 2026). FaithTrace replaces concept-image similarity ranking with a directional-derivative influence score in classifier feature space and reports higher directional scores and influence-curve values across models (Yamauchi et al., 16 May 2026). This does not invalidate TEXTER’s contribution; rather, it identifies a methodological tension between two notions of faithfulness: semantic agreement with an estimated decision-critical visualization, and local causal influence on the classifier’s logits.

7. Relation to prior work, limitations, and scope

Relative to attribution methods such as Grad-CAM and Integrated Gradients, TEXTER shifts from spatial heatmaps to textual concepts by applying attribution over neurons or sparse units and then mapping the resulting concept to language (Yamauchi et al., 8 Dec 2025). Relative to concept-based methods such as Network Dissection, TCAV, and concept bottleneck models, it avoids human-defined concept inventories, concept labels, and classifier retraining (Yamauchi et al., 8 Dec 2025). Relative to zero-shot explanation methods such as Text-To-Concept and ZSNLE, its novelty lies in explicitly isolating decision-critical features before alignment and retrieval, rather than aligning global features directly (Yamauchi et al., 8 Dec 2025).

Its scope is nevertheless limited. TEXTER assumes access to intermediate features $\ell$9 and is therefore out of scope for purely black-box models (Yamauchi et al., 8 Dec 2025). For Transformers, an SAE is practically required, which adds training cost and complexity (Yamauchi et al., 8 Dec 2025). The expressiveness of explanations is constrained by CLIP’s semantic space and by the quality and diversity of concepts generated by GPT-3.5 and Qwen2.5-VL; rare or highly domain-specific concepts may not be well captured (Yamauchi et al., 8 Dec 2025). The method can also be misled if Integrated Gradients or concept visualization attribute importance to non-causal features, and alignment quality may deteriorate on out-of-distribution images because the aligner is trained on a subset of data (Yamauchi et al., 8 Dec 2025). The paper further notes that it lacks a direct human-grounded faithfulness metric; its quantitative measures are proxy metrics based on concept images rather than, for example, counterfactual editing (Yamauchi et al., 8 Dec 2025).

These constraints delimit the kind of interpretability TEXTER offers. It is neither a full natural-language rationale generator nor a black-box explanation interface. It is a structured post-hoc method for turning internal classifier evidence into short retrieved phrases, with a particular emphasis on separating decision-critical evidence from global image content. Within that narrower but technically precise scope, it represents a distinct stage in the development of zero-shot textual explanations for image classifiers (Yamauchi et al., 8 Dec 2025).