Semantic Interpretability Paradigm

• Semantic interpretability paradigm is a research approach that defines model interpretability as the alignment of machine representations with human-understandable concepts.
• It formalizes interpretability through inference equivariance and semantic alignment, ensuring that the translation from machine states to human variables reliably mirrors model computations.
• This paradigm is applied across domains like semantic segmentation, word embeddings, and time series, enhancing diagnosis, intervention, and user trust via concept-based transparency.

Searching arXiv for recent and foundational papers on semantic interpretability across deep learning, time series, embeddings, and XAI. First, I’ll look up the main formalization papers and adjacent semantic interpretability work. Semantic interpretability paradigm is a line of research that treats interpretability as a semantic relation between machine representations and human concepts rather than as a purely visual or post-hoc description of model behavior. Across the literature, a model is semantically interpretable when its internal variables, intermediate concepts, features, attributes, or mechanisms are expressible in terms meaningful to a user, and when those semantics remain aligned with the model’s actual inference, prediction process, or temporal dynamics (Barbiero et al., 17 Feb 2025, Felice et al., 2 Feb 2026, Silva et al., 2019).

1. Conceptual scope and historical framing

The paradigm emerged in response to a recurring limitation of black-box AI systems: high predictive performance does not by itself make internal computation understandable in human semantic terms. Several strands of work therefore shifted interpretability away from raw feature inspection and toward concept-bearing representations, semantic bottlenecks, semantic vectors, semantic attributes, or semantically aligned latent states. In this view, interpretability is not exhausted by heat maps, feature attributions, or transparent syntax; it concerns whether model behavior can be understood through concepts that users can name, validate, and sometimes intervene on (Losch et al., 2019, Gu et al., 2019, Guo et al., 2018).

A broader survey perspective characterized semantic interpretability as a human-centered property spanning machine learning, distributional semantics, fuzzy logic, topic models, and other AI systems, and distinguished interpretability introduced inside the model from interpretability added by an extra-model layer (Silva et al., 2019). A later cognitive-science-inspired taxonomy sharpened this distinction by separating semantic interpretation, which asks what latent properties are learned and represented, from algorithmic interpretation, which asks what operations are performed over those representations (Davies et al., 2024). This suggests that the semantic interpretability paradigm is concerned primarily with representational content and its relation to human concepts, even when the underlying model remains highly expressive.

Within this family of approaches, semantic interpretability is typically contrasted with mechanistic transparency alone. The central claim is that a model is not interpretable merely because one can inspect hidden states, attention weights, saliency maps, or surrogate rules. Rather, interpretability requires that the model’s variables and mechanisms be stated in a semantic reference system that an end user can understand, assess, and use for prediction or intervention (Felice et al., 2 Feb 2026).

2. Formal criteria: inference equivariance and semantic alignment

A formal centerpiece of the paradigm is the framework of semantic interpretability based on inference equivariance. In this formulation, machine variables $V^{(m)}$ and human variables $V^{(h)}$ are connected by a translation map

$\tau: V^{(m)} \to V^{(h)}.$

Interpretability is defined by the commutation of translation and inference: $$P\bigl(\tau(V^{(m)}_q) \mid \tau(V^{(m)}_e)\bigr) = \tau \Bigl( P\bigl(V^{(m)}_q \mid V^{(m)}_e\bigr) \Bigr).$$ Under this criterion, a system is interpretable when a user can reliably predict its outputs by reasoning in their own reference system, because translating the machine state and then inferring yields the same result as inferring in the machine system and translating afterward (Barbiero et al., 17 Feb 2025).

This formalization emphasizes several properties. Equivariance can be asymmetric, because prediction only requires a mapping from machine variables to human concepts. It can be global or local, because interpretability may hold only on subsets of the state space. It also clarifies why explanation is not automatically interpretability: conditioning on evidence selects a subtable $P(V \mid V' = v')$, but that selection is interpretable only if commutation is preserved. The same framework further argues that post-hoc explanation can increase verification burden, because introducing a surrogate explanation system adds additional equivariance constraints rather than removing them (Barbiero et al., 17 Feb 2025).

A second formal line, developed for deep time series, defines semantic alignment through exact correspondence between latent variables and concept processes. The paper treats a human-meaningful variable as a concept

$C:\Omega \to \mathcal{C},$

with temporal extension $\{C_t\}_{t\in\mathcal{T}}$, and defines alignment by

$$P(U_t = C_t^U \mid X_{\le t}) = 1 \quad \text{a.s.}, \ \forall t$$

for instantaneous concepts and

$$P(Z_{t+1} = C_{t+1}^Z \mid X_{\le t}) = 1 \quad \text{a.s.}, \ \forall t$$

for dynamic concepts. The distinctive claim is that, in time series, semantic alignment must be preserved under temporal evolution; otherwise, alignment established at time $t$ can decay exponentially over time (Felice et al., 2 Feb 2026).

The same work extends semantic alignment from variables to mechanisms by requiring that a mechanism $V^{(h)}$0 belong to a human-allowed set $V^{(h)}$1. Examples of such constraints include known physical laws, monotonicity, stability, sparsity, invariances, or a fixed functional basis. This implies that semantic interpretability is not only about naming latent variables; it also concerns whether the relations among those variables satisfy user-recognized constraints (Felice et al., 2 Feb 2026).

3. Structural principles and architectural patterns

A recurring architectural principle is that semantic interpretability should be built into the intermediate representation rather than appended after training. Semantic Bottleneck Networks operationalize this by inserting a Semantic Bottleneck (SB) layer whose channels are explicitly tied to semantic concepts such as object parts and materials. Downstream layers are then finetuned so that final predictions depend only on those semantic outputs. In the Cityscapes case study, the paper uses a manually selected subset of 70 task-relevant concepts and also experiments with larger and smaller concept sets up to all 377 available part/material concepts from Broden+ (Losch et al., 2019).

Another recurring pattern is semantic supervision of hidden states. In video captioning, semantically meaningful topics are extracted from human descriptions with LDA, encoded as a binary topic vector, and injected through an interpretive loss

$V^{(h)}$2

The full objective combines task loss and semantic supervision: $V^{(h)}$3 This forces hidden features to predict human-derived topics such as people, woman, dance, walk, eat, and dog, and enables prediction difference maximization to associate individual neurons with topics (Dong et al., 2017).

A related but distinct approach treats semantic concepts as directions in feature space rather than as dedicated neurons. In the semantic vector (SeVec) framework, a named concept $V^{(h)}$4 is mapped to a vector $V^{(h)}$5 in feature space, learned from binarized deep features by a cosine-alignment objective whose optimal solution is

$V^{(h)}$6

These semantic vectors support both global interpretation, such as semantic hierarchy and concept relationships, and local interpretation, such as semantic-guided saliency that suppresses neurons not relevant to the target concept (Gu et al., 2019).

Semantic interpretability has also been formulated at the filter and decision levels through probability distributions over named visual attributes. In Semantic Network Interpretation, filter-level interpretation is represented by a posterior

$V^{(h)}$7

with TF/IDF priors and image-mediated likelihoods, while decision-level interpretation combines filter-attribute distributions and filter importance to yield a class-attribute distribution $V^{(h)}$8. The output is a textual summary of semantic evidence rather than only a visualization (Guo et al., 2018).

The most explicit synthesis of these structural ideas appears in Neural Interpretable Reasoning, which treats interpretability as a Markovian property and proposes semantic and functional re-parametrization. The resulting design pattern, neural generation and interpretable execution, separates a neural generator of concepts $V^{(h)}$9 and transparent parameters $\tau: V^{(m)} \to V^{(h)}.$0 from a symbolic executor $\tau: V^{(m)} \to V^{(h)}.$1. This is intended to preserve deep-model expressiveness while making verification of inference equivariance tractable (Barbiero et al., 17 Feb 2025).

4. Domain-specific realizations

The paradigm has been instantiated across several representational domains. In semantic segmentation, Semantic Bottleneck Networks replace thousands of latent channels with semantic concepts tied to object parts and materials. The paper reports that the baseline PSPNet achieves 78.5% mIoU and 95.2% pixel accuracy, while the full 377-concept setting reaches 76.4% mIoU and 95.7% pixel accuracy. It also reports that 76.3% of all pixels can be identified with 99% accuracy by a confidence estimator built on the semantic bottleneck, showing that semantic intermediate layers can support both diagnosis and calibrated trust (Losch et al., 2019).

In word embeddings, semantic interpretability has been studied through category-based decomposition. One line of work constructs the SEMCAT dataset with 110 categories and 6559 unique words, then measures how strongly each embedding dimension separates category words from the remaining vocabulary using the Bhattacharya distance. A semantic space is built as

$\tau: V^{(m)} \to V^{(h)}.$2

and interpretability is quantified by a dataset-driven score designed as an alternative to the word intrusion test (Senel et al., 2017). A complementary line modifies the GloVe objective with an additive interpretability term so that predefined concept groups from Roget’s Thesaurus align with designated embedding dimensions, while preserving performance on similarity and analogy benchmarks (Senel et al., 2018).

In face verification, semantic interpretability is operationalized through user-defined facial landmarks grouped into semantic facial regions. Global concept extraction is performed with semantic regions rather than arbitrary superpixels, and local explanation is expressed through similarity maps and textual explanations. A user study reports that the semantic approach was preferred by 78% overall, with SET_2 preferred by 50% overall, indicating that human users favored explanations stated in terms of eyes, nose, mouth, cheeks, forehead, and background over traditional superpixel heatmaps (Doh et al., 6 Jan 2025).

In CNN image classification, semantic explainable AI (S-XAI) extracts common traits across many samples using row-centered PCA and a genetic algorithm over SLIC superpixels, then identifies semantically sensitive neurons by comparing masked and unmasked semantic concepts. The framework introduces semantic probability to quantify the presence of a semantic concept in an image and uses semantic spaces for trustworthiness assessment and semantic sample searching (Xu et al., 2022).

In deep time series, the paradigm becomes explicitly temporal. The proposed blueprint decomposes modeling into concept encoding,

$\tau: V^{(m)} \to V^{(h)}.$3

concept propagation,

$\tau: V^{(m)} \to V^{(h)}.$4

and task decoding $\tau: V^{(m)} \to V^{(h)}.$5, together with a three-part supervision objective containing task, concept, and propagation terms. This explicitly generalizes concept bottleneck ideas to temporal settings (Felice et al., 2 Feb 2026).

5. Evaluation, utility, and trust-related properties

Semantic interpretability research has consistently linked concept-structured representations to diagnosis, intervention, and trust. In Semantic Bottleneck Networks, failure case analysis becomes concept-level: a construction beacon can be misclassified as a person because the bottleneck activates on skin-like material and foot/leg-like parts; a person can be misclassified because person-part evidence is missing; and suppressing building-related concepts changes predictions in the expected direction. This makes the representation inspectable and manipulable in semantic terms rather than only statistically interpretable (Losch et al., 2019).

The video-captioning framework extends this logic to human-in-the-loop correction. Once neurons are associated with topics, a user can identify a missing concept, enhance the corresponding activations, and fine-tune the encoder with

$\tau: V^{(m)} \to V^{(h)}.$6

The paper reports more accurate captions for 17 out of 20 videos after refinement, illustrating a concept-level correction interface rather than direct weight editing (Dong et al., 2017).

In time series, semantic alignment is argued to support Actionability, Verifiability, Fairness analysis, and Robustness. A semantically aligned model should permit user intervention on concepts or mechanisms, checking against domain constraints with formal verification tools such as model checking, tracing of protected or safety-critical attributes through time, and robustness grounded in stable concepts rather than opaque latent states (Felice et al., 2 Feb 2026).

Information-theoretic work on representations as language contributes a complementary evaluation perspective. It treats the mapping from sentences to vectors as a language-like code and measures information, variation, regularity, and disentanglement in latent space. The paper reports two distinct phases of transformer training: an initial phase of in-distribution learning and a later phase in which representations become robust to noise, with generalization improving in the second phase. This suggests that semantic structure in latent space can be studied not only through explicit concept labels but also through language-like regularity and compressibility (Conklin et al., 2024).

A plausible implication is that semantic interpretability is often most useful when it supports operations unavailable to purely local explanation methods: counterfactual editing, temporally consistent intervention, concept-conditioned generation, semantic retrieval, confidence estimation, and structured debugging. The literature repeatedly treats these capabilities as evidence that semantic representations are not merely descriptive but operationally meaningful (Felice et al., 2 Feb 2026, Xu et al., 2022).

6. Debates, limitations, and emerging directions

A central controversy concerns faithfulness. One critique argues that interpretability research has been dominated by an intrinsic paradigm and a post-hoc paradigm, both of which struggle with faithful explanation. The proposed alternatives are faithfulness measurable model paradigm, learn-to-faithfully-explain paradigm, and self-explaining model paradigm, all of which treat faithfulness as a first-class design objective rather than an afterthought (Madsen et al., 2024). This does not reject semantic interpretability, but it places pressure on semantic methods to show that their concepts or explanations are true to model behavior rather than merely plausible.

A second debate concerns the relationship between semantic interpretation and algorithmic interpretation. The cognitive-science taxonomy argues that identifying latent properties and understanding the computations over them are distinct goals. Probing, causal probing, dictionary learning, and sparse auto-encoders address representational content, whereas circuit discovery and causal abstraction address operations over representations. This implies that semantic interpretability may be necessary for understanding what is represented, but not sufficient for understanding the full procedure by which a model produces an output (Davies et al., 2024).

Another limitation is verification burden. The inference-equivariance framework argues that direct verification of interpretability scales exponentially with the number of variables, and that post-hoc explanation can increase the burden because it introduces a second system with its own translation and inference constraints. The proposed response is local, Markovian verification and neural re-parametrization, but the paper does not claim that interpretability becomes trivial; it claims only that verification can be made scalable enough to coexist with deep-model expressiveness (Barbiero et al., 17 Feb 2025).

Recent work on vision-language attribution identifies a further problem: semantic explanations can themselves hallucinate. In this account, explanation hallucination is a consequence of Linear Semantic Leakage in non-orthogonal text embedding spaces, and attribution maps inherit ghost signals from semantically related distractors. Orthogonal Semantic Projection is proposed as a geometric intervention that removes shared semantic components before attribution, indicating that semantic interpretability must also confront the geometry of representation spaces if it is to remain robust (Bilgiç et al., 8 Jun 2026).

Taken together, these debates show that the semantic interpretability paradigm is not a single method but a family of positions. Its strongest common claim is that interpretability should be anchored in human-meaningful variables, concepts, or mechanisms. Its unresolved questions concern faithfulness, verification, temporal persistence, representation level, and the relation between semantic content and learned algorithms. This suggests that the paradigm is best understood as a unifying orientation: interpretability is achieved when model reasoning can be translated into semantic structures that users can inspect, predict with, verify, and in some cases directly manipulate (Barbiero et al., 17 Feb 2025, Madsen et al., 2024).