---
title: Levels of Self-Explainability
url: https://www.emergentmind.com/topics/levels-of-self-explainability
type: topic
---

# Levels of Self-Explainability

Searching arXiv for recent and foundational papers on self-explainability, self-explaining models, and levels/frameworks of explanation.
Levels of self-explainability denote graded ways in which a system can account for its own behavior, ranging from merely exposing decision-relevant artifacts to autonomously generating runtime explanations tailored to stakeholders and contexts. Across recent work, the term has been used in at least three closely related senses: as a typology of explanation content such as **procedure**, **evidence**, and **proximal mechanism** [2106.11988]; as a structural property of models whose explanatory quantities are part of the forward computation, often at multiple granularities such as words, spans, phrases, or concepts [2012.01786], [2103.12279]; and as a system-level maturity hierarchy in which explanation evolves from passive runtime availability to autonomous, target-specific, interactive, and eventually full behavioral self-explanation [2606.09568]. Taken together, these lines of research suggest that “level” can refer simultaneously to explanatory depth, representational granularity, faithfulness to internal decision processes, and the degree of autonomy with which explanations are generated.

## 1. Three foundational explanation types

A major conceptual starting point is the claim that what NLP has often called “explanations” actually covers three qualitatively different kinds of information: **proximal mechanism**, **evidence**, and **procedure** [2106.11988]. These categories can be interpreted as three basic levels of self-explainability.

**Proximal mechanism** explanations answer how a label follows from the input. They aim to provide “the mechanism behind the predicted label,” that is, “how to infer the label from the text,” and correspond to “efficient cause” in Lombrozo’s terminology [2106.11988]. In NLP datasets, such explanations often take the form of textual reasoning chains. They are typically local, instance-specific, and inherently partial: they do not enumerate all logical axioms, anaphora rules, or world knowledge needed for a complete derivation [2106.11988].

**Evidence** explanations identify where relevant support is located in the input. They are rationales or highlights, such as selected tokens or sentences, but “evidence alone does not explain” because it does not specify how the evidence implies the label [2106.11988]. This form is concrete and justificatory rather than mechanistic. It dominates many dataset designs, including sentiment and fact verification corpora, where proxy questions such as “Highlight ALL words that reflect this sentiment” or “If I was given only the selected sentences, do I have strong reason to believe the claim is true/false?” shape the resulting annotations [2106.11988].

**Procedure** explanations specify how to operate the task through rules or instructions. They are stepwise, operational, and often directly translatable into programmatic forms, as in natural-language explanations that become labeling functions or constrained templates [2106.11988]. From the psychological perspective summarized in the paper, procedure is not itself explanation in the stronger sense; it is “how-to” knowledge rather than explanatory understanding of underlying mechanisms [2106.11988].

This tripartite distinction implies an explanatory ordering. Procedure gives operational policies, evidence gives local support, and proximal mechanism comes closest to genuine explanation. The same paper relates this to the progression “Prediction < Understanding < Explanation < Theory,” placing evidence and procedure below full explanation and treating proximal mechanisms as the closest available analogue [2106.11988].

## 2. Architectural levels: from tokens to spans to concepts

A second sense of levels of self-explainability concerns the granularity at which a model exposes decision-relevant units. In this line of work, self-explainability is instantiated as an architectural property: the mechanism that explains a prediction is also the mechanism that computes it.

In “Self-Explaining Structures Improve NLP Models” [2012.01786], the explanatory units are **all contiguous spans** of an input sequence. The model augments a standard encoder with a **Span Info Collecting (SIC) layer** and an **interpretation layer**. For span $\mathbf{x}(i,j)$, the model computes a span representation
$$
\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),
$$
scores spans by
$$
o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),
$$
normalizes them to obtain
$$
\alpha(i,j) = \frac{\exp(o(i,j))}{\sum_{i=1}^N \sum_{j=i}^N \exp(o(i,j))},
$$
and forms the decisive representation
$$
\tilde{\mathbf{h}} = \sum_{i=1}^N \sum_{j=i}^N \alpha(i,j)\,\mathbf{h}(i,j).
$$
Because $\alpha(i,j)$ is both an explanatory score and a causal mixing coefficient in the forward pass, span weights are treated as self-explanations rather than post-hoc probes [2012.01786]. The same architecture supports explanation at token level, short phrase level, clause or sentence level, and potentially paragraph level, depending on span length [2012.01786].

A related but distinct architecture is “SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers” [2103.12279], which organizes explanations around **phrase-based concepts** extracted from constituency parses. It introduces a **Locally Interpretable Layer (LIL)** and a **Globally Interpretable Layer (GIL)**. LIL scores input phrases by their relevance to the predicted class through activation-difference computations, while GIL retrieves influential training-set concepts by similarity to the sentence representation [2103.12279]. This yields a two-level explanatory decomposition: local, phrase-wise contributions from the input and global, prototype-like phrase influences from the training distribution [2103.12279].

These works jointly establish that “level” may refer to text granularity. A model can be self-explaining at word, phrase, clause, sentence, or document scale if explanatory quantities are explicitly represented and used in prediction. The literature explicitly contrasts this with gradient saliency and attention-based explanations, which are usually native only at token level and require ad hoc aggregation for larger units [2012.01786].

## 3. Degrees of coupling, explicitness, and faithfulness

A third major axis is the degree to which explanation is structurally coupled to prediction. “Explainability in Human-Agent Systems” formalizes explainability as
$$
\E = \I(L(R \times F,T)),
$$
where $L$ is the learning algorithm, $\I$ is an interpretation function, and $\E$ is the human-centric aim to understand the agent’s logic [1904.08123]. This framework distinguishes **explicitness**, the extent to which the output of $\I$ is immediate and understandable, from **faithfulness**, the degree to which the logic within $\I$ is similar to that of $L$ [1904.08123].

On this view, **transparency** is a high level of self-explainability, attained when the logic within $\I$ is readily understandable and identical to $L$, expressed as $\I \simeq L$ [1904.08123]. By contrast, **justification** is a much weaker level: it provides reasons why a decision is correct without information about how it was made and therefore “requires no connection between $\I$ and $L$ and no faithfulness” [1904.08123].

This distinction recurs across recent self-explaining model design. The span-based architecture in [2012.01786] argues that decoupled probing and surrogate models inherently limit faithfulness because they approximate effects without being part of the predictive mechanism. The same concern appears in work on human explanations, which warns that human-provided mechanisms may actually be post-hoc rationalizations rather than faithful reports of cognitive process, and therefore should not be treated as unquestioned ground truth for model training or evaluation [2106.11988].

A closely related perspective in graph learning distinguishes minimal and faithful explanations formally. “Beyond Topological Self-Explainable GNNs” analyzes popular self-explainable GNNs as optimizing **Trivial Explanations (TEs)**, also called minimal explanations: subgraphs $R \subseteq G$ such that $g(R)=g(G)$ and no smaller subgraph preserves the label [2502.02719]. The paper proves that standard sparsity- and information-bottleneck-based SE-GNN losses recover TEs under idealized conditions, but also shows that TEs can be less informative than **Prime Implicant (PI)** explanations and can be misaligned with perturbation-based faithfulness [2502.02719]. This yields a finer hierarchy: minimal self-explanation, PI-level robust sufficiency, and stronger notions of faithfulness based on necessity and sufficiency under perturbation [2502.02719].

## 4. Runtime maturity levels in autonomous and adaptive systems

A fourth use of levels of self-explainability is explicitly system-level. A recent systematic review on self-adaptive and self-organising systems defines **Self-Explainability** as “the ability to generate and output explanations of behaviour at runtime” and introduces a six-level maturity model [2606.09568]. Here the “self” in self-explanation refers to explanations generated autonomously from within the system, either by embedded mechanisms or by higher-level explanatory components that are still part of the system architecture [2606.09568].

The six levels are **Level 0 – No Explainability**, **Level 1 – Explainability (SX-ready)**, **Level 2 – Self-Explainability**, **Level 3 – Target-Specific SX**, **Level 4 – Adaptable SX**, and **Level 5 – Full SX** [2606.09568]. Level 1 requires that explanations of behavior exist and can be accessed at runtime, but they are not generated autonomously. Level 2 adds autonomous recognition of behavior that requires explanation and autonomous generation of the explanation. Level 3 introduces tailoring to different target groups. Level 4 adds interactivity and self-optimisation, allowing explanations to be elaborated on request or adapted over time. Level 5 is a long-term vision in which all relevant past, present, future, and hypothetical behavior is explainable for all target groups [2606.09568].

This maturity hierarchy is orthogonal to the content taxonomy of mechanism, evidence, and procedure. A system may expose evidence-level explanations yet remain at Level 1 if explanations must be queried manually. Conversely, a system could autonomously deliver procedure-level or mechanism-level explanations at runtime and thus qualify for Level 2 or beyond. The review emphasizes that most current approaches do not exceed Level 2 in realized systems, and that Level 3–5 remain largely conceptual [2606.09568].

This system view also makes timing itself part of self-explainability. Explanations may be **real-time**, **post-action**, or **forensic**, provided they still concern runtime behavior rather than only offline model inspection [2606.09568]. The distinction is consequential in autonomous traffic, smart environments, industrial CPS, and robotics, where explanations must be synchronized with adaptation, anomaly detection, or human oversight [2606.09568].

## 5. Human cognition, stakeholders, and contextual alignment

A further strand of the literature treats levels of self-explainability as levels of alignment with human cognitive and social requirements. One proposal uses **Bloom’s taxonomy** to structure explanation design at the levels of **Knowledge**, **Comprehension**, and **Application**, with counterfactual explanations serving as the operative medium [2211.00103]. At the Knowledge level, the system provides definitions, terms, and basic facts; at Comprehension, it supports “why” questions, examples, and actionable changes; at Application, it allows users to alter inputs and observe coherent outcome changes [2211.00103]. Although this work is conceptual, it suggests that a self-explaining system could adapt explanation form to a user’s cognitive state and support transitions from fact recall to causal understanding to interactive use [2211.00103].

Stakeholder-centered work similarly proposes multilevel frameworks where explanations are aligned with developers, domain experts, end-users, regulators, and society. One such framework defines three layers: **algorithmic and domain-informed explainability**, **human-centered and interactive explainability**, and **societal transparency and trust**, with LLMs highlighted as explanation mediators at the social layer [2506.05887]. Another maps explanation modes to Malle’s five cognitive categories—**Knowledge Structures**, **Simulation/Projection**, **Covariation**, **Direct Recall**, and **Rationalization**—and uses them to recommend different XAI techniques depending on whether the task is perceived as emulation-like “Actions” or discovery-like “Experiences” [2509.02388].

These stakeholder and cognition-oriented frameworks do not always use the phrase “self-explainability,” but they extend the concept by making clear that self-explanation is not exhausted by structural coupling or formal faithfulness. Explanations must also be appropriate for their recipients. This introduces levels of **audience adaptation** and **dialogic refinement**, closely matching Level 3 and Level 4 in the runtime maturity model [2606.09568], [2506.05887].

## 6. Formal and modal extensions beyond text classification

The idea of levels of self-explainability has also been operationalized outside standard NLP classification.

In cooperative-game-theoretic work on black-box explainability, a **levels structure** is defined as a hierarchy of partitions from singleton features to the grand coalition, enabling explanations at multiple abstraction levels through **intermediate concepts** [2303.11920]. Each level corresponds to grouped features or meronomies defined by domain experts, and **Contextual Importance and Utility (CIU)** is computed for coalitions at any level. This yields explanations at raw-feature level, intermediate-concept level, and very abstract whole-system level [2303.11920].

In neural networks, “Feature Leveling Deep Neural Networks” proposes isolating low-level from high-level features on a per-layer basis so that a final GLM-like layer becomes interpretable over distinct feature levels [1905.10009]. Here level means the number of hidden layers required before a feature can be “sufficiently utilized by the GLM layer” [1905.10009]. The result is a stratified form of self-explanation in which prediction is linear with respect to various levels of features, rather than with respect to an opaque final representation [1905.10009].

In graph learning, recent work has pushed from **instance-level** to **class-level** self-explainability. “Towards Faithful Class-level Self-explainability in Graph Neural Networks by Subgraph Dependencies” defines a class-level self-explainer as a mapping
$$
\Phi: f_{\text{GNN}}(\mathcal{G}) \rightarrow \{W, \mathcal{SG}\},
$$
where $\mathcal{SG}$ is a global set of salient subgraphs and $W \in \mathbb{R}^{|\mathcal{C}|\times|\mathcal{SG}|}$ is a class–subgraph dependency matrix [2508.11513]. This work explicitly contrasts instance-level explanations, which highlight graph components for individual predictions, with class-level explanations, which uncover patterns consistently predictive for an entire class [2508.11513]. The distinction amounts to another elevation in explanatory scope.

In reinforcement learning, “Explainability Via Causal Self-Talk” develops a semantic belief layer $z_t$ that the agent communicates to itself across time and that is made causally active in future policy through interventions [2211.09937]. This creates an implicit hierarchy linking mechanistic hidden state $m_t$, semantic self-talk $z_t$, and policy behavior $\pi_t$, and the paper interprets progress from no explanation, to readable but non-causal decoders, to causally faithful self-explanations as a progression in self-explainability [2211.09937].

In large language models, “Beyond External Monitors” proposes improving transparency by structuring hidden representations so that same-concept examples cluster together and different concepts are separated in representation space [2502.05242]. This paper criticizes both chain-of-thought and external monitoring tools as insufficiently faithful and argues for a stronger level in which explanations are “carried by representations synchronously with the LLMs’ output” [2502.05242]. Here the relevant levels are output-level rationales, representation-level monitoring by external tools, and intrinsic representation-level transparency induced by training [2502.05242].

## 7. Tensions, limitations, and common misconceptions

Several recurring cautions cut across the literature. The first is that **prediction does not entail explanation**. Humans may produce correct labels without being able to explain why, and models may behave analogously [2106.11988]. This undermines any simple assumption that explanation supervision is easy to collect or that explanation quality follows from task accuracy.

A second caution is that **not all explanations are equal**. The procedure–evidence–mechanism distinction shows that datasets and systems labeled “explainable” may in fact target profoundly different objects [2106.11988]. Evidence rationales do not by themselves provide a mechanism, and procedure may amount only to operational know-how [2106.11988].

A third is that **self-explaining architectures do not automatically guarantee full faithfulness**. Minimal explanations in SE-GNNs can be tractable and useful but may fail stronger perturbational faithfulness criteria [2502.02719]. Human natural-language explanations may be incomplete, inaccurate, or biased, so alignment to them may reward plausible stories rather than genuine reasoning [2106.11988]. Likewise, output-level chain-of-thought in LLMs can be unfaithful to internal computation [2502.05242].

A fourth tension concerns the trade-off between **informativeness and tractability**. Prime implicant and fully faithful explanations are often more informative than minimal ones, but they can be computationally intractable or prohibitively large [2502.02719]. Class-level and global explanations reduce cognitive load by summarizing recurring patterns, but they require evaluation criteria different from those for instance-level rationales [2508.11513].

A plausible implication is that levels of self-explainability should not be treated as a single linear ranking. The literature instead supports a multidimensional interpretation: explanatory **type** (procedure, evidence, mechanism), **granularity** (token, span, concept, subgraph, class), **coupling** to the predictive mechanism (post-hoc, intrinsic, transparent), **runtime autonomy** (manual access to self-triggered explanation), and **audience adaptation** (generic to target-specific to interactive) [2106.11988], [2012.01786], [1904.08123], [2606.09568]. This suggests that a system’s level of self-explainability is best understood as its position in a structured space of explanatory capabilities rather than as a single score.

Source: https://www.emergentmind.com/topics/levels-of-self-explainability