Levels of Self-Explainability

• Levels of self-explainability are frameworks that categorize system explanations by depth, granularity, and autonomy, such as evidence, mechanism, and procedure.
• Architectural levels detail how models expose decision-relevant units from tokens to spans to concepts, aligning explanation with prediction mechanisms.
• Maturity models address the trade-off between transparency and tractability, illustrating progress from basic, manual explanations to fully autonomous, interactive systems.

Searching arXiv for recent and foundational papers on self-explainability, self-explaining models, and levels/frameworks of explanation. Levels of self-explainability denote graded ways in which a system can account for its own behavior, ranging from merely exposing decision-relevant artifacts to autonomously generating runtime explanations tailored to stakeholders and contexts. Across recent work, the term has been used in at least three closely related senses: as a typology of explanation content such as procedure, evidence, and proximal mechanism (Tan, 2021); as a structural property of models whose explanatory quantities are part of the forward computation, often at multiple granularities such as words, spans, phrases, or concepts (Sun et al., 2020, Rajagopal et al., 2021); and as a system-level maturity hierarchy in which explanation evolves from passive runtime availability to autonomous, target-specific, interactive, and eventually full behavioral self-explanation (Beyer et al., 8 Jun 2026). Taken together, these lines of research suggest that “level” can refer simultaneously to explanatory depth, representational granularity, faithfulness to internal decision processes, and the degree of autonomy with which explanations are generated.

1. Three foundational explanation types

A major conceptual starting point is the claim that what NLP has often called “explanations” actually covers three qualitatively different kinds of information: proximal mechanism, evidence, and procedure (Tan, 2021). These categories can be interpreted as three basic levels of self-explainability.

Proximal mechanism explanations answer how a label follows from the input. They aim to provide “the mechanism behind the predicted label,” that is, “how to infer the label from the text,” and correspond to “efficient cause” in Lombrozo’s terminology (Tan, 2021). In NLP datasets, such explanations often take the form of textual reasoning chains. They are typically local, instance-specific, and inherently partial: they do not enumerate all logical axioms, anaphora rules, or world knowledge needed for a complete derivation (Tan, 2021).

Evidence explanations identify where relevant support is located in the input. They are rationales or highlights, such as selected tokens or sentences, but “evidence alone does not explain” because it does not specify how the evidence implies the label (Tan, 2021). This form is concrete and justificatory rather than mechanistic. It dominates many dataset designs, including sentiment and fact verification corpora, where proxy questions such as “Highlight ALL words that reflect this sentiment” or “If I was given only the selected sentences, do I have strong reason to believe the claim is true/false?” shape the resulting annotations (Tan, 2021).

Procedure explanations specify how to operate the task through rules or instructions. They are stepwise, operational, and often directly translatable into programmatic forms, as in natural-language explanations that become labeling functions or constrained templates (Tan, 2021). From the psychological perspective summarized in the paper, procedure is not itself explanation in the stronger sense; it is “how-to” knowledge rather than explanatory understanding of underlying mechanisms (Tan, 2021).

This tripartite distinction implies an explanatory ordering. Procedure gives operational policies, evidence gives local support, and proximal mechanism comes closest to genuine explanation. The same paper relates this to the progression “Prediction < Understanding < Explanation < Theory,” placing evidence and procedure below full explanation and treating proximal mechanisms as the closest available analogue (Tan, 2021).

2. Architectural levels: from tokens to spans to concepts

A second sense of levels of self-explainability concerns the granularity at which a model exposes decision-relevant units. In this line of work, self-explainability is instantiated as an architectural property: the mechanism that explains a prediction is also the mechanism that computes it.

In “Self-Explaining Structures Improve NLP Models” (Sun et al., 2020), the explanatory units are all contiguous spans of an input sequence. The model augments a standard encoder with a Span Info Collecting (SIC) layer and an interpretation layer. For span $\mathbf{x}(i,j)$, the model computes a span representation

$$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$

scores spans by

$o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$

normalizes them to obtain

$$\alpha(i,j) = \frac{\exp(o(i,j))}{\sum_{i=1}^N \sum_{j=i}^N \exp(o(i,j))},$$

and forms the decisive representation

$$\tilde{\mathbf{h}} = \sum_{i=1}^N \sum_{j=i}^N \alpha(i,j)\,\mathbf{h}(i,j).$$

Because $\alpha(i,j)$ is both an explanatory score and a causal mixing coefficient in the forward pass, span weights are treated as self-explanations rather than post-hoc probes (Sun et al., 2020). The same architecture supports explanation at token level, short phrase level, clause or sentence level, and potentially paragraph level, depending on span length (Sun et al., 2020).

A related but distinct architecture is “SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers” (Rajagopal et al., 2021), which organizes explanations around phrase-based concepts extracted from constituency parses. It introduces a Locally Interpretable Layer (LIL) and a Globally Interpretable Layer (GIL). LIL scores input phrases by their relevance to the predicted class through activation-difference computations, while GIL retrieves influential training-set concepts by similarity to the sentence representation (Rajagopal et al., 2021). This yields a two-level explanatory decomposition: local, phrase-wise contributions from the input and global, prototype-like phrase influences from the training distribution (Rajagopal et al., 2021).

These works jointly establish that “level” may refer to text granularity. A model can be self-explaining at word, phrase, clause, sentence, or document scale if explanatory quantities are explicitly represented and used in prediction. The literature explicitly contrasts this with gradient saliency and attention-based explanations, which are usually native only at token level and require ad hoc aggregation for larger units (Sun et al., 2020).

3. Degrees of coupling, explicitness, and faithfulness

A third major axis is the degree to which explanation is structurally coupled to prediction. “Explainability in Human-Agent Systems” formalizes explainability as

$\E = \I(L(R \times F,T)),$

where $L$ is the learning algorithm, $\I$ is an interpretation function, and $\E$ is the human-centric aim to understand the agent’s logic (Rosenfeld et al., 2019). This framework distinguishes explicitness, the extent to which the output of $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$0 is immediate and understandable, from faithfulness, the degree to which the logic within $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$1 is similar to that of $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$2 (Rosenfeld et al., 2019).

On this view, transparency is a high level of self-explainability, attained when the logic within $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$3 is readily understandable and identical to $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$4, expressed as $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$5 (Rosenfeld et al., 2019). By contrast, justification is a much weaker level: it provides reasons why a decision is correct without information about how it was made and therefore “requires no connection between $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$6 and $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$7 and no faithfulness” (Rosenfeld et al., 2019).

This distinction recurs across recent self-explaining model design. The span-based architecture in (Sun et al., 2020) argues that decoupled probing and surrogate models inherently limit faithfulness because they approximate effects without being part of the predictive mechanism. The same concern appears in work on human explanations, which warns that human-provided mechanisms may actually be post-hoc rationalizations rather than faithful reports of cognitive process, and therefore should not be treated as unquestioned ground truth for model training or evaluation (Tan, 2021).

A closely related perspective in graph learning distinguishes minimal and faithful explanations formally. “Beyond Topological Self-Explainable GNNs” analyzes popular self-explainable GNNs as optimizing Trivial Explanations (TEs), also called minimal explanations: subgraphs $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$8 such that $$\mathbf{h}(i,j) = F(\mathbf{h}_i^K, \mathbf{h}_j^K),$$9 and no smaller subgraph preserves the label (Azzolin et al., 4 Feb 2025). The paper proves that standard sparsity- and information-bottleneck-based SE-GNN losses recover TEs under idealized conditions, but also shows that TEs can be less informative than Prime Implicant (PI) explanations and can be misaligned with perturbation-based faithfulness (Azzolin et al., 4 Feb 2025). This yields a finer hierarchy: minimal self-explanation, PI-level robust sufficiency, and stronger notions of faithfulness based on necessity and sufficiency under perturbation (Azzolin et al., 4 Feb 2025).

4. Runtime maturity levels in autonomous and adaptive systems

A fourth use of levels of self-explainability is explicitly system-level. A recent systematic review on self-adaptive and self-organising systems defines Self-Explainability as “the ability to generate and output explanations of behaviour at runtime” and introduces a six-level maturity model (Beyer et al., 8 Jun 2026). Here the “self” in self-explanation refers to explanations generated autonomously from within the system, either by embedded mechanisms or by higher-level explanatory components that are still part of the system architecture (Beyer et al., 8 Jun 2026).

The six levels are Level 0 – No Explainability, Level 1 – Explainability (SX-ready), Level 2 – Self-Explainability, Level 3 – Target-Specific SX, Level 4 – Adaptable SX, and Level 5 – Full SX (Beyer et al., 8 Jun 2026). Level 1 requires that explanations of behavior exist and can be accessed at runtime, but they are not generated autonomously. Level 2 adds autonomous recognition of behavior that requires explanation and autonomous generation of the explanation. Level 3 introduces tailoring to different target groups. Level 4 adds interactivity and self-optimisation, allowing explanations to be elaborated on request or adapted over time. Level 5 is a long-term vision in which all relevant past, present, future, and hypothetical behavior is explainable for all target groups (Beyer et al., 8 Jun 2026).

This maturity hierarchy is orthogonal to the content taxonomy of mechanism, evidence, and procedure. A system may expose evidence-level explanations yet remain at Level 1 if explanations must be queried manually. Conversely, a system could autonomously deliver procedure-level or mechanism-level explanations at runtime and thus qualify for Level 2 or beyond. The review emphasizes that most current approaches do not exceed Level 2 in realized systems, and that Level 3–5 remain largely conceptual (Beyer et al., 8 Jun 2026).

This system view also makes timing itself part of self-explainability. Explanations may be real-time, post-action, or forensic, provided they still concern runtime behavior rather than only offline model inspection (Beyer et al., 8 Jun 2026). The distinction is consequential in autonomous traffic, smart environments, industrial CPS, and robotics, where explanations must be synchronized with adaptation, anomaly detection, or human oversight (Beyer et al., 8 Jun 2026).

5. Human cognition, stakeholders, and contextual alignment

A further strand of the literature treats levels of self-explainability as levels of alignment with human cognitive and social requirements. One proposal uses Bloom’s taxonomy to structure explanation design at the levels of Knowledge, Comprehension, and Application, with counterfactual explanations serving as the operative medium (Suffian et al., 2022). At the Knowledge level, the system provides definitions, terms, and basic facts; at Comprehension, it supports “why” questions, examples, and actionable changes; at Application, it allows users to alter inputs and observe coherent outcome changes (Suffian et al., 2022). Although this work is conceptual, it suggests that a self-explaining system could adapt explanation form to a user’s cognitive state and support transitions from fact recall to causal understanding to interactive use (Suffian et al., 2022).

Stakeholder-centered work similarly proposes multilevel frameworks where explanations are aligned with developers, domain experts, end-users, regulators, and society. One such framework defines three layers: algorithmic and domain-informed explainability, human-centered and interactive explainability, and societal transparency and trust, with LLMs highlighted as explanation mediators at the social layer (Bello et al., 6 Jun 2025). Another maps explanation modes to Malle’s five cognitive categories—Knowledge Structures, Simulation/Projection, Covariation, Direct Recall, and Rationalization—and uses them to recommend different XAI techniques depending on whether the task is perceived as emulation-like “Actions” or discovery-like “Experiences” (Jean et al., 2 Sep 2025).

These stakeholder and cognition-oriented frameworks do not always use the phrase “self-explainability,” but they extend the concept by making clear that self-explanation is not exhausted by structural coupling or formal faithfulness. Explanations must also be appropriate for their recipients. This introduces levels of audience adaptation and dialogic refinement, closely matching Level 3 and Level 4 in the runtime maturity model (Beyer et al., 8 Jun 2026, Bello et al., 6 Jun 2025).

6. Formal and modal extensions beyond text classification

The idea of levels of self-explainability has also been operationalized outside standard NLP classification.

In cooperative-game-theoretic work on black-box explainability, a levels structure is defined as a hierarchy of partitions from singleton features to the grand coalition, enabling explanations at multiple abstraction levels through intermediate concepts (Patil et al., 2023). Each level corresponds to grouped features or meronomies defined by domain experts, and Contextual Importance and Utility (CIU) is computed for coalitions at any level. This yields explanations at raw-feature level, intermediate-concept level, and very abstract whole-system level (Patil et al., 2023).

In neural networks, “Feature Leveling Deep Neural Networks” proposes isolating low-level from high-level features on a per-layer basis so that a final GLM-like layer becomes interpretable over distinct feature levels (Lu et al., 2019). Here level means the number of hidden layers required before a feature can be “sufficiently utilized by the GLM layer” (Lu et al., 2019). The result is a stratified form of self-explanation in which prediction is linear with respect to various levels of features, rather than with respect to an opaque final representation (Lu et al., 2019).

In graph learning, recent work has pushed from instance-level to class-level self-explainability. “Towards Faithful Class-level Self-explainability in Graph Neural Networks by Subgraph Dependencies” defines a class-level self-explainer as a mapping

$o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$0

where $o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$1 is a global set of salient subgraphs and $o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$2 is a class–subgraph dependency matrix (Liu et al., 15 Aug 2025). This work explicitly contrasts instance-level explanations, which highlight graph components for individual predictions, with class-level explanations, which uncover patterns consistently predictive for an entire class (Liu et al., 15 Aug 2025). The distinction amounts to another elevation in explanatory scope.

In reinforcement learning, “Explainability Via Causal Self-Talk” develops a semantic belief layer $o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$3 that the agent communicates to itself across time and that is made causally active in future policy through interventions (Roy et al., 2022). This creates an implicit hierarchy linking mechanistic hidden state $o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$4, semantic self-talk $o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$5, and policy behavior $o(i,j) = \hat{\mathbf{h}}^\top \mathbf{h}(i,j),$6, and the paper interprets progress from no explanation, to readable but non-causal decoders, to causally faithful self-explanations as a progression in self-explainability (Roy et al., 2022).

In LLMs, “Beyond External Monitors” proposes improving transparency by structuring hidden representations so that same-concept examples cluster together and different concepts are separated in representation space (Chen et al., 7 Feb 2025). This paper criticizes both chain-of-thought and external monitoring tools as insufficiently faithful and argues for a stronger level in which explanations are “carried by representations synchronously with the LLMs’ output” (Chen et al., 7 Feb 2025). Here the relevant levels are output-level rationales, representation-level monitoring by external tools, and intrinsic representation-level transparency induced by training (Chen et al., 7 Feb 2025).

7. Tensions, limitations, and common misconceptions

Several recurring cautions cut across the literature. The first is that prediction does not entail explanation. Humans may produce correct labels without being able to explain why, and models may behave analogously (Tan, 2021). This undermines any simple assumption that explanation supervision is easy to collect or that explanation quality follows from task accuracy.

A second caution is that not all explanations are equal. The procedure–evidence–mechanism distinction shows that datasets and systems labeled “explainable” may in fact target profoundly different objects (Tan, 2021). Evidence rationales do not by themselves provide a mechanism, and procedure may amount only to operational know-how (Tan, 2021).

A third is that self-explaining architectures do not automatically guarantee full faithfulness. Minimal explanations in SE-GNNs can be tractable and useful but may fail stronger perturbational faithfulness criteria (Azzolin et al., 4 Feb 2025). Human natural-language explanations may be incomplete, inaccurate, or biased, so alignment to them may reward plausible stories rather than genuine reasoning (Tan, 2021). Likewise, output-level chain-of-thought in LLMs can be unfaithful to internal computation (Chen et al., 7 Feb 2025).

A fourth tension concerns the trade-off between informativeness and tractability. Prime implicant and fully faithful explanations are often more informative than minimal ones, but they can be computationally intractable or prohibitively large (Azzolin et al., 4 Feb 2025). Class-level and global explanations reduce cognitive load by summarizing recurring patterns, but they require evaluation criteria different from those for instance-level rationales (Liu et al., 15 Aug 2025).

A plausible implication is that levels of self-explainability should not be treated as a single linear ranking. The literature instead supports a multidimensional interpretation: explanatory type (procedure, evidence, mechanism), granularity (token, span, concept, subgraph, class), coupling to the predictive mechanism (post-hoc, intrinsic, transparent), runtime autonomy (manual access to self-triggered explanation), and audience adaptation (generic to target-specific to interactive) (Tan, 2021, Sun et al., 2020, Rosenfeld et al., 2019, Beyer et al., 8 Jun 2026). This suggests that a system’s level of self-explainability is best understood as its position in a structured space of explanatory capabilities rather than as a single score.