Self-Explanation (SE): Mechanisms & Applications

• Self-Explanation (SE) is a process where agents articulate their internal reasoning, balancing sufficiency to support correct outputs with conciseness to remove irrelevant details.
• It spans multiple operational variants such as chain-of-thought in LLMs, saliency maps for training, logic-rule explanations in interpretable models, and introspective dialogue analyses.
• SE enhances both model performance and transparency by serving as a training signal, enabling improved prediction, optimization, and alignment through intermediate representations.

Self-Explanation (SE) denotes a family of procedures in which a learner, model, or agent generates an explicit account of its own reasoning, decision basis, or functional organization, and then uses or exposes that account for prediction, optimization, critique, or communication. In contemporary work, SE appears as chain-of-thought and natural-language rationales in LLMs, class-wise saliency maps used as privileged information during training, logic rules and sufficient feature subsets produced in the forward pass of self-explaining models, relational predicate utilities for reinforcement learning from demonstrations, and introspective question answering over an explicit self-model in social agents. A recent synthesis frames one central tension as a trade-off between sufficiency—retaining enough information to support the correct answer—and conciseness—compressing away irrelevant detail (Zahedzadeh et al., 15 Feb 2026).

1. Conceptual scope and operational variants

Across the literature, SE is not a single mechanism but a shared design pattern: the system externalizes an intermediate representation of “why this output” or “how this task is carried out,” and that representation becomes functionally consequential. In the learning sciences, SE is “a cognitive process wherein individuals elucidate to themselves the meaning of new information, often integrating it with existing cognitive schema,” and is treated as an intra-individual generative activity rather than peer-directed explanation (Gao et al., 25 Mar 2025). In LLM reasoning, SE usually denotes intermediate natural-language reasoning such as chain-of-thought, natural-language rationales, or utterance-level dialogue interpretations (Zahedzadeh et al., 15 Feb 2026). In model training, SE may be class-wise saliency maps that shape soft targets, or explanations judged by an internal critic and fed back into optimization (Gu et al., 2020). In intrinsically interpretable architectures, SE may take the form of logic rules, sufficient subsets, or learned explanation vectors computed as part of the forward pass (Lee et al., 2022).

Setting	Explanatory object	Functional role
LLM reasoning	Chain-of-thought, rationale, compressed explanation	Support answer prediction and enable sufficiency scoring
Dialogue understanding	Utterance-by-utterance explanations	Improve contextual comprehension before task execution
Introspective training	Class-wise saliency maps or critic-evaluable explanations	Guide optimization and regularize learning
Self-explaining architectures	Logic rules, sufficient subsets, explanation vectors	Constrain prediction through interpretable intermediates
Social and RL agents	Predicate utilities, future-behavior instructions, self-model QA	Support coordination, reward shaping, and transparency

This breadth matters because different traditions optimize different explanation properties. Some works emphasize model-facing sufficiency, meaning that another model can recover the correct answer from the explanation. Others emphasize human precision, meaning that humans judge the explanation to be a correct and reasonable rationale. Still others treat explanations as privileged training signals that improve generalization even when they are not directly user-facing (Zahedzadeh et al., 15 Feb 2026).

2. Formal objectives: sufficiency, conciseness, faithfulness, and human precision

A prominent formalization casts explanations as an Information Bottleneck (IB). In that view, an explanation $Z$ is a compressed representation of input $X$ that should preserve answer-relevant information $Y$, with canonical objective

$\max_Z \; I(Z; Y) - \beta I(X; Z).$

For ARC Challenge multiple-choice QA, this mapping is instantiated as $X$ = question plus options, $Y$ = gold answer, and $Z$ = the explanation generated by the LLM. Sufficiency is approximated by the scorer model’s probability of the correct option given question, explanation, and options, while conciseness is approximated by explanation length reduction (Zahedzadeh et al., 15 Feb 2026).

The same paper operationalizes sufficiency for an explanation $Z_v$ by

$\text{Sufficiency}(Z_v) = p(y \mid P_v),$

where $P_v$ is the prompt consisting of the question, compressed explanation, and options. At dataset level, sufficiency is averaged over examples. Conciseness is enforced by prompting the model to regenerate explanations whose length is at most $X$0 of the original word count, for $X$1. Explicit mentions of option labels or verbatim option text are masked with [MASK], so sufficiency reflects reasoning cues rather than direct answer leakage (Zahedzadeh et al., 15 Feb 2026).

Other traditions formalize SE with different targets. In "SELOR" (Lee et al., 2022), prediction is explicitly factored through a logic-rule explanation $X$2, with

$X$3

and the central evaluation notion is human precision, defined as the degree to which humans agree that the explanation is a correct and reasonable rationale. In "Explain Yourself, Briefly!" (Bassan et al., 5 Feb 2025), explanations are minimal sufficient reasons: smallest feature subsets that keep the prediction unchanged under baseline, robust, or probabilistic substitutions. In LSX, explanations are useful if an internal critic can perform the same task given those explanations, which makes explanation quality explicitly task-functional rather than merely descriptive (Stammer et al., 2023).

These objectives are related but not equivalent. A concise explanation can be sufficient for another model yet omit details that humans prefer. A human-plausible explanation can be unfaithful to the model’s internal computation. A sufficient subset can be faithful in the sense of preserving the prediction while remaining opaque as natural language. This suggests that SE research is best understood as a space of partially overlapping criteria rather than a single metric family.

3. Self-explanation in LLM reasoning and dialogue

In recent LLM work, SE is most visibly instantiated as intermediate reasoning text. On ARC Challenge, seven explanation-generating models were prompted to read a science question with four answer options, produce a final answer, and generate a natural-language explanation as part of the reasoning trajectory. A fixed scorer model, Qwen3 1.7B, then received only the question, explanation, and options, and its probability of the gold option served as the sufficiency score. This yields a strict distinction between the generator’s internal reasoning artifacts and the same text re-used as an externalized explanation for another model (Zahedzadeh et al., 15 Feb 2026).

The empirical result is a broad sufficiency–conciseness plateau. Explanations can often be reduced by roughly 30–50% with only small or negligible losses in sufficiency and accuracy, whereas performance typically drops more steeply beyond about 60–70% reduction. In English, full explanations substantially exceed the scorer’s no-explanation baseline of 71.17% accuracy and 80.71% sufficiency; in Persian, where the scorer baseline is 47.85% accuracy and 72.82% sufficiency, explanations remain helpful but are less robust to compression. High embedding similarity between full and compressed explanations does not guarantee sufficiency, indicating that structural or causal details can be lost even when semantic overlap remains high (Zahedzadeh et al., 15 Feb 2026).

Dialogue work specializes SE to context understanding rather than answer derivation. In "Self-Explanation Prompting Improves Dialogue Understanding in LLMs" (Gao et al., 2023), the model is instructed to analyze a dialogue utterance by utterance, explain each utterance, and only then answer the task query. This task-agnostic prompt improved all six evaluated dialogue-centric benchmarks. On gpt-3.5-turbo, MultiWOZ 2.1 Joint Goal Accuracy rose from 35.93 with vanilla prompting to 44.44 with SE; STARv2 weighted F1 rose from 51.88 to 63.66. On MultiWOZ, a targeted ablation showed that “understand the dialogue first” barely helped, summarization improved to 40.98, and utterance-by-utterance explanation produced the strongest result at 44.44. The reported failure modes addressed by SE were time-involved slot confusion, omission of information in long dialogues, and confusion between user-provided and system-provided information (Gao et al., 2023).

A further LLM-specific line treats SE quality itself as an alignment target. "Anchored Alignment for Self-Explanations Enhancement" evaluates explanations using five criteria—logical coherence, clarity, relevance, depth of argumentation, and factual accuracy—and uses anchor preference pairs constructed from ground-truth labels to drive DPO without annotated rationales. The paper reports that task SFT improves accuracy but degrades explanation quality, whereas anchored self-alignment improves explanation quality while maintaining accuracy on AQuA-Rat, ARC-Challenge, LogiQA, and OpenbookQA (Villa-Arenas et al., 2024).

4. Self-explanation as a training signal

One major research trajectory uses explanations not as end-user artifacts but as privileged information during training. In "Introspective Learning by Distilling Knowledge from Online Self-explanation" (Gu et al., 2020), the model generates class-wise saliency maps for each training example,

$X$4

and converts similarities between the ground-truth class explanation and non-ground-truth explanations into a soft target distribution $X$5. Training then combines standard cross-entropy with a self-explanation-based soft-label loss. On CIFAR-10, this introspective learning procedure improved ResNet-14 accuracy from $X$6 under baseline cross-entropy to $X$7 with Grad-CAM-based LE, and improved ResNet-8 from $X$8 to $X$9 (Gu et al., 2020).

LSX generalizes the same idea to an explicit learner–critic loop. The learner first solves the task, then produces explanations, and an internal critic is trained or specified to assess whether those explanations are useful for the same task. The total objective takes the form

$Y$0

Across CNN, neuro-symbolic, and VLM instantiations, LSX improved generalization in low-data regimes, reduced the influence of confounding factors such as DecoyMNIST shortcuts, and produced more task-relevant and faithful explanations. For example, on DecoyMNIST with confounded training and unbiased test data, the CNN baseline achieved 63.52% while CNN-LSX achieved 78.99% (Stammer et al., 2023).

LeaSE pushes this logic into architecture search. An explainer model learns, explains via adversarial perturbation masks, an audience model learns from explanation-reweighted inputs, and the explainer’s architecture is updated based on both its own validation loss and the audience’s validation loss. This is formalized as a four-level optimization over explainer architecture $Y$1, explainer weights $Y$2, explanation masks $Y$3, and audience weights $Y$4. Applied to DARTS, P-DARTS, and PC-DARTS, LeaSE consistently improved CIFAR-10, CIFAR-100, and ImageNet performance relative to the underlying NAS baselines (Hosseini et al., 2020).

A closely related LLM-RL direction uses SE to create positive samples when standard RL post-training cannot find any. ExPO conditions the current policy on the ground-truth answer to generate self-explanations

$Y$5

and uses these trajectories inside DPO- or GRPO-style optimization. The paper argues that effective positive samples should be both likely under the current policy and increase the likelihood of the correct answer; self-explanations satisfy both properties better than expert-written CoTs, and ExPO improves learning efficiency and final performance in hard reasoning settings such as MATH level-5 (Zhou et al., 3 Jul 2025).

5. Intrinsic self-explaining architectures

A different line makes explanations part of the model’s forward computation. "SELOR" produces logic-rule explanations $Y$6 as integral mediators of prediction rather than post-hoc summaries. Human studies on Yelp and Adult reported that SELOR explanations were judged “good” at rates of 94.4% and 90.7%, respectively, and “best” at rates of 46.7% and 65.1%, while maintaining predictive performance close to the base deep models (Lee et al., 2022).

"Explain Yourself, Briefly!" replaces free-form rationales with sufficient feature subsets. A classifier is trained to output both a prediction and an explanation mask, and a second forward pass enforces that the selected subset remains sufficient under baseline, probabilistic, or robust perturbations. The total loss is

$Y$7

so prediction quality, sufficiency, and cardinality are optimized jointly. The paper also proves that obtaining cardinally minimal sufficient reasons is NP-Complete for baseline sufficient reasons, $Y$8-Complete for robust sufficient reasons, and $Y$9-Hard for probabilistic sufficient reasons, which motivates training-time approximation rather than exact post-hoc search (Bassan et al., 5 Feb 2025).

Graph models instantiate yet another intrinsic SE design. X-Node equips each node with a structured context vector

$\max_Z \; I(Z; Y) - \beta I(X; Z).$0

a Reasoner that maps this interpretable context to a compact explanation vector $\max_Z \; I(Z; Y) - \beta I(X; Z).$1, a decoder that reconstructs the latent node embedding, and a classifier that consumes both the GNN embedding and $\max_Z \; I(Z; Y) - \beta I(X; Z).$2. The joint loss combines classification, alignment, and reconstruction terms. On OrganAMNIST with a GCN backbone, X-Node improved accuracy from 91.85 to 93.64 and F1 from 91.19 to 93.16 while producing faithful per-node explanations (Sengupta et al., 14 Aug 2025).

These architectures share a strong anti-post-hoc commitment. Explanations are not auxiliary reports added after training; they are computational intermediates that predictions depend on, or that are explicitly reconstructed from, aligned with, or used to regularize the model’s latent state.

6. Educational and human-learning perspectives

The term SE originates in the learning sciences, where it names a constructive activity through which learners explain material to themselves, integrate new information with prior knowledge, fill in missing steps, and monitor understanding. Recent scoping reviews in postsecondary mathematics and statistics and in middle and secondary mathematics both report that SE is typically implemented as written, oral, or gestural explanation, is distinct from peer explanation and explanation to fictitious others, and tends to benefit conceptual understanding, proof comprehension, and transfer when prompts are well designed (Gao et al., 25 Mar 2025). The school-level review emphasizes that SE is particularly effective when combined with worked examples and that prompt design spans open-ended, focused, scaffolded, resource-based, and menu-based forms (Gao et al., 20 Aug 2025).

A contemporary calculus experiment adds LLM-supported feedback to this tradition. In a between-subjects study with $\max_Z \; I(Z; Y) - \beta I(X; Z).$3, open-ended SE with GPT-5.1 feedback, menu-based SE, and a no-SE control all produced positive learning gains during a fixed 60-minute session, but no significant between-condition differences in post-test performance. The open-ended condition nevertheless produced significantly higher-quality explanations than control on Not Enough Information transfer problems, with $\max_Z \; I(Z; Y) - \beta I(X; Z).$4 percentage points and $\max_Z \; I(Z; Y) - \beta I(X; Z).$5, despite completing only 16.9 practice problems on average versus 58.9 in the control condition (Chen et al., 31 Mar 2026).

Game-based implementations show that SE quality is sensitive to interface design. In iSTART, Extended Practice uses automated word-based and LSA-based assessment of student self-explanations, whereas MiBoard, a multiplayer board-game reformulation, reduced engagement and did not benefit the quality of self-explanations relative to the original module. Its redesign, Self-Explanation Showdown, simplified the interaction, used the same 0–3 quality scoring algorithms from iSTART, and tied competition directly to explanation quality rather than peer strategy identification (Brunelle et al., 2010).

These educational results clarify a recurrent theme in machine SE: explanations are beneficial when they are generative, targeted, and functionally coupled to feedback. They are less effective when the format is overly constrained, when explanation quality is not itself assessed, or when the surrounding task structure turns explanation into a distracting meta-task.

7. Agentic, social, and embodied self-explanation

In agentic settings, SE frequently becomes a mechanism for coordination or policy shaping rather than verbal rationalization. Instruction-based Behavior Explanation (IBE) lets an RL agent explain its own future behavior by reusing human instruction signals acquired during interactive RL. The agent predicts future state change $\max_Z \; I(Z; Y) - \beta I(X; Z).$6, clusters these changes, and maps clusters to normalized instruction-like explanation signals. Human studies in Lunar Lander showed that such explanations could improve prediction of where the rocket would land, while also revealing that poor clustering can mislead users (Fukuchi et al., 2018).

SERLfD addresses ambiguous demonstrations by learning utility weights over symbolic predicates,

$\max_Z \; I(Z; Y) - \beta I(X; Z).$7

and using these utilities for both state augmentation and potential-based reward shaping,

$\max_Z \; I(Z; Y) - \beta I(X; Z).$8

Here the self-explanation is a state-dependent weighting of relational predicates such as object–location or ghost–capsule relations. Across Robot-Push, Robot-Remove-and-Push, and Pacman variants, the framework improved training stability and performance relative to RLfD baselines by using success–failure discrimination to identify which predicates actually explain success (Zha et al., 2021).

Social agents add an explicitly introspective variant. In "Self-Explanation in Social AI Agents" (Basappa et al., 19 Jan 2025), a social AI assistant answers natural-language questions about its own operation by reasoning over a Task–Method–Knowledge self-model. The pipeline classifies the question, localizes relevant parts of the semantic TMK model with FAISS, and uses Chain of Thought plus ChatGPT to generate an explanation. On 66 evaluated questions, 49 explanations were correct and 48 were complete, and deployment in two live online classes produced 20 questions from 11 students, with several students reporting that the answers were clear and improved their understanding of the agent (Basappa et al., 19 Jan 2025).

These agentic systems expose a final fault line in the SE literature. Some explanations are primarily for humans coordinating with an agent; others are internal scaffolds for improving the agent’s own learning. That distinction sharpens current open problems. Faithfulness remains unresolved when explanations are judged only by downstream sufficiency or by LLM-based evaluators. Length is an imperfect proxy for information content. Resource-limited languages, as shown by the English–Persian ARC study, can alter both explanation quality and compression robustness (Zahedzadeh et al., 15 Feb 2026). Many systems depend on hand-specified predicates, concept vocabularies, or self-models, which raises scalability questions. The field therefore increasingly treats SE not as a single interpretability primitive but as a design space defined by four recurrent tensions: human-facing plausibility versus model-facing sufficiency, verbosity versus compression, post-hoc rationalization versus forward-pass integration, and domain-general abstraction versus domain-specific structure.