Natural Language Explanations (NLEs)

• Natural Language Explanations (NLEs) are human-readable justifications that elucidate model decision-making by ensuring both plausibility and faithful representation of internal logic.
• They integrate explanation generation with prediction using methods like joint generative models and rationale extraction to align outputs closely with model reasoning.
• Evaluation frameworks use metrics such as attribution-similarity, sufficiency, and causal faithfulness to verify that explanations accurately reflect decision mechanisms in diverse applications.

Natural Language Explanations (NLEs) are structured, human-readable justifications provided by models to elucidate their decision-making process in natural language. Unlike feature importance scores or visual attribution maps, NLEs offer comprehensive textual accounts that can articulate model reasoning at a level readily accessible to humans with varying degrees of technical background. As NLEs have rapidly gained prominence in fields such as natural language understanding, vision-language tasks, and high-stakes applications like medicine, their generation, evaluation, and use as both a transparency and robustness tool have become central areas of research attention.

1. Foundations and Methodological Principles

NLEs are typically generated either during the model’s prediction process (intrinsic/integrated explanations) or as a post-hoc justification (extrinsic/post-hoc explanations) for a given output. The foundational principle behind NLEs is that explanations should be both plausible—convincing and intelligible to humans—and, critically, faithful, i.e., reflective of the true internal decision-making of the model.

Early work demonstrated the potential of collecting human-written rationales and explanations (e.g., as in e-SNLI), leading to supervised learning frameworks in which models are prompted to generate explanations in natural language conditioned on input features and predicted outputs. Subsequent approaches have sought to bridge gaps in faithfulness, reduce the prevalence of spurious or misleading explanations, and address variability in human explanation styles.

Recent methodologies often employ architectures that tightly couple prediction and explanation. For example, NLX-GPT and Uni-NLX leverage unified sequence generation frameworks to jointly output a prediction and an explanation conditioned on a shared embedding of the input and (optionally) multimodal features. Others, such as LIREx (2012.09157), introduce rationale extraction modules to highlight the tokens or subspaces most responsible for a decision, and instance selectors to ensure only the most plausible explanations guide the final label prediction.

Key methodological steps in modern NLE pipelines include:

• Extraction or identification of salient input features (rationales) that underpin the model’s decision.
• Conditioning the explanation generator on these rationales and/or the model’s internal representations (as opposed to the output label alone).
• Employing robust selector or filtering mechanisms to select explanations that are both plausible and faithful.

2. Evaluation Frameworks and Faithfulness Metrics

Evaluating NLEs involves multiple, often complementary, dimensions. Early evaluation relied heavily on standard natural language generation metrics (BLEU, METEOR, ROUGE, CIDEr). However, it was soon recognized that such metrics do not reliably capture either the semantic adequacy or the alignment with model reasoning.

Recent research has advanced the development of targeted faithfulness metrics:

• Attribution-Similarity: The cosine similarity between model attribution vectors for answers and explanations (e.g., calculated using Integrated Gradients). This quantifies whether the explanation “attends” to the same features as the model’s answer function (2304.08174).
• NLE-Sufficiency and NLE-Comprehensiveness: Derived from perturbation analysis, these metrics quantify the sufficiency (does using only the explanation-supporting features suffice to yield the same decision confidence?) and comprehensiveness (does removing these features cause confidence to drop?) of explanations.
• Counterfactual and Reconstruction Tests: Introduced to detect whether NLEs mention the input changes responsible for a counterfactual output, or if the reasons given in explanations can reproduce the model’s label in isolation (2305.18029).
• Causal Faithfulness via Activation Patching: This recent metric uses activation patching to intervene in a model’s hidden states at the token and layer level, measuring the causal impact of explanation-supporting components on the final output. The Causal Faithfulness (CaF) score is defined as:

$\text{CaF} = 1 - \mathrm{CD}(C_e, C_a)$

where $C_e$ and $C_a$ are aggregated causal attribution vectors for explanation and answer, and $\mathrm{CD}$ denotes cosine distance (2410.14155).

There is strong evidence that no single metric suffices: correlations between plausibility, sufficiency, comprehensiveness, and attribution-similarity are modest; comprehensive evaluation requires applying several metrics in parallel.

3. Model Architectures and Generation Strategies

Architectures for NLE generation have evolved from two-stage pipelines (separate prediction and explanation models) to unified and multi-task architectures:

• Joint Generative Models: Approaches such as NLX-GPT (2203.05081) and Uni-NLX (2308.09033) treat the task as joint sequence generation, improving parameter and runtime efficiency.
• Rationale-enabled Pipelines: LIREx (2012.09157), RExC (2106.13876), and similar frameworks explicitly extract or select rationales before generating an NLE conditioned on these. This effectively grounds the explanation in features proven to influence the output, increasing alignment with the model’s reasoning chain.
• Knowledge Augmentation: For domains where textual evidence is insufficient (e.g., medicine), retrieval-augmented methods incorporate external knowledge graphs. For instance, KG-LLaVA integrates a medical KG datastore and fuses relevant knowledge triplets with visual features to enhance explanations for thoracic pathologies (2410.04749).
• Post-hoc Pipeline Methods: In vision, some pipelines first construct structured representations reflecting the model’s internal logic (e.g., neuron importances from LRP), then translate these into natural language with LLMs (2407.20899).

Iterative refinement methods have recently been introduced to increase NLE faithfulness by exploiting self-critique (SR-NLE (2505.22823)), external critic-based feedback (Cross-Refine (2409.07123)), or both. Such cycles can substantially reduce unfaithfulness rates over simple single-stage generation.

4. Data Resources and Transferability

Several large-scale datasets have been developed to support NLE research, including:

• e-SNLI: Natural language explanations for NLI.
• e-SNLI-VE, VQA-X, VCR: Vision-language explanation datasets, supporting cross-modal research and benchmarking (2105.03761).
• MIMIC-NLE: Explanations for chest X-ray findings, covering ten thoracic pathologies with over 38,000 high-quality NLEs (2207.04343).

Transfer learning of NLE capability is a major concern given the annotation cost. Few-shot transfer regimes show that explanation-generation can be transferred from parent to child tasks, especially when abundant label data coexists with scarce NLEs (2112.06204). Prompting strategies and parameter-efficient fine-tuning (e.g., SparseFit (2305.13235)) have further reduced the resource requirements for deployment in new domains.

Efficient synthetic data generation pipelines leveraging large vision-LLMs and advanced prompting now enable the scalable creation of VQA-NLE datasets with minimal quality loss and greatly increased annotation speed (2409.14785).

5. Impact, Applications, and Challenges

The interpretability enabled by NLEs has led to notable applications:

• Clinical AI: In medical imaging, NLEs supplement image-based diagnoses with textual rationales that improve clinician trust, transparency, and regulatory compliance (2207.04343, 2410.04749). Knowledge augmentation (via domain KGs) proves particularly effective for precise and privacy-preserving explanations.
• Vision-Language Systems: In VQA, visual recognition, and entailment, NLEs support both user-facing interpretability and system debugging.
• Information Retrieval: As part of ranking pipelines, NLEs have been used for calibration, making ranker output scores more informative and trustworthy via LLM-generated explanations (2402.12276).

Challenges persist. Many models generate explanations that, while plausible and fluent, are not truly faithful to their decision logic. Spurious explanations can arise due to over-reliance on label priors or superficial patterns. Systems can also fail when explanations do not reference features responsible for label changes under counterfactual interventions (2305.18029). Ensuring that explanations are robust, resistant to adversarial inputs, and tailored to diverse user contexts (situated NLEs (2308.14115)) are ongoing research directions.

6. Advances in Evaluation, Faithfulness, and Robustness

Recent studies highlight several critical findings:

• Iterative Refinement and Critique: Self-critique and external-critic-based refinement cycles can substantially improve the faithfulness of post-hoc NLEs, with the best methods achieving absolute reductions in unfaithfulness rates approaching 19% compared to simple generation (2505.22823, 2409.07123).
• Causal Mediation-Based Faithfulness: Metrics based on activation patching (i.e., direct intervention in model hidden states) provide strong theoretical and empirical grounding for causal faithfulness evaluation, overcoming issues in earlier attribution or perturbation-based metrics. Alignment-tuned models (“chat” models) tend to yield more faithful explanations (2410.14155).
• Role of Model Size vs. Reasoning Ability: In time series forecasting, explanation quality correlates less with model size than with numerical reasoning ability, emphasizing the importance of foundation model pretraining strategies (2410.14180).
• Synthetic Data Augmentation: Prompted LVLMs now produce high-quality, relevant NLE datasets for VQA and related settings, achieving 20x speedups over manual annotation with minimal qualitative performance drop (2409.14785).

7. Future Directions

Major avenues for continued research into NLEs include:

• Integration of Mechanistic Interpretability: Combining neuron- or circuit-level interpretability with NLE frameworks to further bridge the gap between internal computation and surface-level explanation (2410.14155).
• Adaptive, Audience-aware Explanations: Developing situated NLEs that optimize length, complexity, and style for different user roles or decision contexts (2308.14115).
• Faithfulness as a Training Objective: Directly incorporating faithfulness-based loss or constraint terms (e.g., attribution-similarity, causal alignment) during model optimization to ensure explanations are tightly coupled to model reasoning.
• Scalable Evaluation Methodologies: Formalizing simulatability and other user-centered evaluation strategies for NLEs, particularly in domains where ground truth is ambiguous or domain knowledge is required (2410.14180).
• Extending to Multi-Modal and Domain-Specific Tasks: Expanding NLE methods to new application domains (e.g., legal reasoning, finance) and additional modalities.

NLEs continue to evolve as a core technology for interpretable, trustworthy AI, with research focusing on reconciling human expectations for plausible explanations with the imperative for faithfulness to model internals. Advances in architecture, evaluation, and synthetic data generation underpin their growing practical deployment in real-world systems.