Highlight Explanations (HEs)

• Highlight Explanations (HEs) are artifacts that mark critical evidence—such as tokens, phrases, spatial regions, or state-action pairs—used by models to generate outputs.
• HEs vary by domain and are implemented via methods like post hoc attribution or training modifications to selectively expose explanatory units and improve interpretability.
• Evaluation frameworks for HEs focus on measuring faithfulness, context reliance, and robustness, addressing challenges like positional bias and encoding of spurious information.

Highlight explanations (HEs) are explanatory artifacts that mark the specific evidence a model or agent relied on when producing an output, prediction, or action. In current research, the highlighted unit varies by domain: tokens or passages in NLP, phrase pairs across aligned texts, span interactions across input segments, spatial locations in images, decisive states in reinforcement learning, salient feature parts in case-based image explanations, and compact path summaries in graph recommenders. Across these settings, HEs are used to answer closely related questions: whether a system relied on provided context rather than parametric memory, which exact inputs were causally or operationally relevant, and whether the resulting explanation is faithful, useful, and resistant to spurious encoding (Sun et al., 3 Oct 2025, Li et al., 2023, Draelos et al., 2020).

1. Conceptual scope and terminology

The modern literature uses “highlight explanation” in at least two closely related senses. In text and multimodal prediction, HEs usually denote feature-attribution-style explanations that highlight important input fragments for a prediction; examples include important tokens in text classification and highlighted context pieces for language-model responses (Li et al., 2023, Sun et al., 3 Oct 2025). In reinforcement learning, “HIGHLIGHTS” is also the name of a global explanation method that presents policy summaries consisting of selected decisive states rather than explanations for a single action (Septon et al., 2022).

In LLMs, HEs are motivated by the opacity of context utilisation: users often cannot tell whether a response came from parametric knowledge or from the provided context, nor which specific context pieces informed the answer. The stated appeal of HEs is that they can point the exact context pieces and tokens that influenced model outputs (Sun et al., 3 Oct 2025). In text classification, the same family of methods is described as highlighting important input tokens as explanations to model predictions (Li et al., 2023).

The RL variant formalizes HIGHLIGHTS as a policy summarization method. If $T = \langle t_1, \ldots, t_k \rangle$ is the summary, each $t_i$ is a sequence of $l$ consecutive state-action pairs,

$$t_i = \langle (s_{i,1}, a_{i,1}), \ldots, (s_{i,l}, a_{i,l}) \rangle.$$

State importance is environment-specific. In the Highway environment,

$I(s) = \max_a Q^\pi(s,a) - \min_a Q^\pi(s,a),$

whereas in Pacman,

$$I(s) = \max_a Q^\pi(s,a) - \operatorname{second\text{-}highest}_a Q^\pi(s,a).$$

The resulting summary is presented as video clips or static images centered on the selected important states (Septon et al., 2022).

Taken together, these works indicate that HEs are not a single algorithmic object but a representational family whose common property is selective exposure of evidence. A plausible implication is that any general account of HEs must distinguish the highlighted unit, the inference target, and the intended explanatory granularity.

2. Representational units and structural variants

A major axis of variation is the unit being highlighted. The most basic form is the individual token. Token-level HEs are central in text classification and in context attribution for LLMs, where evaluation can ask whether the explanation identifies the exact answer token or tokens used by the model (Li et al., 2023, Sun et al., 3 Oct 2025).

Several papers argue that token-wise saliency is often structurally inadequate. For text-pair tasks, “contrastive phrasal highlighting” replaces isolated tokens with aligned phrase pairs across two texts and asks which minimal contrasting phrase pairs explain a predicted semantic divergence. Given a sentence pair $S = (x,y)$, aligned phrase pairs $p = (p_x,p_y)$ are erased from both sides; phrase pairs are retained when

$P' = \{p \in P : R(S_{-p}) > R(S) + \epsilon\},$

and brevity is rewarded by maximizing

$\arg\max_{p \in P'} R(S_{-p}) \cdot BR(S,p).$

This formulation makes the explanatory object the contrast itself, rather than independently salient tokens on each side (Briakou et al., 2023).

For NLI and fact-checking, “span interaction explanations” move one step further by emphasizing interactions between semantically coherent spans in different parts of the input. SpanEx provides 7,071 human-annotated instances over NLI and fact-checking, with span boundaries at high and low levels and interaction labels including Synonym, Antonym, Hypernym-P2-P1, Hypernym-P1-P2, and Dangler (Choudhury et al., 2023). This reframes HEs as relational structures rather than flat saliency masks.

In image explanation, the highlighted unit is spatial. HiResCAM highlights locations in a convolutional representation that contribute to a class score by

$t_i$0

For CNNs ending in a single fully connected layer, this is guaranteed to reflect the intermediate computations contributing to the output score (Draelos et al., 2020). Case-based image explanation extends the unit from a spatial region in a test image to matched feature parts across test and training images. One latent-space formulation selects corresponding regions by

$t_i$1

while a superpixel variant uses analogous matching in image space (Kenny et al., 2023).

Graph-guided textual explanation generation introduces three HE types for encoder-decoder models: highlight token explanations, token interactive explanations, and span interactive explanations. Tokens are graph nodes, and edges connect top-$t_i$2 important tokens, important token pairs, or tokens within selected span interactions, after which a GNN layer encodes the graph to guide NLE generation (Yuan et al., 2024). In graph recommenders, the highlighted object becomes a subgraph summarizing recurrent explanation paths over a recommendation set rather than a single instance (Karidi et al., 2024).

3. Generation mechanisms

One prominent class of HE methods is post hoc attribution. For context attribution in LLMs, a recent framework evaluates Feature Ablation (FA), Integrated Gradients (IG), Attention visualization (ATTN), and MechLight, a mechanistic interpretability approach adapted to produce user-readable HEs by selecting attention heads most responsible for context-use (Sun et al., 3 Oct 2025). In text classification, IG is also used inside REGEX to score tokens: $t_i$3 REGEX then masks low-ranked tokens and aligns model attention with feature attribution using KL divergence (Li et al., 2023).

Another class modifies model training to improve the quality of downstream HEs. REGEX combines classification loss with input gradient regularization,

$t_i$4

virtual adversarial training,

$t_i$5

and an attribution-attention alignment term inside the overall objective

$t_i$6

This is explicitly intended to produce more faithful explanations rather than merely post hoc scores (Li et al., 2023).

Structured extraction methods are increasingly common when independent token saliency is insufficient. Contrastive phrasal highlighting uses phrase-alignment-guided erasure to identify phrase pairs whose removal makes two texts more equivalent (Briakou et al., 2023). Span-interaction extraction builds a bipartite directed graph from cross-part attention and applies Louvain community detection, using modularity gain

$t_i$7

to identify communities that induce candidate span-pair explanations (Choudhury et al., 2023).

In vision, HiResCAM is presented as a correction to Grad-CAM’s gradient averaging, which can blur location-wise contributions and highlight regions not actually used by the model (Draelos et al., 2020). In case-based explanation, latent and superpixel algorithms isolate multiple clear feature parts and connect them to corresponding parts in relevant training examples, but only if those regions are themselves salient for the model (Kenny et al., 2023).

4. Evaluation frameworks and metrics

A defining problem in HE research is that usefulness, plausibility, faithfulness, and transparency do not coincide. For context attribution in LLMs, the first gold standard HE evaluation framework uses controlled test cases with known ground-truth context usage, thereby avoiding indirect proxy evaluations (Sun et al., 3 Oct 2025). It evaluates four context scenarios: Conflicting, Irrelevant, Double-Conflicting, and Mixed; in dual-context settings, context order is swapped to control for order effects. The framework asks three questions: whether the HE reveals reliance on context knowledge or parametric knowledge, which document was used, and whether the exact answer token or tokens can be pinpointed (Sun et al., 3 Oct 2025).

Its metrics include rank-based and information-theoretic quantities. Rank Margin is defined as

$t_i$8

Normalized Mutual Information at $t_i$9 is

$l$0

and token recovery is measured with Mean Reciprocal Rank,

$l$1

The same paper also reports AOPC-based comprehensiveness and sufficiency for completeness (Sun et al., 3 Oct 2025).

A different evaluation tradition measures whether explanations teach a student model to simulate a teacher. In the student–teacher framework, the student sees teacher explanations during training but not at test time, and explanation quality is operationalized as the simulation gain

$l$2

where

$l$3

This design is explicitly intended to reduce gaming by preventing test-time label leakage through the explanation channel (Pruthi et al., 2020).

Text-classification work continues to rely on sufficiency and comprehensiveness. REGEX defines normalized sufficiency and comprehensiveness using the effect of retaining or removing highlighted rationales from an input, and further audits robustness with randomization tests such as the Initialization Invariance Test and the Untrained Model Test (Li et al., 2023). By contrast, the encoding-explanations line argues that predictive-retention scores alone can be misleading. An explanation is encoding if the label remains conditionally dependent on which subset was selected even after conditioning on the values of the selected subset,

$l$4

Its “encode-meter” is

$l$5

and STRIPE-X penalizes encoded information via

$l$6

The paper proves that ROAR and FRESH are not detectors of encoding, EVAL-X is only a weak detector, and STRIPE-X is both a weak and strong detector for sufficiently large $l$7 (Puli et al., 2024).

5. Empirical findings across domains

In context attribution for LLMs, MechLight performs best across all four context scenarios, four datasets, and five LLMs, while FA is generally second best, especially in short contexts (Sun et al., 3 Oct 2025). IG and attention visualization often fail to reveal context usage accurately and frequently match random or trivial baselines, particularly for distinguishing context-based from memory-based answers. The same evaluation reports that all methods struggle on longer contexts and exhibit positional biases: FA and IG tend to prefer later tokens, while ATTN and MechLight often overemphasize earlier context passages (Sun et al., 3 Oct 2025).

For text classification, REGEX improves faithfulness in all reported settings across six datasets and five attribution methods, including out-of-domain evaluation. It also increases DIT and UIT Jaccard similarities, indicating more consistent rationales under re-initialization and between trained and untrained models. In downstream use, select-then-predict models trained on REGEX-produced rationales can achieve task performance comparable to end-to-end models using full text (Li et al., 2023).

In visual explanation, HiResCAM is reported to be exactly correct in controlled CNN settings where the true contributing locations can be computed, while Grad-CAM remains nonzero in L2 distance because its spatial averaging highlights irrelevant locations (Draelos et al., 2020). The paper additionally argues that object-overlap metrics such as IoU should not be treated as explanation-quality measures, because explanation aims to show what the model used, not what it ideally should have used.

For text-pair comparison, contrastive phrasal highlighting matches human rationales of cross-lingual semantic differences better than LIME and SHAP and helps annotators detect fine-grained meaning differences and critical machine-translation errors (Briakou et al., 2023). In human studies, access to highlights increases precision, recall, and F1 for divergence detection, raises Cohen’s Kappa from 0.51 to 0.66 for English–French and from 0.33 to 0.52 for English–Spanish, and is especially helpful for nuanced local errors such as negation flips (Briakou et al., 2023).

In reinforcement learning, the integration of HIGHLIGHTS with reward decomposition yields mixed but informative results. Reward decomposition is generally more useful for identifying an agent’s priorities, while HIGHLIGHTS improves participant understanding in Pacman and in subtle cases where agents’ preferences differ only slightly (Septon et al., 2022). In explanation-guided NLE generation, G-Tex uses HEs as graph-structured faithful cues and improves NLE faithfulness by up to 12.18% relative to baseline methods, while also improving lexical and semantic similarity to human explanations and reducing redundant content in human evaluation (Yuan et al., 2024).

6. Limitations, misconceptions, and open directions

A recurring misconception is that a visually plausible or semantically appealing highlight is therefore faithful. The recent HE literature repeatedly rejects this equivalence. In context attribution, widely used methods can appear informative yet fail to identify whether the model relied on context or memory (Sun et al., 3 Oct 2025). In vision, smoother or larger heatmaps can look better to humans while being less faithful to the model’s actual computation (Draelos et al., 2020). In explanation evaluation, predictive power can itself be misleading when the explanation encodes information in the act of selection rather than in the values shown to the user (Puli et al., 2024).

Another limitation concerns ecological validity. LiveNLI shows that people may choose the same NLI label for different reasons, and that explanations are therefore crucial for navigating label interpretations in general (Jiang et al., 2023). The dataset contains 1,415 explanations for 122 MNLI items, each with at least 10 annotations, and highlights that within-label variation can remain hidden even when annotators agree on the label. A plausible implication is that HE benchmarks tied to a single canonical rationale may underrepresent genuine interpretive diversity.

Current frontier work therefore pushes HEs in two directions. One is greater structural expressivity: phrase pairs, span interactions, path summaries, and hierarchical, module-specific explainers extend beyond token-wise saliency and single-instance explanations (Briakou et al., 2023, Choudhury et al., 2023, Karidi et al., 2024, Love et al., 6 Jan 2026). The other is stricter evaluation: gold-standard context-attribution benchmarks, robustness tests, student–teacher simulation, and encoding-aware scores attempt to distinguish true explanatory content from plausible but non-transparent artifacts (Sun et al., 3 Oct 2025, Pruthi et al., 2020, Puli et al., 2024).

The immediate research agenda stated in the literature is correspondingly concrete. For context-utilisation HEs, future work is directed toward methods that explicitly counter position bias and length sensitivity and toward extensions to more complex tasks such as multi-hop QA, summarization, open-domain QA, and human-centered evaluations (Sun et al., 3 Oct 2025). More broadly, the field increasingly treats HEs not as sufficient explanations in themselves, but as auditable evidence carriers that can anchor richer explanation systems, including graph-guided NLE generation and modular explainability architectures (Yuan et al., 2024, Love et al., 6 Jan 2026).