---
title: 'SALU: Self-Aware LLM for Unanswerability'
url: https://www.emergentmind.com/topics/self-aware-llm-for-unanswerability-salu
type: topic
---

# SALU: Self-Aware LLM for Unanswerability

Searching arXiv for recent SALU and related unanswerability/self-awareness papers to ground the article in the current literature.
Self-Aware LLM for Unanswerability (SALU) denotes a class of large language model systems designed to determine when a query should not be answered definitively and to abstain, refuse, or request clarification instead of generating a hallucinated response. In the literature, this capability is framed as a form of self-knowledge: the ability to distinguish questions a model can answer from questions it cannot, including cases where the input is underspecified, subjective, based on a false premise, or otherwise lacks a determinate answer [2305.18153]. SALU also appears as a concrete end-to-end architecture for conversational information retrieval, where unanswerability detection is integrated into the generative process and refined with confidence-score-guided reinforcement learning with human feedback (RLHF) [2507.16951]. Recent work extends the same objective to latent-state probing, activation-space steering, text-to-SQL refusal, and explicit uncertainty attribution, showing that reliable abstention is related to, but not reducible to, raw answer accuracy [2509.22449][2601.10398][2604.17293][2506.09038].

## 1. Conceptual foundations

The foundational notion behind SALU is that an LLM should distinguish “Known Knows” from “Known Unknows” and refrain from spurious answers when it lacks sufficient information [2305.18153]. In that formulation, self-knowledge is operationalized as the $F_1$ score obtained when unanswerable questions are treated as the positive class. This places abstention in a standard detection framework rather than treating refusal as a purely stylistic generation behavior.

Subsequent work generalizes this idea into answerability gating. In text-to-SQL, safe refusal is formalized as a binary gating problem over a user query–schema prompt $x$, where a frozen LLM $\mathcal M$ produces hidden states $\mathbf H^{(\ell)} \in \mathbb R^{T \times d}$ and a lightweight classifier outputs $\hat p = f_{\rm gate}(h)$; the system refuses whenever $\hat p < \tau$ [2601.10398]. In extractive QA, the problem is framed as determining whether a passage contains sufficient information for answering a question, with abstention preferred when the evidence is absent [2509.22449].

A further conceptual refinement distinguishes *data uncertainty* from *model uncertainty*. Data uncertainty arises when the question itself is ambiguous, underspecified, or lacks key facts; model uncertainty arises when the question is well-posed but the model cannot answer it with its current parametric knowledge or reasoning capacity [2604.17293]. This distinction matters operationally because the appropriate downstream action differs: a data-uncertain query calls for clarification or additional context, whereas a model-uncertain query may call for retrieval, a calculator, or another external tool.

The unanswerability space described in the literature is broad. The SelfAware benchmark defines five categories—No Scientific Consensus, Imagination, Completely Subjective, Too Many Variables, and Philosophical [2305.18153]. AbstentionBench organizes 20 datasets into six scenarios: Answer Unknown, False Premise, Stale, Subjective, Underspecified Context, and Underspecified Intent [2506.09038]. Other taxonomies emphasize Incomplete, Futuristic, Incorrect, and Ambiguous unknown questions [2402.15062], or functional, contextual, ethical, temporal, and ambiguity-related feasibility boundaries [2503.11256]. Taken together, these taxonomies show that SALU is not limited to “missing evidence” in the narrow extractive-QA sense.

## 2. Detection signals and latent representations

Early SALU-style systems combine textual and probabilistic indicators of uncertainty. One pipeline assembles a reference set $U$ of sentences that explicitly express uncertainty, splits a model response into overlapping windows, computes embedding similarity to each $u \in U$, and marks the answer as uncertain if any similarity exceeds a threshold; in the same framework, Shannon entropy over the next-token distribution is used as a second signal, and the two are combined with a logical OR rule [2305.18153]. A related abstention framework uses post-hoc uncertainty measures such as Negative Log-Likelihood, Predictive Entropy, Semantic Entropy, and a verbalized measure called In-Dialogue Uncertainty (InDU), which counts hedge words in the raw response text [2404.10960]. These methods require no model finetuning and operate as inference-time filters.

A more mechanistic line of work seeks unanswerability signals directly in hidden activations. In the linear-direction approach, candidate vectors are defined by a difference-in-means objective over answerable and unanswerable examples:
$$
v_{\ell,p}
=
\frac{1}{M}\sum_{j=1}^{M} h_j^{(\ell,p)}
-
\frac{1}{N}\sum_{i=1}^{N} h_i^{(\ell,p)}.
$$
Candidates are selected by activation-addition steering on a held-out validation set, and the chosen vector is normalized to a direction $d$ [2509.22449]. A new example is then scored by projection,
$$
\phi_{\text{unans}}(h)=\langle h,d\rangle,
$$
and thresholded for classification. The same work reports that adding or ablating the direction causally controls abstention behavior, with $r(+2) \approx 96$–$97\%$ abstention and $r(-2) \approx 2\%$ on answerable inputs and $\approx 19\%$ on unanswerable inputs [2509.22449].

Latent probing is also central to refusal in text-to-SQL. LatentRefusal reads refusal cues from $\mathbf H^{(\ell^\star)}$, applies layer normalization and mean pooling, and feeds the result into the Tri-Residual Gated Encoder (TRGE), whose block combines self-attention, MLP, and SwiGLU branches with learned scalar gates $\alpha_i$ [2601.10398]. The stated role of the SwiGLU branch is to suppress high-frequency “schema boilerplate” and amplify sparse mismatch cues such as references to non-existent columns [2601.10398]. This is a latent-signal refusal mechanism rather than an output-level instruction-following method.

Recent work also questions whether apparent self-awareness is genuinely model-side. The Approximate Question-side Effect (AQE) estimates how much hallucination prediction can be explained using only question-side information, and Semantic Compression by Answering in One word (SCAO) is proposed to amplify model-side signals by reducing open-ended generation to a one-word answer prompt [2509.15339]. This introduces a methodological caution: strong unanswerability prediction can arise from shortcuts in question phrasing or dataset artifacts rather than introspective access to the model’s own knowledge state.

## 3. Architectural realizations

SALU appears in the literature in several architectural forms, ranging from post-hoc filters to end-to-end generative systems. The principal variants are summarized below.

| Paradigm | Core mechanism | Representative setting |
|---|---|---|
| Prompt- or uncertainty-based SALU | Similarity, entropy, NLL, PE, SE, or InDU thresholding | General QA abstention without finetuning [2305.18153][2404.10960] |
| Latent-signal SALU | Hidden-state projection, linear directions, or lightweight probes | Extractive QA and text-to-SQL gating [2509.22449][2601.10398] |
| Integrated generative SALU | Joint answer/refusal modeling with SFT and RLHF | Conversational IR and hierarchical answerability [2507.16951][2506.01104] |

In the explicit SALU architecture for conversational information retrieval, the model is a pre-trained decoder-only transformer parameterized by $\theta$ that consumes a sequence
$$
X = [\mathrm{CLS}]\, C\, [\mathrm{SEP}]\, q_t\, [\mathrm{SEP}]\, P_{(\mathrm{retrieved})}\, [\mathrm{SEP}],
$$
where $C$ is conversational history, $q_t$ is the current query, and $P_{(\mathrm{retrieved})}$ are retrieved passages [2507.16951]. Multi-task supervised fine-tuning uses separate negative log-likelihood losses for answerable QA and unanswerable abstention,
$$
L_{SFT}(\theta)=\alpha L_{QA}(\theta)+\beta L_{NA}(\theta),
$$
followed by RLHF. The RLHF stage includes a reward model trained on pairwise preferences and a PPO objective with an intrinsic confidence score
$$
S(Y|X)=\frac{1}{m}\sum_{j=1}^{m}\log P(y_j|X,y_{<j};\theta),
$$
so that correct abstentions receive an extra bonus proportional to confidence and high-confidence hallucinations receive heavier penalties [2507.16951].

A related integrated design is Reinforced Unanswerability Learning (RUL). Here a pre-trained sequence-to-sequence LLM $\mathcal M$ is augmented with a discriminative unanswerability head on top of the $[\mathrm{CLS}]$ or pooled embedding and a hierarchical attention mechanism that aggregates answerability from sentence to paragraph to ranking level [2506.01104]. Stage 1 optimizes
$$
L_{SFT}=\lambda_{cls}L_{cls}+\lambda_{gen}L_{gen},
$$
where $L_{cls}$ is binary cross-entropy for answerability prediction and $L_{gen}$ is negative log-likelihood for answer or refusal generation; Stage 2 introduces a reward model and PPO with KL regularization [2506.01104].

Other SALU-like systems emphasize data generation and explanation quality rather than latent gating. Self-Align uses a two-stage class-aware self-augmentation pipeline to synthesize unknown question–response pairs, disparity-driven self-curation with a threshold $\epsilon=80$, and supervised finetuning on curated refusals with explanations [2402.15062]. This line of work is motivated by the claim that an LLM should not only refuse but also explain why a question is unanswerable.

## 4. Benchmarks and evaluation protocols

Benchmark construction has become a central component of SALU research because abstention quality is highly sensitive to the type of unanswerability being measured. The SelfAware dataset contains 3,369 questions, including 1,032 unanswerable and 2,337 answerable items, with unanswerable instances drawn from five categories and answerable counterparts matched from SQuAD, HotpotQA, and TriviaQA via SimCSE nearest-neighbor matching [2305.18153]. The final 1,032 unanswerables were selected by three expert annotators with 100% agreement on the final set [2305.18153].

Task-specific SALU datasets are also used. The conversational-IR SALU paper introduces C-IR_Answerability, built from multiple Chinese QA sources and synthetic negatives, with about 100K examples split evenly between answerable and unanswerable and annotated at sentence-, paragraph-, and ranked-list-level answerability [2507.16951]. RUL uses Enhanced-CAsT-Answerability (ECA), a dataset of about 12K query–context pairs with sentence-level labels, paragraph-level and ranking-level derived labels, and a ground-truth response that is either an answer or a refusal [2506.01104].

Evaluation has broadened beyond binary abstention. UA-Bench contains over 3,500 questions from six datasets spanning knowledge-intensive and reasoning-intensive tasks, and requires models to produce either a correct answer, `<DATA_UNCERTAIN>`, or `<MODEL_UNCERTAIN>` [2604.17293]. AbstentionBench assembles 20 datasets across six scenarios and emphasizes abstention recall on truly abstain-worthy questions while separately measuring answer accuracy on non-abstain questions [2506.09038]. Abstain-QA provides a black-box multiple-choice evaluation with 2,900 zero-shot items, a built-in “I Don’t Know/None of the Above” option, and the Answerable–Unanswerable Confusion Matrix (AUCM), from which Abstention Rate, Answerable Accuracy, Unanswerable Accuracy, and Precision are derived [2407.16221].

The metric landscape is correspondingly diverse. Standard formulations use precision, recall, and $F_1$ with unanswerable as the positive class [2305.18153]. Latent-direction work reports Macro-$F_1$, class-wise precision and recall, steering scores, AUC of the projection score, and abstention–intervention curves as a function of steering strength $\gamma$ [2509.22449]. UA-Bench introduces DU-$F_1$, MU-$F_1$, and AVG-$F_1$ [2604.17293]. “Line of Duty” evaluates self-set feasibility boundaries through Accuracy of Feasibility Boundary $A$, Foresight $F$, Insight $I$, and Confidence Balance $CB$ [2503.11256]. This metric heterogeneity reflects a substantive methodological difference: some SALU systems measure refusal correctness, while others measure introspective consistency or source attribution.

## 5. Empirical results

The empirical record shows that SALU-like behavior is attainable but remains incomplete. In the SelfAware study over 20 LLMs, instruction tuning improves self-knowledge relative to base prompting, in-context learning further improves it, and GPT-4-0314 reaches $F_1 \approx 75.5\%$, while the human benchmark on 100 samples is $F_1 \approx 84.9\%$ [2305.18153]. This gap is large enough that the paper explicitly describes model self-knowledge as promising but still substantially below human proficiency [2305.18153].

Latent-state methods report strong but nuanced gains. For linear unanswerability directions, same-dataset performance reaches direction-based $F_1 \approx 83\%$ on Llama-3, compared with classifier $F_1 \approx 86\%$; on unseen datasets without calibration, classifier recall drops by about $30\%$ whereas direction recall drops by about $7\%$ [2509.22449]. Threshold calibration improves cross-dataset $F_1$ by $2.7$–$23.7\%$ and reduces the gap to same-dataset performance to about $2.6\%$; the same directions also extend beyond extractive QA to SelfAware, with calibrated $F_1 = 71.3$–$78.7\%$, and to CREPE, with calibrated $F_1 = 59.1$–$61.9\%$ [2509.22449]. In text-to-SQL, LatentRefusal raises average $F_1$ to $88.5\%$ on both 8B backbones across four benchmarks while adding about 2 milliseconds of probe overhead [2601.10398].

Integrated training pipelines report the largest end-to-end gains on their target tasks. In conversational IR, SALU achieves Unanswer Accuracy/Precision/Recall/$F_1 = 0.931/0.928/0.934/0.931$, Answerable QA $F_1 = 0.835$, and Overall Accuracy $= 0.908$ on the test set [2507.16951]. The same study reports hallucination rates on unanswerable queries of $88.7\%$ for a QA-only LLM, $15.2\%$ for QA LLM + BERT-C, $8.9\%$ for SALU without RLHF, and $1.3\%$ for SALU with RLHF [2507.16951]. In RUL, unanswerability accuracy rises to $0.840$ at sentence level, $0.945$ at paragraph level, and $0.910$ at ranking level; generation metrics include answerable-question $F_1 = 0.785$ and Refusal Rate $= 0.920$ [2506.01104].

Inference-time abstention remains attractive because it avoids retraining, but the resulting performance is lower and more sensitive to calibration. Uncertainty-Based Abstention reports that, on RLHF models, semantic-entropy rejection of the top $5\%$ most uncertain TriviaQA samples raises accuracy from $84.4\%$ to $86.0\%$, and at $25\%$ rejection accuracy rises by $8.2\%$ to about $92.6\%$ [2404.10960]. On unanswerability, InDU thresholding detects about $50\%$ of unanswerable questions while wrongly refusing only $10\%$ of answerable ones, with AUROC about $0.75$ for RLHF models versus about $0.69$ for base models [2404.10960]. In the black-box Abstain-QA setting, Chain-of-Thought with an Abstain Clause is reported as the strongest prompting strategy, substantially increasing both unanswerable accuracy and answerable accuracy for GPT-4-Turbo and smaller models [2407.16221].

At the benchmark level, abstention is still described as unsolved. AbstentionBench reports that reasoning fine-tuning degrades abstention by $24\%$ on average, even on math and science domains, and that scaling Llama models from 8B to 70B and beyond changes mean recall by less than 1 percentage point [2506.09038]. Qwen 2.5 32B reaches the highest mean abstention recall at $0.71$, but no model dominates all 20 datasets [2506.09038]. This is consistent with UA-Bench, which finds that even state-of-the-art models struggle to discriminate data uncertainty from model uncertainty and that high answer accuracy does not necessarily imply strong uncertainty attribution ability [2604.17293].

## 6. Interpretation, failure modes, and open problems

A central methodological issue is whether SALU systems are genuinely introspective. AQE-based analysis argues that much of the reported success in hallucination prediction arises from question-side shortcuts rather than model-side introspection, and that hidden-state probes can overestimate self-awareness when datasets contain superficial cues [2509.15339]. This does not invalidate latent SALU methods, but it does imply that evaluation must control for question-side artifacts if the goal is to measure self-awareness rather than shortcut exploitation. A plausible implication is that benchmark design and representation auditing are as important as the refusal mechanism itself.

Another failure mode concerns the granularity of abstention. Prior work often treats refusal as a generic “I don’t know,” but UA-Bench argues that SALU should attribute uncertainty as DU versus MU because the correct downstream behavior differs [2604.17293]. Similarly, Self-Align argues that refusal alone is insufficient and that the model should provide class-aware explanations for unknown questions such as ambiguity, futurity, incorrectness, or incompleteness [2402.15062]. These works suggest that SALU is evolving from binary refusal into a structured decision layer over clarification, tool use, and explanatory abstention.

Studies of intrinsic consistency show additional weaknesses. In “Line of Duty,” even frontier models reach only about $A \approx 0.80$ on consistency between self-generated feasibility boundaries and later self-classification, with contextual awareness and temporal perception emerging as the weakest self-knowledge types [2503.11256]. The same study reports confidence imbalance, with functional-ceiling tasks showing strong overconfidence and contextual or ambiguity-related tasks showing conservatism [2503.11256]. Separately, the Self-Challenge framework produces SC-G4, a benchmark of 1,835 instances over eight error patterns, on which GPT-4 answers only $44.96\%$ correctly; those patterns also challenge Claude-3 and Llama-3 and are reported as not fully resolved through fine-tuning [2408.08978]. The eight discovered patterns include assumption of existence, complex counting, logical paradoxes, temporal ambiguity, and text manipulation [2408.08978].

The current literature therefore presents SALU as an unresolved systems problem rather than a settled module. Prompting can help, post-hoc uncertainty can help, latent probes can help, and integrated SFT+RLHF systems can reduce hallucination substantially, but generalization across domains, disentanglement of question-side and model-side signals, attribution of uncertainty source, and robustness under reasoning-oriented post-training remain open [2506.09038][2509.15339][2604.17293]. This suggests that the mature form of SALU is likely to combine calibrated latent detection, explicit uncertainty attribution, and response policies that choose among abstention, clarification, explanation, and external tool invocation rather than treating all non-answerable cases as a single refusal class.

Source: https://www.emergentmind.com/topics/self-aware-llm-for-unanswerability-salu