Alignment for Answerability in QA

• Alignment for answerability is a framework that defines when AI models should provide an answer or safely refuse based on available evidence.
• It integrates methodologies like semantic alignment, hidden-state probing, and verification to enhance model decision-making in QA systems.
• Empirical findings indicate that mixing safe completions with reasoning data can degrade answer quality, recommending architectural separation.

Searching arXiv for recent and foundational papers on answerability-related alignment, refusal, and QA verification. I’m querying arXiv for “alignment answerability”, “unanswerable question verification”, and related QA alignment work to ground the synthesis. Alignment for answerability denotes a family of methods, objectives, and evaluation regimes that make a model answer when the required information is available and refuse, abstain, or defer when it is not. In current arXiv literature, the term spans several related formulations: honesty-oriented refusal when a model lacks knowledge (Yang et al., 2023); semantic support between evidence and a candidate answer or question–answer pair (Zha et al., 2023); explicit alignment between question sub-parts and context in structured QA (Chen et al., 2020); balanced answerable/unanswerable supervision in Video-LLMs (Yoon et al., 7 Jul 2025); and answerability-based comparison of document coverage relations (Aperstein et al., 10 Sep 2025). A central complication is that “alignment” is not uniformly beneficial: when refusal-style safe completions are mixed into supervised fine-tuning data, they can act as a source of answerability degradation rather than improvement (Bekbayev et al., 2023).

1. Conceptual scope and formal definitions

The literature operationalizes answerability through several distinct but compatible formalisms. In honesty-oriented LLM alignment, a model is expected either to answer correctly if it “knows” the answer or to emit an explicit idk-style refusal otherwise. Yang et al. formalize this with a response category function $c(x,y)\in\{+1,0,-1\}$, a behavioral proxy for knowledge $k(x)$ derived from expected accuracy across multiple samples, and an honesty value $v(x,y)=1$ when $k(x)\cdot c(x,y)=+1$ (Yang et al., 2023). In Video-LLMs, answerability is defined at the input-pair level: $k(v,x)=+1$ if all information required to answer question $x$ is contained in video $v$, and $k(v,x)=-1$ otherwise (Yoon et al., 7 Jul 2025). In document systems, answerability is represented by binary indicators $C_D(q)$ over question sets, enabling formal definitions of Equivalence, Inclusion, and Semantic Overlap between documents (Aperstein et al., 10 Sep 2025). In text-pair verification, alignment is the degree to which information in one text is supported by another, instantiated as binary, three-way, or regression outputs over a pair $(x_1,x_2)$ (Zha et al., 2023).

Structured QA work makes the notion even more explicit. Khot et al. decompose question and context into semantic-role graphs and define an alignment $k(x)$0 mapping each question node to a context node, with total score $k(x)$1 (Chen et al., 2020). Conversational machine reading adopts a related stance: the model first aligns user-provided premises to document segments, then reasons over those aligned pairs to decide whether the response should be YES, NO, IRRELEVANT, or MORE (Luo et al., 2023).

Taken together, these formulations indicate that answerability is not a single property of a prompt alone. It can be treated as a property of model knowledge, of evidence support, of multimodal scope, or of informational coverage across texts. This suggests that “alignment for answerability” is best understood as a unifying perspective on selective answering rather than a single algorithmic recipe.

2. Safety alignment as a source of answerability degradation

Bekbayev et al. define “alignment” narrowly as the inclusion of model-generated refusals or safe completions such as “I’m sorry, I can’t help with that request” in the supervised fine-tuning dataset, with no explicit probabilistic safety term or augmented loss; the intervention is entirely data-level (Bekbayev et al., 2023). Their ablation uses LLaMA 2 7B, a merged GoatAssistant + Guanaco dataset, one epoch of training with DeepSpeed ZeRO-3, bfloat16, batch size 512, and AdamW at learning rate $k(x)$2. The “with alignment” condition retains all safe-refusal responses, while the “no alignment” condition removes approximately one third of examples whose bot replies were refusals or overly cautious safe completions.

Under otherwise identical training, the no-alignment model scores 49.31 on MMLU versus 45.63 with alignment, 35.69 versus 34.28 on BBH, 12.20 versus 9.15 on HumanEval, and 28.10 versus 22.61 on DROP (Bekbayev et al., 2023). The corresponding relative degradations are reported as 8.1% for MMLU, 4.1% for BBH, 33.3% for HumanEval, and 24.3% for DROP, with absolute drops ranging from 3.68 to 5.49 points. The paper characterizes this effect as a form of poisoning: when roughly one third of targets are refusals, the model learns a bias toward non-answering.

The study’s main recommendation is architectural and procedural separation. Safety-only data should be kept apart from the core instructional QA set, and refusal behavior should be applied in a downstream filtering stage or via a dedicated safety head rather than being interleaved with reasoning data (Bekbayev et al., 2023). This is a direct challenge to the assumption that more refusal data necessarily improves safe deployment. In this line of work, answerability is degraded not by insufficient alignment, but by a particular way of encoding alignment into supervised targets.

3. Detection, abstention, and verification signals

A major branch of the literature focuses on deciding when not to answer. In “Alignment for Honesty,” answerability is framed through the joint behavior of prudence, over-conservativeness, honesty, and task accuracy. On TriviaQA with LLAMA2-CHAT-13B, prompt-only alignment yields prudence 33.8, over-conservativeness 12.5, honesty 60.6, and accuracy 64.7. Supervised variants do better: ABSOLUTE gives prudence 47.7, over-conservativeness 9.9, honesty 68.9, and accuracy 71.3; CONF-VERB gives 58.9, 10.7, 74.1, and 73.3; MULTISAMPLE gives 67.7, 15.9, 75.9, and 68.9 (Yang et al., 2023). The same work reports that helpfulness on Eval-P$k(x)$3 remains essentially unchanged: the unaligned model receives 5.56/10 from AUTO-J and 8.62/10 from GPT-4, while CONF-VERB receives 5.54/10 and 8.61/10, and MULTISAMPLE 5.52/10 and 8.56/10.

A distinct line seeks answerability signals before any text is generated. Geometric deviation measures the cosine distance between a query’s mean-centered hidden state and the centroid of a small answerable reference set. For a query $k(x)$4, the score is

$k(x)$5

Across Llama 3.1-8B-Instruct, Qwen 2.5-7B-Instruct, and Mistral-7B-Instruct-v0.3, Math prompts show strong separation between answerable and unanswerable inputs, with ROC-AUC 0.78–0.84 and large effect sizes $k(x)$6; Code prompts show AUC approximately 0.77–0.82 with higher variance; Fact prompts show no reliable signal (Du, 4 May 2026). The layer-wise analysis reports that the gap rises sharply from embeddings, peaks in early layers around $k(x)$7–5, and attenuates toward the final layer.

Post-generation verification uses a different mechanism. ALIGN trains a RoBERTa-based text alignment model on 5.9M examples from 28 datasets and then uses a binary alignment head as an unanswerable verifier by scoring the context against “question + predicted answer” (Zha et al., 2023). In a two-stage QA pipeline, the unanswerable score is $k(x)$8, with threshold selected on validation data. When stacked on GPT-3.5 or FLAN-T5, the verifier improves average exact match by 17.94 points and average F1 by 15.05 across SQuAD v2, ACE-whQA, and Simplified NQ; on SQuAD v2, GPT-3.5 rises from EM 52.53 / F1 63.96 to EM 67.19 / F1 77.63, while FLAN-T5 rises from EM 75.72 / F1 79.01 to EM 83.72 / F1 86.55 (Zha et al., 2023).

These results support three complementary strategies: training models to refuse appropriately, probing hidden-state geometry before generation, and verifying semantic support after generation. This suggests that answerability can be monitored at multiple stages of the inference pipeline rather than only at the output surface.

4. Explicit evidence alignment in question answering and conversational reasoning

In robust textual QA, alignment is not merely a confidence score but the core inference object. “Robust Question Answering Through Sub-part Alignment” parses both question and context into semantic-role graphs, represents predicate and argument spans with BERT-based span vectors, and scores alignments by dot product within a structured SVM objective (Chen et al., 2020). Approximate inference uses beam search under constraints such as locality, Hard Entity Match, and Worst-Alignment-Gap (WAG). The WAG criterion computes

$k(x)$9

and rejects a prediction when $v(x,y)=1$0. Trained only on SQuAD 1.1, the model is more robust than a standard BERT QA baseline on adversarial and cross-domain tests: on addSent, BERT QA drops from 87.8 to 39.2 F1, whereas Sub-part Alignment drops from 84.5 to 50.5, a recovery of 11.3 F1 over BERT; in MRQA it improves F1 by 2.1 on Natural Questions, 1.0 on BioASQ, and 13.1 on TextbookQA, with a 0.4 decrease on NewsQA (Chen et al., 2020).

Conversational machine reading uses explicit alignment to determine whether follow-up clarification is necessary. BiAE segments the document into EDUs, splits user information into premises from scenario sentences and dialogue history, computes a bipartite alignment matrix between hypothesis and premise representations, and feeds the aligned pairs into a many-to-many entailment module over entailment, contradiction, and neutral labels (Luo et al., 2023). The joint objective combines decision loss, alignment loss, and entailment loss. On ShARC, BiAE achieves 77.9 micro-accuracy and 81.1 macro-accuracy on the held-out test set, while ablations show that removing alignment loss or entailment loss reduces performance to 74.3/78.9 and 74.1/78.3, respectively; removing both yields 72.6/76.9 (Luo et al., 2023).

Even highly simplified models can exploit alignment effectively. “Sanity Check” proposes one-to-many positive and negative alignment between question terms and answer or justification terms using cosine similarity in GloVe space, with overall score $v(x,y)=1$1 (Yadav et al., 2018). The method is unsupervised and parameter-light, yet reaches 47.00 P@1 on 8th-grade Science QA, 32.93 P@1 on Yahoo! Answers, 64.02 MAP on WikiQA, and 26.56 / 58.36 on ARC Challenge / Easy. The paper shows that one-to-many alignment generally outperforms both one-to-one and one-to-all ablations.

This body of work treats answerability as an evidence-structure problem. A question is answerable when its semantic components can be anchored to support in the context; failure to find such anchors becomes an actionable abstention or clarification signal.

5. Metrics and benchmarks for answerability evaluation

Question generation research has made answerability a first-class evaluation target. PMAN defines a prompting-based metric over triples $v(x,y)=1$2 by querying an LLM for a YES/NO judgment and averaging the indicator of YES:

$v(x,y)=1$3

On a manually crafted HotpotQA-based test set, GPT-4 with Chain-of-Thought reaches approximately 94% accuracy on non-yes/no samples; on model-generated data it achieves approximately 88% accuracy, and the induced system ranking exactly matches human answerability ranks, unlike BLEU or ROUGE in several cases (Wang et al., 2023).

Earlier work approached the same problem by modifying overlap metrics. “Towards a Better Metric for Evaluating Question Generation Systems” defines an Answerability score as a harmonic mean of weighted precision and recall over four token classes: question words, named entities, relevant content words, and function words (Nema et al., 2018). This score is interpolated with a base metric $v(x,y)=1$4 to form $v(x,y)=1$5. On WikiMovies, BLEU-4 has Pearson correlation $v(x,y)=1$6 with human answerability judgments, whereas Q-BLEU4 reaches 0.826; on SQuAD, BLEU-1 rises from 0.167 to 0.258 when answerability is added; on VQA, Q-ROUGE-L reaches Pearson 0.402 and Spearman 0.385 (Nema et al., 2018).

Answerability has also been studied as a platform-level prediction problem rather than a model output property. On Quora, Maity et al. predict whether a question will eventually be answered using lexical, syntactic, semantic, topic, editorial, and psycholinguistic features, with SVM performing best (Maity et al., 2017). The reported accuracy is 76.26% for $v(x,y)=1$7 month and 68.33% for $v(x,y)=1$8 months, with linguistic style features alone accounting for 74.18% accuracy. The analysis associates answerability with language use, edit patterns, and topic-tag structure rather than with any downstream answer model.

These benchmarks and metrics broaden the topic beyond refusal. They evaluate whether a generated question is answerable, whether a platform question is likely to receive an answer, and whether automatic metrics respect the informational requirements of answering. This suggests that alignment for answerability includes both model behavior and dataset-level measurement design.

6. Document, multimodal, and safety-critical extensions

Document-level alignment can be defined directly over sets of answerable questions. In Semantic Coverage Relations, each document $v(x,y)=1$9 is associated with an indicator $k(x)\cdot c(x,y)=+1$0 for whether it can answer question $k(x)\cdot c(x,y)=+1$1, and document pairs are classified as Equivalence, Inclusion, or Semantic Overlap based on these answerability sets (Aperstein et al., 10 Sep 2025). The synthetic SQuAD-derived benchmark contains 2,952 labeled pairs from 82 sources, split into 2,361 train and 591 test instances. On the test set, RoBERTa-base achieves the highest accuracy at 61.4%, while Random Forest attains the best macro-F1 at 52.9%; GPT-4.1 zero-shot reaches 33.9% accuracy and 34.1% macro-F1, and GPT-4o variants range from 27.9% to 39.9% accuracy (Aperstein et al., 10 Sep 2025). Here answerability is used not to judge a single answer, but to recover document-level semantic coverage.

In Video-LLMs, the central problem is scope mismatch between question and video. Existing systems are predominantly trained on questions generated directly from video content and therefore answer even when the question exceeds the informational scope of the clip (Yoon et al., 7 Jul 2025). The UVQA pipeline constructs 30K synthetic unanswerable training pairs, 300 human-filtered unanswerable test pairs, and supplements them with 30K answerable pairs from Video-ChatGPT. Evaluation uses metrics such as absolute accuracy $k(x)\cdot c(x,y)=+1$2, excessive refusal $k(x)\cdot c(x,y)=+1$3, permissiveness $k(x)\cdot c(x,y)=+1$4, and discretion $k(x)\cdot c(x,y)=+1$5. Unaligned models score essentially 0 on the answerability-specific metrics; after supervised fine-tuning or DPO, answerability F1 rises to approximately 0.6–0.7, discretion often exceeds 0.6, excessive refusal remains below 0.5, and overall absolute accuracy on the combined answerable and unanswerable test set roughly doubles from approximately 0.25 to approximately 0.5 (Yoon et al., 7 Jul 2025).

Safety-critical EHR QA introduces a combined classification-and-correction problem. SCARE evaluates post-hoc verifiers over 4,200 triples of natural-language questions, schemas, candidate SQL queries, and expected outputs, balanced across answerable-correct, answerable-incorrect, ambiguous, and unanswerable cases for MIMIC-III, MIMIC-IV, and eICU (Lee et al., 13 Nov 2025). Metrics separate preservation of already-correct SQL, correction of incorrect SQL, and precision/recall/F1 for ambiguous and unanswerable labels. Among the reported methods, Single-Turn-Veri-Cls achieves preservation rate 95.9, correction rate 53.8, ambiguous F1 53.9, and unanswerable F1 80.8 (Lee et al., 13 Nov 2025). The benchmark emphasizes a direct trade-off: improving question classification can reduce coverage on truly answerable items, while aggressive preservation can under-detect ambiguous or unanswerable cases.

Across these domains, answerability becomes a general systems property. It mediates document comparison, multimodal scope control, and safe query execution, not only QA correctness in the narrow sense.

7. Alignment architectures, process-level optimization, and unresolved tensions

Several recent methods attempt to improve answerability without relying solely on refusal labels. Aligner learns correctional residuals between preferred and dispreferred answers with a small residual model $k(x)\cdot c(x,y)=+1$6 added to the upstream model’s logits, producing corrected logits $k(x)\cdot c(x,y)=+1$7 and aligned distribution $k(x)\cdot c(x,y)=+1$8 (Ji et al., 2024). The method is model-agnostic and can be deployed on open-source or API-based LLMs. The abstract reports that Aligner-7B achieves an average improvement of 68.9% in helpfulness and 23.8% in harmlessness across 11 tested LLMs while also effectively reducing hallucination; a community report on Alpaca-Eval states that stacking Aligner-2B on GPT-4 Turbo improves LC Win Rate from 55.0% to 58.3%, exceeding GPT-4 Omni’s 57.5% (Ji et al., 2024). In this framework, answerability is improved by output correction rather than by direct abstention modeling.

Process-level alignment targets the reasoning trajectory itself. MR-ALIGN identifies a reasoning–answer hit gap, where the model surfaces the correct fact during chain-of-thought but fails to place it in the final answer, and then estimates transition probabilities over atomic meta-reasoning states to construct a transition-aware implicit reward (Wang et al., 27 Oct 2025). On Qwen3-8B, NQ-Open accuracy rises from 34.10 to 37.34 while Misleading falls from 9.89 to 7.20; on SciQ, accuracy rises from 67.10 to 70.70 and Misleading falls from 14.10 to 11.70 (Wang et al., 27 Oct 2025). The intervention is aimed at factual answerability by shifting probability mass toward more beneficial reasoning transitions.

A more general theoretical approach is given by Distributional Alignment Games for Answer-Level Fine-Tuning. Here the objective depends on the final answer distribution $k(x)\cdot c(x,y)=+1$9 rather than the latent reasoning trace, and the intractable optimization is rewritten as a two-player game between a Policy $k(v,x)=+1$0 and a Target distribution $k(v,x)=+1$1 (Mohri et al., 29 Apr 2026). The paper proves that the Nash equilibrium of the game corresponds to the solution of the original answer-level optimization problem. In experiments, Pairwise-GRPO improves GSM8K accuracy by 4.55–9.18 percentage points and TriviaQA exact match by 7.2% to 42.1%, while Coherence-GRPO gives smaller but still positive gains in several settings (Mohri et al., 29 Apr 2026). This connects answerability to answer-level optimization rather than only to refusal or support checking.

The current literature leaves several persistent tensions. Safety-refusal data can degrade reasoning when mixed indiscriminately into instruction tuning (Bekbayev et al., 2023). Geometric deviation is reliable in structured domains such as Math but fails on factual prompts, indicating strong form-conditionality (Du, 4 May 2026). ALIGN-style verifiers are effective but threshold-sensitive across domains (Zha et al., 2023). Video-LLMs still exhibit excessive refusal after alignment (Yoon et al., 7 Jul 2025), and EHR verifiers still struggle with a trade-off between classification and SQL correction, especially on nuanced ambiguity and global repair errors (Lee et al., 13 Nov 2025). A plausible implication is that alignment for answerability will remain a multi-component problem requiring separate mechanisms for core answering, abstention, verification, and safety intervention rather than a single unified loss.