Hypothesis-Conditioned Query Rewriting (HCQR)

• The paper introduces HCQR, a training-free method that leverages hypothesis-driven query rewriting to focus retrieval on evidence that decisively supports a candidate answer.
• HCQR generates three distinct queries—SUPPORT, DISTINCTION, and KEY FEATURES—to verify clues, differentiate answer options, and confirm decision boundaries.
• Empirical evaluations on MedQA and MMLU-Med demonstrate that HCQR improves accuracy by up to 5.9 points and greatly enhances the decision-usefulness of retrieved context.

Searching arXiv for the cited HCQR paper and closely related query-rewriting work to ground the article in the relevant literature. {"query":"arXiv (Chang et al., 19 Mar 2026) Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval", "max_results": 5} {"query":"arXiv (Ye et al., 2023) Enhancing Conversational Search LLM-Aided Informative Query Rewriting", "max_results": 5} Hypothesis-Conditioned Query Rewriting (HCQR) is a training-free pre-retrieval method for Retrieval-Augmented Generation (RAG) designed for settings in which a model must make a decision among competing answers rather than merely produce a generally informed response. Its central claim is that, in such settings, retrieval should not merely be topically relevant; it should be decision-useful. The method was introduced in "Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval" (Chang et al., 19 Mar 2026), which frames retrieval as evidence-oriented retrieval: retrieved context is useful when it narrows the answer space by supporting one answer, distinguishing leading alternatives, or verifying salient clues in the question.

1. Concept and problem setting

HCQR targets a specific failure mode of standard RAG. In many multiple-choice or decision problems, retrieving documents with the raw question as a single query tends to return passages that are about the topic but do not actually help decide between answer options. Such passages may provide background exposition or keyword overlap while failing to support one option, rule out alternatives, or verify the crucial clues in the question stem (Chang et al., 19 Mar 2026).

The method is motivated especially by what the paper calls optional-retrieval decision settings. In these settings, the base LLM may already know enough to answer some questions from parametric memory, so retrieval helps only if it adds evidence that materially improves the final choice. Weakly aligned retrieved context can therefore be harmful: irrelevant or generic passages can distract the model, waste limited context budget, and degrade reasoning. This is a central contrast with standard topic-oriented retrieval, where ordinary relevance may be sufficient for broad answer generation.

The paper’s medical multiple-choice examples make the distinction concrete. In such tasks, the correct answer may hinge on a diagnostic criterion, a contrast between close alternatives, or a question-specific clue such as age, symptom pattern, or test result. A single query based on the raw question may retrieve disease overviews that are relevant in the ordinary information-retrieval sense but are not useful for answer selection. HCQR therefore reframes the retrieval objective from topic-oriented retrieval to evidence-oriented retrieval.

A common misconception is that HCQR is merely a variant of multi-query retrieval. The method explicitly rejects that interpretation. Its rewritten queries are not paraphrases or semantic variants of the same request. They are functionally distinct evidence-seeking queries with different verification roles.

2. Formal task formulation

The paper formalizes the setting with a question $x$, candidate answers $A=\{a_1,\ldots,a_K\}$, and a corpus $\mathcal{D}$. A retrieval method $m$ constructs a final context set $C_m(x,A)\subseteq\mathcal{D}$ under a maximum final-context budget $B_{\max}$:

$|C_m(x,A)| \le B_{\max}$

The no-retrieval baseline is:

$C_m(x,A)=\emptyset$

An important design point is that $B_{\max}$ constrains the evidence shown to the final generator, not necessarily the number of intermediate retrieval candidates examined. Methods may therefore inspect different numbers of candidates while remaining comparable under the same final context budget (Chang et al., 19 Mar 2026).

HCQR frames retrieval planning as hypothesis-guided verification. Rather than retrieving from the raw question alone, the method first forms a working answer hypothesis and then asks what evidence would be needed to verify or refute it. The pipeline is formalized as follows:

1. A hypothesis formulator $H$ produces a structured hypothesis state:

$A=\{a_1,\ldots,a_K\}$0

where $A=\{a_1,\ldots,a_K\}$1 is the working hypothesis, $A=\{a_1,\ldots,a_K\}$2 the discriminating features, $A=\{a_1,\ldots,a_K\}$3 the confirming evidence, and $A=\{a_1,\ldots,a_K\}$4 the brief reasoning.

2. A query rewriter $A=\{a_1,\ldots,a_K\}$5 uses the question and hypothesis state to produce three queries:

$A=\{a_1,\ldots,a_K\}$6

3. A shared retriever $A=\{a_1,\ldots,a_K\}$7 retrieves evidence for those queries and fuses them:

$A=\{a_1,\ldots,a_K\}$8

4. The final generator $A=\{a_1,\ldots,a_K\}$9 predicts the answer from question, options, and fused context:

$\mathcal{D}$0

A key constraint is that the working hypothesis is not passed directly to the final generator by default. It affects retrieval only through query rewriting. This is intended to reduce direct answer anchoring and keep the intervention retrieval-side rather than generation-side.

3. Query structure and retrieval mechanics

HCQR is a training-free, two-stage pre-retrieval planning method. First, an LLM-based hypothesis formulator reads the question and candidate options and produces a compact structured state consisting of a provisional best answer, discriminating features, confirming evidence, and brief reasoning. The paper stresses that this is not the final answer; it is a temporary hypothesis used to organize retrieval (Chang et al., 19 Mar 2026).

Second, the query rewriter generates exactly three queries, each with a distinct retrieval role.

Query type	Inputs emphasized	Retrieval role
SUPPORT ($\mathcal{D}$1)	hypothesized answer text, confirming evidence, brief reasoning	retrieve evidence that would directly support the provisional answer
DISTINCTION ($\mathcal{D}$2)	discriminating features, candidate answers	retrieve criteria that separate the leading candidate from competing options
KEY FEATURES ($\mathcal{D}$3)	salient clues in the question stem	verify the question’s own evidence structure

The SUPPORT query asks, in effect, what supporting evidence should exist if the current answer were correct. The DISTINCTION query is contrastive: it seeks evidence that would distinguish the current candidate from nearby alternatives. The KEY FEATURES query verifies salient clues in the question independently of whether the initial hypothesis is accurate. The paper presents this third query as an important robustness mechanism when the working hypothesis is partially wrong.

Retrieval is then performed independently for each query using the same retriever. In the experiments, top 5 documents per query are retrieved, the three sets are merged, duplicates are removed, and the final set is kept within the shared budget $\mathcal{D}$4. The fusion mechanism is therefore simple merge-plus-deduplicate under budget. The method does not introduce a learned ranker, a weighted fusion rule, or an HCQR-specific post-retrieval scoring function. Its algorithmic novelty lies in query planning before retrieval, not in post-retrieval selection.

The implementation recipe reported in the paper is correspondingly lightweight: input question and options; run the hypothesis formulator; parse best_guess, best_guess_text, discriminating_features, confirming_evidence, and reasoning; run the query rewriter; parse exactly three queries; retrieve top 5 documents per query; union and deduplicate; truncate to $\mathcal{D}$5; then feed final context, question, and options to the generator using the shared MIRAGE prompt. The experiments use MedCPT over a medical textbook corpus, with all LLM-based components inside a method using the same model as the final generator. The tested model configurations are Llama3.2-3B-instruct, Llama3.1-8B-instruct, Qwen3-4B-instruct-2507, and Qwen3-30B-A3B-instruct-2507 (Chang et al., 19 Mar 2026).

An important misconception is that the hypothesis commits the system to an answer. The paper explicitly argues otherwise. The final generator is intended to use retrieved evidence to confirm or overturn the provisional hypothesis.

4. Empirical evaluation and decision-useful retrieval

The evaluation is conducted on MedQA and MMLU-Med, both medical multiple-choice QA benchmarks. The main metrics are accuracy and Decision-Useful Rate (DUR). Let $\mathcal{D}$6 be the number of questions in dataset $\mathcal{D}$7, and $\mathcal{D}$8 and $\mathcal{D}$9 the counts judged Entailed and Useful under dataset $m$0, method $m$1, and model $m$2. The paper defines:

$m$3

Retrieved context sets are labeled by GPT-4.1 as Entailed, Useful-but-Not-Entailed, or Not Useful. These labels denote, respectively, that the retrieved context alone decisively supports the gold answer, strongly helps but needs one bridge, or is not decision-useful (Chang et al., 19 Mar 2026).

The main average accuracy results reported in Table 1 are:

• MedQA: CoT 71.0, Simple RAG 69.1, Rerank-RAG 69.9, Rewriting 70.7, HyDE 71.1, MAIN-RAG 69.0, HCQR 75.0.
• MMLU-Med: CoT 78.6, Simple RAG 77.5, Rerank-RAG 78.8, Rewriting 78.2, HyDE 79.1, MAIN-RAG 78.0, HCQR 81.1.

Thus HCQR improves average accuracy over Simple RAG by 5.9 points on MedQA and 3.6 points on MMLU-Med. The paper also notes a stronger qualitative pattern: Simple RAG underperforms no-retrieval CoT in every dataset-model cell. This supports the claim that naive retrieval can be harmful in optional-retrieval decision settings.

The DUR results provide the paper’s main mechanistic evidence. Average DUR in Table 2 is:

• MedQA: Simple RAG 30.0, Rerank-RAG 44.1, Rewriting 56.1, HyDE 53.4, MAIN-RAG 25.4, HCQR 82.1.
• MMLU-Med: Simple RAG 48.7, Rerank-RAG 56.8, Rewriting 55.3, HyDE 44.9, MAIN-RAG 35.6, HCQR 70.3.

The utility breakdown averaged across datasets and models in Table 3 is also distinctive:

• Simple RAG: Entailed 14.3 / Useful 25.1 / Not Useful 60.7
• Rerank-RAG: 19.9 / 30.6 / 49.5
• Rewriting: 19.6 / 36.1 / 44.3
• HyDE: 16.6 / 32.5 / 50.8
• MAIN-RAG: 11.6 / 18.9 / 69.5
• HCQR: 34.7 / 41.5 / 23.8

These numbers indicate that HCQR does not merely improve answer accuracy; it retrieves context that is much more often judged actually useful for the decision.

The utility-conditioned results in Table 4 reinforce the same interpretation. On MedQA, accuracy is 94.4 for Entailed contexts with a same-subset gain of $m$4 over CoT, 77.9 for Useful with $m$5, and 57.6 for Not Useful with $m$6. On MMLU-Med, the corresponding values are 95.4 and $m$7, 83.9 and $m$8, and 70.2 and $m$9. This suggests that the paper’s central distinction between useful and non-useful retrieval is not merely descriptive; it is directly connected to downstream decision quality.

5. Baselines, ablations, and relation to broader query rewriting

The paper compares HCQR against six baselines: No-RAG (CoT), Simple RAG, Rewriting, HyDE, Rerank-RAG, and MAIN-RAG. The conceptual contrast is explicit. Simple RAG retrieves what is relevant to the question. Rewriting and HyDE reformulate the question to improve semantic match or recall. Rerank/filter methods retrieve a larger pool and improve selection afterward. HCQR instead first infers the likely decision boundary and then retrieves evidence specifically for verifying that boundary (Chang et al., 19 Mar 2026).

This distinction is especially important in relation to generic query rewriting. The Rewriting baseline uses three generic rewritten queries following the LC-MQR prompt family. HCQR differs because its queries are not paraphrases or alternate phrasings; they are role-structured verification queries. The paper therefore treats HCQR as an intervention on first-pass candidate generation rather than a post hoc repair step.

The ablations address two common alternative explanations. First, Table 5 tests whether gains come merely from exposing the retrieval stage to answer options. The results are Rewriting 70.7 / 78.2, Rewriting + options 71.3 / 79.2, HCQR - options 73.1 / 79.9, and HCQR 75.0 / 81.1 on MedQA / MMLU-Med. This shows that adding options to generic rewriting helps a little, but HCQR remains stronger.

Second, Table 6 tests the three query roles. Removing any of them degrades performance: HCQR 75.0 / 81.1, HCQR - $C_m(x,A)\subseteq\mathcal{D}$0 72.4 / 79.0, HCQR - $C_m(x,A)\subseteq\mathcal{D}$1 73.2 / 80.1, HCQR - $C_m(x,A)\subseteq\mathcal{D}$2 73.0 / 80.5. Appendix DUR ablations show even larger drops: on MedQA, HCQR 82.1, HCQR - $C_m(x,A)\subseteq\mathcal{D}$3 63.6, HCQR - $C_m(x,A)\subseteq\mathcal{D}$4 70.8, HCQR - $C_m(x,A)\subseteq\mathcal{D}$5 73.2; on MMLU-Med, HCQR 70.3, HCQR - $C_m(x,A)\subseteq\mathcal{D}$6 53.0, HCQR - $C_m(x,A)\subseteq\mathcal{D}$7 59.2, HCQR - $C_m(x,A)\subseteq\mathcal{D}$8 58.1. The largest average drop comes from removing the support query, but all three contribute substantially.

In a broader query-rewriting context, "Enhancing Conversational Search: LLM-Aided Informative Query Rewriting" (Ye et al., 2023) is relevant because it treats query rewriting as a controlled generation problem whose quality is defined by retrieval effectiveness rather than by minimal decontextualization alone. That work defines four properties for a well-formed rewrite—correctness, clarity, informativeness, and nonredundancy—and introduces a rewrite-then-edit process. A plausible implication is that HCQR belongs to a wider family of controlled rewriting methods in which the rewrite is conditioned by a structured objective rather than by surface paraphrase alone. The difference is that HCQR’s conditioning signal is a provisional answer hypothesis and its associated evidence needs, whereas the conversational-search work conditions on contextual completeness and informativeness for standalone retrieval.

6. Interpretive significance, case study, and limitations

The paper’s case study is designed to show that HCQR can succeed even when the initial hypothesis is wrong. In the reported MedQA example, the hypothesis formulator initially guessed a clubfoot-type management option, but the retrieved evidence from distinction and key-feature queries surfaced clues pointing instead to metatarsus adductus, leading the final generator to revise the initial guess and answer reassurance correctly (Chang et al., 19 Mar 2026). This example is central because it supports the claim that the hypothesis is a retrieval-planning device rather than a hard commitment.

Several strengths follow directly from the reported experiments. HCQR is training-free; it improves first-pass retrieval instead of relying on larger reranking pools; it achieves the best performance in every dataset-model cell reported; and its gains are mechanistically interpretable because they align with higher DUR and a better utility breakdown. The case study also suggests robustness to imperfect hypotheses because the distinction and key-feature queries can surface evidence that contradicts the provisional answer.

The paper also states several limitations. HCQR depends on hypothesis quality: if the intermediate hypothesis state is inaccurate or incomplete, the rewritten queries may be misaligned with the true decision boundary. The key-feature query mitigates this risk but does not eliminate it. The empirical scope is limited to medical multiple-choice QA, and transfer to other domains or open-ended tasks remains open. The method is naturally suited to tasks where candidate answers are observed; broader generalization to tasks with a small set of plausible hypotheses is suggested conceptually but not experimentally validated. HCQR also adds computational overhead relative to Simple RAG: one LLM call for hypothesis formulation, one LLM call for query rewriting, and three retrievals instead of one. The experiments further assume a fixed corpus, a fixed retriever family, and a fixed final context budget, and the paper does not claim that gains transfer unchanged to open-web retrieval, much larger corpora, or fully agentic multi-turn retrieval.

A broader interpretive lesson emerges from both HCQR and the controlled rewriting literature. The conversational search work on informative rewrites emphasizes that retrieval-effective rewriting is not synonymous with linguistic paraphrase; it is a constrained generation problem in which the rewrite should surface the information that actually improves retrieval (Ye et al., 2023). HCQR specializes that idea for answer selection under competition among options. Its central lesson is that, under a fixed context budget, better first-pass query construction can matter more than retrieving more documents and filtering later.