Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation

Abstract: Retrieval-Augmented Generation (RAG) enhances the factuality of LLMs by incorporating retrieved documents and/or generated context. However, LLMs often exhibit a stylistic bias when presented with mixed contexts, favoring fluent but hallucinated generated content over factually grounded yet disorganized retrieved evidence. This phenomenon reveals that the utility of retrieved information is bottlenecked by its presentation. To bridge this gap, we propose QREAM, a style-controlled rewriter that aligns retrieved documents with a question-oriented style while preserving facts, better for LLM readers to utilize. Our framework consists of two stages: (1) QREAM-ICL, which uses stylistic seeds to guide iterative rewriting exploration; and (2) QREAM-FT, a lightweight student model distilled from denoised ICL outputs. QREAM-FT employs dual-criteria rejection sampling, filtering based on answer correctness and factual consistency to ensure high-quality supervision. QREAM seamlessly integrates into existing RAG pipelines as a plug-and-play module. Experiments demonstrate that QREAM consistently enhances advanced RAG pipelines, yielding up to 8% relative improvement with negligible latency overhead, effectively balancing question relevance with factual grounding.

• The paper introduces QREAM, a post-retrieval rewriting module that aligns document style to question-oriented formats while preserving factual content.
• It employs an Explore-then-Distill paradigm, combining in-context stylistic exploration with bidirectional denoising to enhance downstream QA performance.
• QREAM achieves up to 8% accuracy improvements and significantly reduces hallucination rates, ensuring robust and efficient ODQA.

Question-Oriented Document Rewriting for RAG: An Expert Analysis of QREAM

Context and Motivation

Retrieval-Augmented Generation (RAG) pipelines combine parametric generation with external retrieval to improve the factuality and coverage in knowledge-intensive applications—especially Open-Domain Question Answering (ODQA). However, a disjunction exists between the retrieved evidence (factually reliable but often stylistically mismatched, verbose, or irrelevant) and generated documents (question-oriented in style, but prone to hallucination). LLM readers can be strongly biased towards the fluency and style of generated content, potentially ignoring the more factual but unstructured retrievals. This stylistic bias constrains RAG effectiveness by reducing the utility of the most reliable evidence.

The paper "Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation" (2604.17325) introduces QREAM, a post-retrieval document rewriting module that aligns the style of retrieved documents to the question orientation of generated ones while preserving factuality. This method aims to make retrieved evidence more accessible and persuasive to downstream LLM readers and deliver robust improvements across ODQA tasks.

Figure 1: QREAM integrates the factual accuracy of retrieved documents with the stylistic, question-oriented structure typical of generated contexts.

QREAM Framework: Explore-then-Distill Paradigm

QREAM operates in a two-stage pipeline:

1. QREAM-ICL (In-Context Learning Exploration): Uses stylistic seeds to iteratively rewrite retrieved passages, guiding LLMs to generate question-oriented variants without contaminating content with hallucinated facts. Multiple rewriting rounds promote diversity, while unrelated seeds ensure style-content disentanglement.
2. QREAM-FT (Fine-Tuned Student): Denoises the set of explored rewrites via a Bidirectional Denoising mechanism—incorporating both downstream utility (does the rewrite help the RAG reader answer correctly?) and upstream fidelity (does the rewrite remain grounded in the raw retrieved evidence?). The optimal rewrites are selected via dual-criteria rejection sampling and used to fine-tune a lightweight student model for efficient, high-quality rewriting at inference.

Figure 2: QREAM runs in an Explore-then-Distill paradigm: Stage I iteratively explores stylistic rewrites; Stage II distills purified rewrites with strict rejection sampling.

Stage I: In-Context Stylistic Exploration

• M randomly selected seed questions with LLM-generated "background documents" serve as style exemplars.
• Iterative rewriting (N rounds) proceeds by feeding previous outputs truncated to a length constraint, encouraging high information density.
• Unlike self- or zero-shot style transfer, unrelated seeds allow the model to focus exclusively on stylistic alignment without importing irrelevant or hallucinated content.

Stage II: Bidirectional Denoising Distillation

• Each candidate rewrite is filtered under two constraints:
  • Downstream Utility: The answer predicted by the QA reader must include the ground truth (hard filter); further, answer prediction F1 is scored.
  • Upstream Fidelity: Each atomic fact in the rewrite is verified against the original evidence (using decomposition and entailment), penalizing unsupported content.
• A composite score selects the best rewrite for each question, constructing a denoised supervision set for the QREAM-FT student model.

Figure 3: Ablation on QREAM-ICL shows the necessity of stylistic seeds and optimal seed number for maximizing QA performance.

Empirical Results

QREAM is systematically evaluated on four ODQA datasets: Natural Questions (NQ), TriviaQA (TQA), HotpotQA, and 2WikiMultiHopQA. Multiple LLM readers (Llama-3-8B, Mistral-7B-Instruct, and GPT-5 mini) and integration with state-of-the-art RAG frameworks (Self-RAG, HippoRAG) are considered.

• Performance: QREAM-ICL and QREAM-FT both outperform strong baselines, including document compression (LongLLMLingua, CompAct, RECOMP, FaviComp) and hybrid generation-retrieval schemes. On multi-hop reasoning datasets, QREAM-FT achieves relative improvements up to 8% in accuracy over standard RAG. Notably, QREAM-FT attains comparable accuracy to its 8B teacher model while being $\sim$13× faster.
• Document Quality: The rewritten outputs match the style alignment of generated documents (style score $s_{\text{orient}}$) but maintain factual inconsistency rates close to retrieved text. Hallucination rates decline from 33.7% (generated) and 13.8% (QREAM-ICL) to 9.2% (QREAM-FT).
• Plug-and-Play: Gains are robust across frameworks and persist even with strong proprietary models such as GPT-5 mini.
• Mitigation of Stylistic Bias: Under the context-conflicting setting (mixing correct retrieved evidence and misleading generation), QREAM rewrites shift the LLM reader's preference towards actual evidence, with accuracy restoring from $\sim$19% to over 77% (NQ-CC, Llama-3-8B).

Figure 4: QREAM-ICL performance saturates after several rewriting iterations, demonstrating efficient convergence for document refinement.

Component Analysis and Ablations

• Distillation Criteria: Dual-criteria rejection sampling (utility + fidelity) is necessary; relying exclusively on either increases hallucinations and reduces QA F1.
• Stylistic Seeds: The absence or misuse of stylistic seeds (e.g., self-rewriting) significantly reduces performance.
• Rewriting Iteration: A few iterations ($N=3$) suffice to reach optimal quality, with performance drops if explored further.

Qualitative Assessment

QREAM enables LLMs to extract only question-relevant information from retrievals, explicitly organizing facts for direct answerability and reducing ambiguity or distraction (see qualitative examples in Table \ref{tab:case_study} of the paper).

Implications and Future Developments

QREAM demonstrates that style alignment, not just factual content, is a critical axis for effective knowledge grounding in LLM pipelines. By acting as a style-aware, content-faithful, and computationally efficient module, it improves ODQA robustness, particularly for complex multi-hop tasks and in adversarial context scenarios.

Practical implications include:

• Immediate deployment without architectural modification in existing RAG frameworks.
• Increased trustworthiness and explainability by reducing reader hallucination and bias.
• Improved inference efficiency, supporting low-latency, large-scale systems.

Theoretically, QREAM establishes a foundation for deeper study of presentation-induced biases in LLMs and data-centric RAG optimization. The explicit decoupling of style and content in rewrite supervision, as well as the bidirectional denoising approach, are likely extensible to other modalities and grounded generation tasks.

Further advances could include joint optimization of retrieval and rewriting, continuous learning of style metrics, and integration with more structured, entity-aware grounding.

Conclusion

QREAM provides a data-centric, modular approach for improving the factual utilization of retrieved documents in RAG by aligning their style to the needs of LLM readers. The Explore-then-Distill framework, realized through in-context stylistic exploration and dual-criteria denoising distillation, consistently delivers superior performance over strong baselines with negligible latency overhead. The findings underscore the necessity of optimizing both the presentation and substance of evidence to unlock the full potential of retrieval-augmented LLMs (2604.17325).