Balance reasoning performance with inference efficiency in RAG

Determine effective approaches to balance reasoning performance with inference efficiency when integrating reasoning large language models into Retrieval-Augmented Generation systems for multi-hop question answering, especially under latency-sensitive deployment constraints.

Background

The paper studies how reasoning models (e.g., o1, DeepSeek-R1, Qwen) improve multi-hop QA within Retrieval-Augmented Generation but at the cost of increased tokens and latency due to explicit intermediate reasoning steps. In real-world applications, especially those requiring low latency, the trade-off between stronger reasoning and efficient inference is critical. The authors highlight that achieving a practical balance remains unresolved and propose LiR^3AG as a lightweight framework to mitigate this trade-off.

This open problem frames the broader challenge of designing systems that retain the benefits of reasoning-enhanced RAG while keeping inference tractable. It motivates techniques that reduce redundancy, improve evidence structuring, and optimize token usage without sacrificing answer quality.