Fixed output token limit for retrieval-augmented reasoning with variable retrieval depth

Determine the fixed output token limit to use when adapting length-controlled generation methods such as L1 and S1 to retrieval-augmented reasoning settings in which the retrieval depth varies across queries, so that the choice of a fixed limit remains appropriate despite variable amounts of retrieved content.

Background

The paper reviews length-control approaches for reasoning-oriented LLMs, highlighting S1 (which regulates length by forbidding certain tokens) and L1 (which uses reinforcement learning to satisfy user-specified length constraints). These methods can control output length but often at the cost of performance and have not been developed in the context of retrieval augmentation.

In retrieval-augmented reasoning, the number of retrieved documents—and thus the number of retrieved tokens—can vary by query, making a single fixed output limit potentially inappropriate. The authors explicitly note uncertainty about how to set such a fixed limit when retrieval depth is variable, identifying a concrete unresolved question for integrating length-controlled methods into dynamic retrieval contexts.