Choosing the Number of QConfig References per Query for Prompt Augmentation

Ascertain the optimal number of historical query-configuration (QConfig) references to include per query in Booster’s prompt augmentation to maximize recommendation quality while managing context length and inference overhead for large language models.

Background

Booster retrieves and includes top-k downstream QConfigs to guide the LLM’s per-query recommendations. The authors observe that using more diverse artifacts can improve outcomes but may also introduce conflicts and higher inference costs.

They explicitly defer deciding how many QConfigs to include per query, pointing to the need for principled selection strategies that account for relevance, diversity, and context-window limitations.