Query-Configuration Contexts (QConfigs)

• Query-Configuration Contexts (QConfigs) are a framework that maps individual queries to optimal IR system configurations using query-specific features.
• They employ a risk-sensitive candidate selection process to balance redundancy and computational cost, yielding a compact set of high-value configurations.
• A nearest-neighbor mapping mechanism efficiently assigns configurations per query, achieving 15-20% improvements in retrieval metrics across benchmarks.

A query-configuration context (QConfig) captures the mapping between a specific query and an information retrieval system configuration, operationalizing adaptive system behavior by selecting configurations tailored to the features of each individual query. This concept underpins risk-sensitive, per-query adaptation in modern retrieval systems, aiming to maximize effectiveness with minimal configuration redundancy and computational cost (Mothe et al., 2023).

1. Conceptual Foundations

Traditional information retrieval systems select a single, globally optimized configuration—consisting of retrieval models, expansion strategies, and hyperparameters—via grid search on validation queries. QConfigs depart radically from this static paradigm, modeling the relationship between query-specific characteristics and system behavior directly. This allows systems to dynamically choose the most suitable configuration from a carefully selected candidate set, for each incoming query, based on measurable query features.

The QConfig framework encompasses:

• The feature vector or context describing a query (e.g., LETOR features, length, ambiguity indicators).
• The configuration: parameterization of the IR system (retrieval model, expansion setting, ranking hyperparameters, etc.).
• The mapping mechanism: a function or procedure (often based on similarity in the query feature space) that assigns an optimal configuration to a given query.

2. Risk-Sensitive Configuration Subset Selection

A critical challenge in deploying QConfigs is determining a tractable set of candidate configurations from the combinatorial explosion of all possible system parameter combinations (often >20,000). The approach addresses this by introducing a risk-sensitive, incremental selection method that balances redundancy, coverage, and computational feasibility.

Risk and Reward Measures

Let $\mathcal{Q}$ be the set of training queries, and $p(c, q)$ be the effectiveness (e.g., P@10) of configuration $c$ on query $q$. For the candidate set $S_{k-1}$ already selected and a candidate configuration $c_k$:

• Effectiveness-based Risk:

$$E_{Risk}(c_k, S_{k-1}) = \frac{1}{|\mathcal{Q}|} \sum_{q_i \in \mathcal{Q}} \max\big(0, \max_{c_j \in S_{k-1}} p(c_j, q_i) - p(c_k, q_i)\big)$$

• Effectiveness-based Reward:

$$E_{Reward}(c_k, S_{k-1}) = \frac{1}{|\mathcal{Q}|} \sum_{q_i \in \mathcal{Q}} \max\big(0, p(c_k, q_i) - \max_{c_j \in S_{k-1}} p(c_j, q_i)\big)$$

• Combined Gain (with risk-reward tradeoff parameter $\beta$):

$$E_{Gain}(c_k, S_{k-1}) = E_{Reward}(c_k, S_{k-1}) - (1+\beta) E_{Risk}(c_k, S_{k-1})$$

A similar formulation exists for query-count-based risk and reward.

Configurations are selected greedily: at each step, add $c^*_k = \arg\max_{c_k} E_{Gain}(c_k, S_{k-1})$ to the candidate set.

This produces a small set ($K \approx 20$) of highly complementary, high-value configurations, drastically reducing overhead with minimal risk of omitting essential configurations.

3. Per-Query Configuration Assignment via Query Feature Matching

Rather than relying on complex learning-to-rank (L2R) models or exhaustive grid search, the assignment mechanism is predicated on measuring similarity in the query feature space. For each new query:

• Extract its feature vector.
• Compute cosine similarity against the feature vectors of training queries.
• Assign the configuration associated with the most similar query.

In training, each query is assigned its highest-performing configuration from the risk-sensitive candidate set.

This nearest-neighbor approach for mapping queries to configurations is empirically shown to outperform heavier-weight ML models for this task, achieving robust gains across ad hoc and diversity-oriented retrieval scenarios.

4. Trade-offs: Configuration Set Size, Effectiveness, and Efficiency

Empirical analysis reveals a sharp trade-off governed by the size of the risk-selected configuration subset:

• Increased set size improves the upper bound of achievable per-query effectiveness (nDCG@10, P@10), but with diminishing returns and higher operational cost.
• Excessive set size risks overfitting and computational inefficiency.
• Empirically optimal: A risk-sensitive set of $\sim$20 QConfigs yields a 15% improvement in P@10 and nDCG@10 over single-configuration (grid search) systems, and a 20% improvement over L2R baseline document models, with computational efficiency and maintainability [Figure 1, (Mothe et al., 2023)].

Aspect	Traditional (Grid, L2R)	Risk-Sensitive QConfig Approach
Config set size	1 or $\gg$1,000	$\sim$20 (selected for complementarity)
Per-query adaptation	No/Limited	Yes (via feature similarity)
Effectiveness gain	Base	+15% (grid); +20% (L2R)
Efficiency/maintenance	Good/Poor	Excellent

5. Evaluation: Datasets, Metrics, and Empirical Findings

The approach is validated across six TREC benchmarks, including ad hoc (e.g., GOV2, TREC78, MS MARCO) and diversity-focused tasks (ClueWeb09B+12B). Rigorous 2-fold cross-validation is used; metrics include Precision@10, nDCG@10, AP, ERR-IA@20, and RBP.

Key findings:

• The risk-sensitive QConfig pipeline consistently improves P@10 and nDCG@10 by ~15% vs. traditional configurations, and ~20% vs. L2R.
• The nearest-neighbor query–configuration mapping method is robust against both shallow and deep relevance assessment regimes.
• Simplicity and transparency of the pipeline ensure maintainability and operational scalability suitable for deployment in large retrieval systems.

6. Significance and Implications

The QConfig formalism provides a concrete operationalization of adaptive, per-query system configuration in IR. It demonstrates that:

• A modest, risk-aware, and carefully curated set of configurations suffices for high-performing per-query adaptation.
• Query feature similarity is a lightweight, effective, and robust mechanism for mapping queries to configurations, often outperforming more elaborate models.
• The method is broadly applicable: it generalizes across ad hoc and diversity-oriented retrieval, and admits straightforward scalability.

This body of work establishes QConfigs as a foundational component for scalable, risk-sensitive, and maintainable adaptive IR systems, and motivates future research in context-aware system adaptation, risk-aware pruning, and cost-effective deployment strategies in high-throughput environments (Mothe et al., 2023).