Query-Adaptive Late Fusion (QAF)

• Query-Adaptive Late Fusion (QAF) is a paradigm that dynamically adjusts fusion weights or representations based on the specific query to enhance retrieval precision.
• It aggregates evidence from multiple sources post-scoring, adapting the fusion process per query to counteract feature noise and aspect imbalance.
• QAF is applied across modalities—including image retrieval, LLM adaptation, and review aggregation—improving recall and efficiency without full system retraining.

Query-Adaptive Late Fusion (QAF) is a class of techniques that perform evidence aggregation at the score or representation level in a manner that dynamically adapts fusion weights, feature usage, or underlying representations based on the current input query. This paradigm extends conventional late fusion—where multiple models, features, retrievers, or adapters are aggregated with static or globally-fixed rules—by allowing the fusion mechanism itself to be modulated by properties of the query, the available candidate set, or extracted query aspects. QAF methodologies appear across retrieval, LLM adaptation, visual and multispectral representation learning, and review-based aggregation, offering robustness to feature noise, aspect imbalance, and domain specialization while maximizing recall and relevance without incurring the computational costs of system-wide retraining or re-indexing.

1. Core Principles and Taxonomy

QAF frameworks share several consistent characteristics:

• Late Fusion Foundation: Aggregation or fusion is performed after initial, independent scoring or representation extraction, as opposed to joint or early fusion. Candidate lists or representations from multiple sources are merged at a decision or ranking stage.
• Query-Adaptivity: Fusion weights, expert routes, or feature importances are computed on a per-query basis, governed by either pre-defined quality measures, neural routing components, or side information extracted from the query.
• Per-Instance Modulation: Adaptive fusion targets the selection or blending of subsystems—features, retrievers, adapters, or modality-specific encoders—such that each query invokes an optimal integration strategy rather than fixed, global weighting.

QAF is instantiated in diverse modalities and contexts:

• Feature-level QAF for image retrieval, using per-query curve analysis to set weights (Wang et al., 2018).
• Fusion of dense/sparse retrievers using reciprocal-rank and query-adaptive adaptation (Jain et al., 8 Jan 2026).
• Query-conditioned backbone or expert mixtures for visual, multispectral, or document models (Abdallah et al., 3 Jun 2026, Guo et al., 2024).
• Layer-wise per-adapter weighting for LLM LoRA adapters (Shukla et al., 12 Dec 2025).
• Aspect-based decomposition and fusion in reviewed-item retrieval (Korikov et al., 2024).

2. Mathematical Formulations and Mechanisms

The following table organizes representative QAF approaches and their mathematical essence:

Domain	Fusion Inputs	Query-Adaptivity Mechanism
Image Retrieval	Feature-wise similarity	Score curve "elbow" analysis / learned weighting
Enterprise Search	Top-K from diverse retrievers	Reciprocal-rank fusion + query encoder adaptation
Visual Retrieval	Spatial-region features	Query-conditioned MoE router per region
LLMs/Adapters	LoRA/module outputs	Layer-wise KL divergence for per-adapter fusion weights
Review Aggregation	Review-level similarities	LLM-decomposed aspect fusion, aspect-wise score merging

Feature Quality Estimation

In unsupervised QAF for image retrieval, feature quality is inferred by analyzing the score curve shape for each feature: highly discriminative features yield "L-shaped" curves, and the area under the normalized curve $A_i(q)$ is used to compute weights $w^{(i)}_q$ for sum or product fusion (Wang et al., 2018). In learning-based variants, this mapping from curve to weight is itself learned as a small neural network, trained to directly optimize retrieval margin.

Candidate Generation and Score Fusion in IR

Multi-retriever QAF gathers top-K candidates from each of $M$ retrievers. For each candidate $d$, a score is computed via reciprocal-rank fusion (RRF):

$$S(q,d)\;=\;\sum_{i=1}^M \frac{1}{k + \mathrm{rank}_i(q,d)}$$

with ranking determined per retriever instance, and $k$ a tunable dampening constant. The fused score list is query-dependent due to the retriever rankings and further downstream adaptation in the query encoder via LoRA (Jain et al., 8 Jan 2026).

Query-Adaptive Expert Mixtures

In visual document retrievers (e.g., Argus), region features are passed through a router module, which incorporates the L2-normalized pooled query context vector $z_q$. The router computes per-region input:

$u_i = R_i + W_{coord}c_i + W_{query}z_q$

resulting in soft, query-dependent mixing weights for a bank of expert sub-networks, which transform region token representations accordingly. The document embedding thus becomes a function of $q$, enabling true query-adaptive fusion while retaining a multi-vector index (Abdallah et al., 3 Jun 2026).

Adapter Fusion in LLMs

For parameter-efficient adaptation in LLMs, multiple LoRA modules can be fused per query by quantifying the distributional divergence at each layer between the base and adapter models' projected vocabulary distributions:

$$\mathrm{div}_j^{(\ell)}(q) = D_{KL}(p^{(\ell)} \| q_j^{(\ell)})$$

$w^{(i)}_q$0

$w^{(i)}_q$1

This framework enables training- and data-free fusion that is responsive to the semantics of $w^{(i)}_q$2 (Shukla et al., 12 Dec 2025).

Aspect-Based Query Decomposition

In review-based retrieval, an LLM decomposes the query into aspects $w^{(i)}_q$3, each yielding an embedding. For each aspect, the system scores reviews, selects the top-K, and fuses item-level aspect scores, typically via mean:

$w^{(i)}_q$4

This score is more robust to aspect-frequency or separation biases, and adaptively ensures coverage across query facets (Korikov et al., 2024).

3. Practical Instantiations

DevRev Search QAF Pipeline

The DevRev Search system illustrates a multi-stage QAF pipeline for large-scale technical support retrieval (Jain et al., 8 Jan 2026):

1. Multi-retriever late fusion: Diverse retrievers supply top-K candidates, fused via RRF.
2. LLM-as-judge filtering: Each (query, doc) pair is classified by an LLM. A hard threshold removes likely negatives.
3. Index-preserving adaptation: Only the query encoder is updated via LoRA; the document index is fixed.
4. Contrastive loss training: InfoNCE loss is computed over positives/negatives selected from the QAF candidate and filtering stages.

Empirically, this approach yields recall@10 improvements from ~60% (zero-shot) to ~91% with query-only QAF adaptation, surpassing joint retriever/document fine-tuning while obviating the need for expensive re-indexing.

Argus QAF for Visual Documents

Argus retriever implements a query-conditioned MoE layer for visual document retrieval, where region encoding is dynamically routed according to query context. Removal of query conditioning leads to a 3–5 point drop in NDCG, underscoring the necessity of query adaptivity. Argus achieves new open-model SOTA on ViDoRe with substantially lower embedding dimension and training fraction relative to baselines, and its approach preserves compatibility with existing late-interaction pipelines (Abdallah et al., 3 Jun 2026).

Review Item Retrieval with QAF

Aspect Fusion for reviewed-item retrieval leverages LLM extraction of query sub-aspects and fusion functions robust to aspect/coverage imbalance. It increases MAP@10 from 0.36 to 0.52 under category skew, with similar performance on balanced data. Reranking further boosts performance, particularly when supplied with well-balanced review–aspect selections (Korikov et al., 2024).

Multispectral Object Detection

DAMSDet’s QAF spans object-level, per-modality query selection and deformable cross-attention, refactoring each detection query to dynamically sample from IR and visible streams depending on object saliency and appearance (Guo et al., 2024). This approach is proven robust to misalignment and complementary modality signals.

Query-Adaptive LoRA Fusion

qa-FLoRA offers a scalable and interpretable approach to adapter fusion, leveraging per-query, per-layer KL divergence-based weights. It closes most of the gap to supervised fusion routers with zero additional training or data, and outputs distinct adapter contributions across layers (e.g., domain adapters peaking mid-model, language adapters late) (Shukla et al., 12 Dec 2025).

4. Experimental Evidence and Quantitative Impact

The table summarizes selected quantitative gains attributable to QAF adoption:

System/Domain	QAF variant	Baseline	QAF Score	Noted Gain (%)
DevRev Search (Retrieval)	Query encoder LoRA adap.	60% R@10	91% R@10	+31 (vs. zero-shot)
SciFact (Retrieval)	Query encoder LoRA adap.	70% R@10	91% R@10	+21 (vs. zero-shot)
Argus (Visual ViDoRe V1+V2)	MoE query-conditioning	64.1 NDCG@5	86.0 NDCG@5	+22 (vs. baseline)
Review Retrieval (Imbalance)	Aspect Fusion	0.36 MAP@10	0.52 MAP@10	+0.16 absolute (+44%)
LLM LoRA Fusion (Math/Code)	Layerwise KL weighting	0.63 (static)	0.68	+5 (vs. static, LLaMA-2)

These results reflect not only improvements under standard metrics, but also robustness to feature noise (image retrieval), aspect-popularity bias (review aggregation), domain and language hybridization (LoRA fusion), and the operational advantage of avoiding costly re-indexing or model retraining.

5. Theoretical and Practical Implications

QAF demonstrates a general recipe for robust, adaptive aggregation in complex retrieval and representation problems:

• Robustness: Adaptive fusion detectably suppresses detrimental or noisy features and enhances signal from highly discriminative cues.
• Flexibility: QAF can function both in data-rich (supervised curve/adapter weighting) and resource-constrained (data-free KL divergence, codebook referencing) settings.
• Scalability: Modular, per-query adaptation sidesteps combinatorial explosion in multi-domain or multi-aspect support. For LoRA-based LLMs, no additional routers nor composite-task data are required, contrasting with static or supervised routes.
• Interpretability: Layer- and query-level weighting provides insight into system routing behaviors, especially for adapter fusion.

Limitations include diminished effect when all base features or adapters are uniformly strong/weak, and a dependency on the availability or quality of auxiliary information (e.g., aspect extraction, reference codebooks).

6. Extensions and Emerging Directions

Extension and open avenues emerging from QAF literature include:

• Automated codebook construction for unsupervised QAF that tracks evolving distributions (Wang et al., 2018).
• Direct integration of LLMs as fusion or reranking agents for zero-shot adaptivity (Korikov et al., 2024).
• Further formalization of expert routing, including alternate divergence metrics and query modulation targets for adapter fusion (Shukla et al., 12 Dec 2025).
• Cross-modal QAF pipelines combining text, image, and multi-sensor data (e.g., DAMSDet for infrared-visible, Argus for layout-sensitive documents) (Abdallah et al., 3 Jun 2026, Guo et al., 2024).
• End-to-end trainable systems coupling retrieval, adaptive fusion, and downstream task heads.

A plausible implication is that as the number of specialized modules or retrievers per system grows, QAF will be indispensable for effective, computationally manageable, and interpretable deployment across hybrid or rapidly evolving domains.