Retrieval-Augmented Learning-to-Match (RAL2M)

• RAL2M is a framework that integrates retrieval with binary LLM matching to prevent hallucinations and enforce compliance.
• It employs a query-adaptive latent ensemble to robustly aggregate LLM judgments and improve decision accuracy.
• Empirical evaluations demonstrate significant accuracy gains and reduced hallucination compared to standard generative pipelines.

Retrieval-Augmented Learning-to-Match (RAL2M) is a framework for constructing compliance-guaranteed, hallucination-resistant AI service systems by tightly coupling information retrieval with robust LLM matching and ensemble calibration. Unlike standard generative pipelines that rely on free-form LLM generation, RAL2M repositions LLMs as binary relevance (or eligibility) judges on retrieved candidates, yielding an explicit guarantee against generation hallucination while maximizing decision accuracy through retrieval grounding and ensemble learning. The approach is instantiated in both open clinical settings (LLM-Match) and compliance-critical QA systems, leveraging latent variable modeling to aggregate heterogeneous LLM judgments.

1. Architectural Overview

RAL2M consists of three fundamental components: a retrieval module, LLM-based matching judges, and a query-adaptive latent ensemble aggregator. The core design objective is to eliminate hallucination and ensure regulatory compliance by constraining output to grounded, human-verified responses or fallback abstention (“I don’t know”). The pipeline is as follows (Hong et al., 6 Jan 2026):

• Given a user query $q$, the retrieval module identifies the top-$k$ candidate Q&A pairs $(q', a')$ from a human-curated knowledge base (KB), along with supporting evidence from unstructured data repositories.
• Each candidate $(q', a')$ is evaluated by multiple LLMs, each acting strictly as a binary judge: $s_i(q, (q', a')) \in \{0, 1\}$, indicating “Yes” or “No” for perfect alignment.
• Judgments are aggregated via a query-adaptive energy-based latent ensemble, which explicitly models latent LLM competences and their dependencies.
• Only answers with predicted relevant matches are surfaced; otherwise, the system returns a safe fallback.

This protocol effectively precludes both generation hallucination and “judgment hallucination” (spurious alignments) by systematically grounding and calibrating LLM outputs (Hong et al., 6 Jan 2026).

2. Retrieval Module

The retrieval stage grounds user queries in both structured and unstructured information, providing the context required for precise matching. In clinical applications (LLM-Match (Li et al., 17 Mar 2025)), a retrieval-augmented generation (RAG) framework is used:

• Patient records are split into chunks $\{n_i\}_{i=1}^{|N_p|}$, each embedded via a domain-specific encoder $M$ (BioBERT).
• Eligibility criteria $C$ for a clinical trial are robustly encoded and aggregated: $v_C = \frac{1}{|C|} \sum_{c \in C} v_c$.
• Chunks are scored by cosine similarity $s(n_i, C)$ and top-$k$0 items are retained.
• Embeddings are stored in efficient vector indices (e.g., FAISS) for scalable nearest-neighbor search.
• In compliance systems (Hong et al., 6 Jan 2026), the structured KB $k$1 contains $k$2 verified $k$3 pairs, while the evidence base $k$4 ($k$5 documents) supplements context.

This infrastructure ensures that all LLM judgments are based exclusively on high-quality, pre-vetted information, a prerequisite for compliance guarantees.

3. LLM-Based Matching Judgers

Within RAL2M, LLMs are exclusively tasked with zero-shot, deterministic binary judgment—never free-form generation. The input prompt is standardized for meticulous QA evaluation, demanding strictly “Yes” or “No” responses (Hong et al., 6 Jan 2026).

• Each judge $k$6 provides $k$7 on QA pair alignment.
• Prompts have the form: system instructions, original user query $k$8, candidate $k$9, and associated evidence.
• In LLM-Match (Li et al., 17 Mar 2025), prompts incorporate both the eligibility criteria and top-$(q', a')$0 patient context chunks in a structured format.
• A classification head atop the LLM consumes the hidden representation $(q', a')$1 (e.g., $(q', a')$2 token), yielding $(q', a')$3, optimized with labeled data and binary cross-entropy.

By using multiple LLMs (e.g., Qwen, Mistral, Llama, GPT-5 for ablation), the system leverages model heterogeneity to mitigate single-model idiosyncrasies or failures.

4. Query-Adaptive Latent Ensemble

The aggregation of LLM judge outputs is realized via a query-adaptive latent energy-based graphical model (Hong et al., 6 Jan 2026). The ensemble mechanism explicitly incorporates:

• Latent competence variables $(q', a')$4 representing the reliability of each judge, governed by a query-conditioned Gaussian prior $(q', a')$5.
• Observed votes $(q', a')$6 interact with $(q', a')$7 via a compatibility potential.
• A learned gate $(q', a')$8 (soft attention over judges) and consensus score $(q', a')$9 are used for final label determination.
• The global energy decomposes into context, compatibility, and interaction terms:

$(q', a')$0

• Inference employs fixed-point iterations to estimate $(q', a')$1, followed by sampling and scoring to yield calibrated probability $(q', a')$2 and decision $(q', a')$3.

This approach outperforms simple majority or static weighting, as it explicitly models query specificity and inter-model error correlations. It implements an energy-based calibration analogous to a Mixture-of-Experts, where per-judge weights are softmax functions of query features but enhanced by richer potentials modeling judge interactions.

5. Training Objectives and Theoretical Guarantees

RAL2M is optimized by focal binary cross-entropy:

$(q', a')$4

with an additional annealed KL regularization on $(q', a')$5 to enforce the context potential. Joint training with gradient-based optimizers (e.g., Adam) updates both LLM parameters (if fine-tuned) and ensemble network components (Hong et al., 6 Jan 2026, Li et al., 17 Mar 2025).

The primary theoretical guarantee is compliance: as the system does not generate any new text but only selects or rejects from pre-verified candidates, generation hallucination is eliminated by construction. The explicit modeling of LLM judge competence and error dependencies further minimizes judgment hallucination.

6. Empirical Evaluations and Performance

Performance is demonstrated across QA and clinical trial matching benchmarks (Hong et al., 6 Jan 2026, Li et al., 17 Mar 2025):

System	Accuracy	Hallucination Rate	Macro-F1 / Precision
Retrieval-only	53%	38%
Single LLM	55%	50–60%
Majority Vote	60.2%	49.2%
Latent Ensemble	70.7%	13.9%	0.634 / 0.730

• In clinical trial matching (LLM-Match), Macro-F1 on n2c2 is 0.87 (prior SOTA: 0.75), and average AUROC on SIGIR/TREC is 0.8157 compared to 0.7979 for GPT-4-based baselines (Li et al., 17 Mar 2025).
• Providing additional evidence context improves some judges’ accuracy but can introduce distraction if context length is excessive.
• Inter-judge vote correlations are heterogeneous, supporting the need for explicit latent modeling.
• The model’s performance saturates after observing approximately 70–90% of labeled training queries.

Additional large-scale evaluation (e.g., proprietary Chinese QA tasks) shows accuracy up to 95% with minimal hallucination, compared to 80–83% for other approaches.

7. Best Practices and Future Directions

Experiments highlight the value of prompt tuning (with zero-shot instructions outperforming few-shot variants), judicious model selection (lightweight open-source LLMs suffice if modeled with latent interactions), and moderate data requirements (~50% of the domain’s queries are sufficient for competitive calibration).

Promising future directions include reducing inference calls for efficiency, supporting incremental knowledge base updates, adopting richer latent structures (multi-label, continuous outputs), and extending the approach to domains with sparse observable signal but latent matching cues (Hong et al., 6 Jan 2026).

A plausible implication is that RAL2M’s restriction of LLMs to calibrated binary matching, combined with dense retrieval and latent aggregation, offers a scalable template for compliance-critical systems seeking robust, explainable, and hallucination-free automation.