CMedTEB & CARE: Benchmarking and Enabling Efficient Chinese Medical Retrieval via Asymmetric Encoders

Abstract: Effective medical text retrieval requires both high accuracy and low latency. While LLM-based embedding models possess powerful retrieval capabilities, their prohibitive latency and high computational cost limit their application in real-time scenarios. Furthermore, the lack of comprehensive and high-fidelity benchmarks hinders progress in Chinese medical text retrieval. In this work, we introduce the Chinese Medical Text Embedding Benchmark (CMedTEB), a benchmark spanning three kinds of practical embedding tasks: retrieval, reranking, and semantic textual similarity (STS). Distinct from purely automated datasets, CMedTEB is curated via a rigorous multi-LLM voting pipeline validated by clinical experts, ensuring gold-standard label quality while effectively mitigating annotation noise. On this foundation, we propose the Chinese Medical Asymmetric REtriever (CARE), an asymmetric architecture that pairs a lightweight BERT-style encoder for online query encoding with a powerful LLM-based encoder for offline document encoding. However, optimizing such an asymmetric retriever with two structurally different encoders presents distinctive challenges. To address this, we introduce a novel two-stage training strategy that progressively bridges the query and document representations. Extensive experiments demonstrate that CARE surpasses state-of-the-art symmetric models on CMedTEB, achieving superior retrieval performance without increasing inference latency.

• The paper introduces CMedTEB—a comprehensive benchmark—and CARE, an asymmetric retrieval framework that pairs a lightweight query encoder with a powerful offline document encoder.
• It employs a novel two-stage training strategy combining InfoNCE and MSE losses to achieve robust semantic alignment between query and document representations.
• Empirical results show CARE-0.3B-8B outperforms larger symmetric models by narrowing accuracy gaps while delivering over 9× higher throughput.

CMedTEB & CARE: A Comprehensive Benchmark and Efficient Asymmetric Architecture for Chinese Medical Retrieval

Introduction

This work addresses two critical limitations in Chinese medical text retrieval: (1) the absence of a robust, high-fidelity, and comprehensive benchmark; (2) the prohibitive online inference costs of state-of-the-art LLM-based embedding models inhibiting real-time use. The authors introduce CMedTEB, a benchmark spanning three key tasks with high-quality annotations, and propose CARE, an asymmetric retriever pairing a lightweight BERT-style encoder for queries with a powerful LLM-based encoder for documents. The CARE architecture is optimized with a novel two-stage training strategy for deep semantic alignment.

CMedTEB Benchmark: Data Construction and Analysis

The CMedTEB benchmark is devised to overcome issues of annotation sparsity and false negatives endemic in existing datasets like C-MTEB and CMIRB. The authors quantify that legacy benchmarks exhibit a high number of false negatives per query (up to 19/query in CmedqaRetrieval), validated by LLM and human annotators. CMedTEB’s construction leverages a multi-LLM consensus pipeline, where only documents that receive unanimous positive judgments from three distinct LLMs are retained, and clinical domain experts validate the output. The resulting datasets are non-trivial, displaying significant performance drops for leading general-purpose embedders, and are strongly distinguished from legacy datasets via low rank correlation.

Figure 1: CMedTEB’s construction workflow, highlighting distinct curation processes for Retrieval/Rerank and STS tasks.

CMedTEB’s diagnostic analysis demonstrates high annotation reproducibility (Fleiss’ Kappa of 0.731 among LLMs, 93.3% agreement with clinicians). It consists of three primary tasks:

• Retrieval: Multi-LLM voting on a candidate pool with a strict unanimous agreement.
• Rerank: Positive/negative pools derived from the retrieval phase for fine-grained reranking.
• STS: Medical synonym pairs generated with expert-built synonym dictionaries and validated by GPT-4o.

Analysis of baseline model generalization reveals that existing large embedders, while performant on CMedQA, experience up to a 27-point accuracy drop on CMedTEB’s new subtasks, establishing its necessity for robust evaluation.

CARE: Asymmetric Architecture and Training Paradigm

CARE leverages architectural asymmetry: a lightweight, fast query encoder for online serving and a large, powerful document encoder for offline embedding. Document representations are pre-computed, allowing for real-time query processing without the high latency of symmetric LLM-based dual encoders.

Figure 2: Inference and training pipeline of CARE, showing two-stage training: query encoder alignment followed by joint fine-tuning.

Key innovations include:

• Independent Pretraining: Both query and document encoders are pre-trained separately with contrastive objectives tailored to the medical domain.
• Matryoshka Representation Learning (MRL) aligns document embedding dimensions to query encoder space for compatibility.
• Stage I – Query Encoder Alignment: The document encoder is frozen; the query encoder is trained using both asymmetric InfoNCE and MSE losses. A self-contrastive (unsupervised) approach over massive unlabeled data robustly bridges the semantic gap.
• Stage II – Joint Fine-tuning: Both encoders are then jointly optimized with domain-specific retrieval triplets using asymmetric contrastive loss, refining end-to-end retrieval quality.

Empirical Results

CARE delivers state-of-the-art performance, uniquely achieving LLM-scale retrieval accuracy without the associated online latency.

Figure 3: CARE uniquely breaks the efficiency-performance trade-off, combining BERT-level QPS and LLM-level retrieval quality.

• CARE-0.3B-8B achieves an average CMedTEB score of 78.94, outperforming the 1.78B symmetric baseline (77.61), with a $9\times$ higher throughput.
• Scaling the document encoder from 4B to 8B yields further gains without affecting online latency due to offline document embedding.
• Asymmetric vs Symmetric: CARE-0.3B-8B achieves only a 0.6% average retrieval gap to a fully symmetric 8B dual encoder, while reducing online inference parameters by $>27\times$.

  Figure 4: CARE bridges the performance-efficiency gap relative to symmetric architectures, scaling quality via offline encoder.

• Comparison with alternative efficient retrieval architectures (such as KALE, ScalingNote, and distillation-based models) confirms CARE’s consistent superiority (by $>2$ points), highlighting the benefit of progressive semantic alignment over simple distillation or layer pruning.

Ablation and Architectural Analysis

Ablations confirm:

• Independent pretraining for query and document encoders is critical; omitting either halves performance.
• Both query encoder alignment and joint fine-tuning stages are necessary—bypassing either causes up to 13-point drops.
• The combination of InfoNCE and MSE objectives in alignment yields the best embedding space compatibility.
• Unsupervised self-contrastive alignment (Stage I) provides better fine-tuning foundations than direct supervised alignment, potentially due to greater variety preventing premature convergence.

Implications and Future Directions

The asymmetric CARE framework redefines efficiency-accuracy tradeoffs in specialized retrieval. It demonstrates that, by decoupling query and document computation and aligning via two-stage progressive training, domain-specific retrieval systems can be both highly performant and suitable for real-time deployment.

Practically, CARE presents an immediately deployable solution for industrial and clinical Chinese medical retrieval systems, as its offline-heavy design does not degrade user-facing latency. The open-sourced CMedTEB establishes a new standard for benchmarking in the domain, and its rigorous annotation pipeline provides a template for other high-stakes, knowledge-intensive settings.

Theoretically, this work highlights the importance of multi-stage, semantically deliberate alignment for leveraging heterogeneous model architectures in IR. It invites further research into scaling asymmetric designs, extending such frameworks to other languages and tasks, and theorizing optimal allocation of parametric capacity between offline and online paths in retriever systems.

Conclusion

CMedTEB & CARE establish a rigorous standard and efficient architecture for Chinese medical text retrieval. CARE achieves LLM-level accuracy with BERT-level online latency through asymmetric design and a novel two-stage alignment strategy. This work lays a foundation for future research in domain-specific, low-latency embedding systems and provides critical resources for reproducible and robust benchmarking in medical IR.