L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification

Abstract: Clinical text classification requires choosing between specialized fine-tuned models (BERT variants) and general-purpose LLMs, yet neither dominates across all instances. We introduce Learning to Defer for clinical text (L2D-Clinical), a framework that learns when a BERT classifier should defer to an LLM based on uncertainty signals and text characteristics. Unlike prior L2D work that defers to human experts assumed universally superior, our approach enables adaptive deferral-improving accuracy when the LLM complements BERT. We evaluate on two English clinical tasks: (1) ADE detection (ADE Corpus V2), where BioBERT (F1=0.911) outperforms the LLM (F1=0.765), and (2) treatment outcome classification (MIMIC-IV with multi-LLM consensus ground truth), where GPT-5-nano (F1=0.967) outperforms ClinicalBERT (F1=0.887). On ADE, L2D-Clinical achieves F1=0.928 (+1.7 points over BERT) by selectively deferring 7% of instances where the LLM's high recall compensates for BERT's misses. On MIMIC, L2D-Clinical achieves F1=0.980 (+9.3 points over BERT) by deferring only 16.8\% of cases to the LLM. The key insight is that L2D-Clinical learns to selectively leverage LLM strengths while minimizing API costs.

• The paper introduces an AI-to-AI deferral framework that adaptively routes clinical texts between a domain-adapted BERT and an LLM based on uncertainty signals.
• It utilizes lightweight, interpretable features—such as softmax entropy and causal language cues—to effectively predict and manage model errors.
• This approach achieves enhanced F1 scores and up to a 93% reduction in LLM API calls, optimizing both accuracy and efficiency in clinical applications.

L2D-Clinical: Adaptive AI-to-AI Deferral for Model Selection in Clinical Text Classification

Motivation and Context

Clinical text classification is central to multiple healthcare NLP applications, including adverse drug event (ADE) detection and extraction of treatment outcomes from EHRs. The performance of specialized, domain-adapted BERT-based models (e.g., BioBERT, ClinicalBERT) and general-purpose LLMs is highly task-dependent, with neither exhibiting consistent superiority across tasks or instance types. Existing model selection heuristics generally operate at the task or dataset level and lack dynamic adaptation to instance-level variation. Conventional Learning to Defer (L2D) frameworks focus primarily on AI-to-human deferral, premised on the expert (human) being universally more reliable—a condition frequently invalid in machine-only clinical pipelines.

The work "L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification" (2604.13285) addresses this critical gap by proposing an L2D framework tailored for AI-to-AI deferral, wherein a domain-adapted BERT classifier learns when to defer to a general-purpose LLM. Unlike earlier L2D approaches, the "expert" (LLM) in this setting is not universally more reliable; the system adaptively learns deferral strategies based on uncertainty and text features, thereby exploiting complementary model strengths.

Methodology

L2D-Clinical comprises three core components: a base BERT classifier, an LLM deferral target, and a logistic regression-based deferral model trained to predict when BERT will err. The workflow proceeds as follows:

1. Base Prediction: The input clinical text is classified by a fine-tuned BERT variant (BioBERT for ADE, ClinicalBERT for outcomes) which outputs softmax probabilities.
2. Deferral Decision: The deferral model consumes uncertainty signals (softmax confidence, entropy, margin) and text-based linguistic features (e.g., presence of causal, severity, or outcome-related terms), predicting the probability that BERT's prediction will be incorrect.
3. Conditional Routing: For instances where the estimated error probability surpasses a threshold (tuned on validation data for optimal F1), the decision is deferred to an LLM (GPT-5-nano invoked via API with task-specific few-shot prompts).

The feature set for deferral is intentionally lightweight and interpretable, emphasizing deployment feasibility and clinical transparency. Logistic regression is empirically justified as it yields comparable F1 to more complex models while providing interpretable coefficients essential for deployment in clinical workflows.

Experimental Design

Two complementary clinical text tasks were chosen to characterize model dynamics:

• ADE Detection (ADE Corpus V2): Binary sentence classification for ADE presence, 23,516 sentences. BioBERT significantly outperforms the LLM (F1: 0.911 vs. 0.765).
• Treatment Outcome Classification (MIMIC-IV): Three-class classification of drug-outcome relations, 2,782 consensus-labeled triplets. Here, the LLM (GPT-5-nano) outperforms ClinicalBERT (F1: 0.967 vs. 0.887).

The consensus ground truth for MIMIC-IV leverages GPT-5.2 and Claude 4.5 agreement followed by spot human expert validation (100% agreement in a random 5% subset), maximizing label trustworthiness given annotation resource limitations.

Comparisons include BERT-only and LLM-only baselines, random and fixed-threshold deferral strategies, and an oracle (upper bound) baseline.

Results

ADE Detection: BERT $\boldsymbol{>}$ LLM

• L2D-Clinical achieves F1=0.928, a +1.7 point gain over BioBERT, despite the LLM's inferior standalone F1 (0.765). Deferral occurs on 7% of instances, which typically contain ambiguous or hedged language where BioBERT is uncertain and the LLM's high recall is beneficial.
• Error analysis indicates that residual errors are evenly split between cases where both models are wrong and false deferrals. Further improvement appears contingent on improved rare entity modeling or ensemble architectures.

Treatment Outcome Classification: LLM $\boldsymbol{>}$ BERT

• L2D-Clinical achieves F1=0.980 (+9.3 over BERT, outperforming the LLM as well) by deferring only 16.8% of cases. Its F1 exceeds that of a fixed-threshold approach (F1=0.971) while invoking the LLM less frequently (16.8% vs. 26.2%).
• Most errors involve ambiguous outcome statements or inherent label ambiguity; L2D-Clinical preserves the strengths of ClinicalBERT on explicit language while leveraging LLM strengths for implicit or context-sensitive cases.

Adaptive Deferral Behavior

L2D-Clinical exhibits adaptive deferral rates, low (7%) when the base BERT is stronger, and higher (16.8%) when the LLM offers better standalone accuracy. This empirically confirms that the system learns "when deferral helps" rather than adhering to a static or naive policy.

Feature Importance and Interpretability

BERT's entropy is the most significant predictor of error/deferral across both tasks. Task-specific features (e.g., presence of causal language or explicit outcome terms) also carry substantial weight, providing interpretable, actionable insights for clinical end-users and system maintainers.

Implications

Theoretical Implications

• Instance-level Model Selection: The study demonstrates that per-instance model selection using L2D is both feasible and beneficial even in AI-to-AI settings with non-uniform expert reliability. This elaborates on and partially generalizes L2D theory beyond human-in-the-loop domains.
• Complementary Error Profiles: Strong aggregate performance measures (e.g., highest macro F1) may still conceal segments of the input space where "weaker" models possess unique advantages.
• Deferral Signal Engineering: The importance and sufficiency of lightweight uncertainty and text features in reliable deferral calibration is affirmed, suggesting that expensive meta-learners are not necessarily required in specialized domains.

Practical Implications

• Resource Optimization: L2D-Clinical achieves 81–93% reduction in LLM API calls compared to pure LLM deployment, translating to substantial cost and latency savings for large-scale clinical NLP pipelines.
• Deployment Safeguards: Interpretability of deferral decisions and the capability for graceful degradation (fallback to BERT-only prediction in the event of LLM unavailability) are crucial for regulated clinical environments.
• Ground Truth Construction: The demonstrated LLM+human consensus labeling workflow provides a feasible template for rapid ground truth construction in domains where expert time is limited.

Limitations and Future Directions

L2D-Clinical's effectiveness is shown on moderate test sizes and English-only tasks; generalization to larger, more diverse datasets, additional model families (Llama, Mistral), and multi-language settings remains to be established. Extension to more complex tasks, online adaptation for distribution shift, and multi-model cascades, as well as rigorous bootstrapping under limited annotated data, are promising areas for future research.

Conclusion

L2D-Clinical operationalizes AI-to-AI adaptive deferral in clinical text classification, demonstrating consistent accuracy improvements and significant increases in computational efficiency. It substantiates that task- and instance-level complementarity between domain-adapted small models and LLMs can be automatically exploited through learned, interpretable routing decisions. This approach sets a precedent for adaptive, model-agnostic hybrid architectures in clinical NLP and related domains, facilitating robust, cost-efficient, and transparent AI deployments as the model and system landscape continues to evolve.

Reference: "L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification" (2604.13285)