Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer

Abstract: Learning-to-Defer routes each input to the expert that minimizes expected cost, but it assumes that the information available to every expert is fixed at decision time. Many modern systems violate this assumption: after selecting an expert, one may also choose what additional information that expert should receive, such as retrieved documents, tool outputs, or escalation context. We study this problem and call it Learning-to-Defer with advice. We show that a broad family of natural separated surrogates, which learn routing and advice with distinct heads, is inconsistent even in the smallest non-trivial setting. We then introduce an augmented surrogate that operates on the composite expert--advice action space and prove an $\mathcal{H}$-consistency guarantee together with an excess-risk transfer bound, yielding recovery of the Bayes-optimal policy in the limit. Experiments on tabular, language, and multi-modal tasks show that the resulting method improves over standard Learning-to-Defer while adapting its advice-acquisition behavior to the cost regime; a synthetic benchmark confirms the failure mode predicted for separated surrogates.

• The paper introduces a decoupled surrogate that separates class and expert optimization to avoid gradient coupling and underfitting issues seen in augmented-action methods.
• It provides theoretical guarantees including O(1) calibration independent of the number of experts, ensuring stable optimization even with large expert pools.
• Empirical results on synthetic, image (CIFAR-10/10H), and tabular datasets validate improved deferral accuracy and balanced routing between classifier and experts.

Decoupling Class and Expert Optimization in Multi-Expert Learning-to-Defer

Introduction

Multi-expert Learning-to-Defer (L2D) extends standard selective classification and reject-option frameworks to systems where, for each instance, the model must dynamically choose between predicting itself or deferring to one of several experts, each with distinct behavior and accuracy. The canonical objective is to minimize expected error by selecting the optimal action—either outputting a class or routing to the expert most likely to be correct. The Bayes-optimal solution is a simple threshold rule leveraging both the model’s own class posterior and the conditional expert utility vector for each expert. However, practical learning must minimize a surrogate loss, necessitating surrogates that are population consistent and well-behaved in optimization.

This paper identifies a systematic architectural flaw in existing multi-expert L2D surrogates: the predominant “augmented-action” reduction, which conflates classes and experts in a shared action space and trains a unified score vector. While these surrogates are often population-wise consistent, they manifest strong underfitting, expert suppression, and scaling issues as the expert pool grows. Through theoretical and empirical analyses, the authors pinpoint a geometric coupling between class and expert updates as the root cause. They introduce a novel “decoupled surrogate” using separate softmax and sigmoid heads, which resolves previous shortcomings both statistically and in optimization, with strong supporting results on synthetic and real data.

Critique of Augmented-Action Surrogates

Augmented-action surrogates constitute the de facto approach in the L2D literature. These methods model the decision as an augmented $(K+J)$-way classification over $K$ classes and $J$ expert actions, optimizing a unified score vector $a(x)\in\mathbb{R}^{K+J}$. The Bayes-optimal rule, however, only involves $\max_k \eta_k(x)$ and $\max_j \alpha_j(x)$—the augmented-action reduction does more, imposing a joint geometry and shared normalization.

The paper demonstrates that all prominent surrogates in this family inherently trade off statistical and optimization properties:

• Additive Cross-Entropy (CE): The surrogate target is a normalization-distorted version of true Bayes quantities, shrinking estimated probabilities proportional to the sum of expert utilities $U(x)$. Sample gradients and Hessians are amplified by $1+|J|$, biasing updates toward redundancy-rich, easy regions and exacerbating conditioning as $J$ grows.
• PiCCE (winner-take-all): To mitigate amplification, PiCCE only reinforces the highest-scoring correct expert per sample. This suppresses rare/weak experts that fail to win the internal competition, a “starvation” effect breaking robustness to specialist expertise.
• A-SM (Asymmetric Softmax): This variant recovers correct class and expert targets with a proper categorical-softmax (for class) and sigmoid (for expert) split, but expert gradients still couple into class logits due to the parameterization, resulting in $O(J)$-scale expert-induced perturbations of class updates.
• Mao25 (set mass): This linear-in-acceptable-set surrogate provides strong theory but cannot distinguish between experts within the correct set (no intra-set ranking), resulting in diluted optimization and poor adaptation when expert quality varies.
• One-vs-All (OvA): By decoupling all $K$0 margins, OvA eliminates all optimization pathologies. However, the class side is now $K$1 independent sigmoids, so the class probability vector is not guaranteed to lie in the simplex $K$2, impairing class/expert comparison for routing.

Notably, for each surrogate, any repair of a per-sample effect (amplification, starvation, coupling) induces a failure elsewhere—either a statistical mismatch or a new geometric pathology. Empirically, performance degrades as $K$3 increases, with coverage and overall accuracy dropping, and rare/valuable experts frequently suppressed.

The Decoupled Surrogate: Formulation and Theoretical Guarantees

The authors propose a “decoupled surrogate” that factorizes the class and expert learning problems:

• The model comprises a class head (softmax, $K$4) and $K$5 expert sigmoid heads ($K$6).
• The surrogate loss is the sum of standard multiclass cross-entropy for the class head and $K$7 independent binary cross-entropies for the expert heads, each weighted by a constant $K$8.
• Routing is performed by a direct comparison: predict if $K$9, otherwise defer to the best expert.

The resulting training dynamics have several crucial characteristics:

• Strict Decoupling: All class and expert gradients and curvatures are block-diagonal; there is no class-expert coupling, no gradient amplification or starvation. Each target is optimized on its native scale.
• $J$0-Independent Calibration: The surrogate admits an $J$1-consistency bound where, for fixed $J$2, the calibration constant is $J$3 as $J$4 grows, in contrast to previous surrogates where the constant can grow at least as $J$5 or $J$6. Thus, theoretical performance guarantees do not degrade with larger expert pools.
• Proper Statistical Targets: The population minimizer estimates the true Bayes-optimal class posterior and expert utility vector, enabling correct plug-in policies for both classification and deferral.
• Robust Optimization: Unlike set-mass-only surrogates (e.g., Mao25), this approach preserves and reinforces both rare specialists and generalists as per their empirical performance, directly aligning with the multi-expert L2D objective.

Empirical Validation

The paper validates all insights via rigorous synthetic and real-world experiments:

• Synthetic Benchmarks: On controlled tasks designed to stress each surrogate’s failure mode (redundancy amplification, specialist starvation, set-mass ranking deficiency, class-expert coupling), only the decoupled surrogate maintains low deferral regret and proper ranking/unbiased utility estimation, while other surrogates degrade sharply.
• Image Classification (CIFAR-10): In experiments with synthetic experts of increasing redundancy, the decoupled surrogate is the only method to consistently improve upon the standalone classifier— all other surrogates (including A-SM and additive CE) increasingly degrade as $J$7 increases, confirming theoretical predictions. Classifier quality is preserved; coverage is balanced.
• Human Annotator Pool (CIFAR-10H): With real human experts, the decoupled surrogate achieves the highest system accuracy and is the sole method effectively preserving classifier utility when scaling to 20 experts—most baselines exhibit large variance and collapse.
• Tabular Data (Covertype): Routing among model experts (random forest, gradient-boosted trees, MLPs) demonstrates that only the decoupled surrogate both improves system accuracy and maintains classifier quality; others over-defer or collapse the classifier.

These findings demonstrate that only strict class-expert decoupling removes the structural impediments to scalable, robust multi-expert L2D, regardless of model, pool structure, or data modality.

Implications and Future Directions

The results have direct implications for the design of L2D and deferred decision systems at scale. Architectures that couple class and expert actions incur unavoidable tradeoffs between statistical fidelity and optimization geometry, leading to failure in high-expert or heavy-redundancy settings. The decoupled surrogate not only closes these gaps but provides a template for further advances in learning-to-defer frameworks, including applications to guided LLM routing, dynamic ensemble methods, and cost-sensitive settings.

This decoupling principle suggests that future research should treat deferrable actions as structurally distinct from prediction actions; this applies to broader decision theoretic frameworks where actions with different statistical semantics are present. The theoretical insight into $J$8-independent calibration constants also motivates the development of scalable L2D for very large expert pools, including over-parameterized weak learners, human-in-the-loop systems, or “expert agents” in AI teams. Moreover, the approach creates a path toward stable, interpretable, and reliable AI delegation in domains with heterogeneous expert performance profiles, such as healthcare, crowdsourcing, or financial modeling.

Conclusion

This work provides a definitive theoretical and empirical analysis of the augmented-action surrogate paradigm in multi-expert L2D. The decoupled surrogate is proven to resolve both statistical and optimization deficiencies by structurally separating class and expert learning while maintaining proper calibration and transfer guarantees. The thorough suite of analyses and experiments demonstrates that only strict decoupling scales to large, redundant, and heterogeneous expert pools without accuracy loss, rare expert suppression, or classifier collapse. The architectural principles and their implications presented herein should guide not only multi-expert L2D but the broader design of modular, delegation-ready learning systems (2604.09414).