- The paper introduces a class-aware dual Mixture-of-Experts architecture that enhances scleral anomaly segmentation in heterogeneous imaging environments.
- It employs dual-stream gated fusion, staged backbone adaptation, and a multi-expert decoder to tackle issues like data scarcity and artifact-induced leakage.
- Quantitative results demonstrate improvements in mDice (+2%), mBF1 (+3.14%), and effective generalization across clinical and unconstrained scenarios.
Class-Aware Hierarchical Dual Mixture-of-Experts for Robust Scleral Anomaly Segmentation
Introduction
The paper "HD-DinoMoE: A Class-Aware Hierarchical Dual Mixture-of-Experts Network for Scleral Anomaly Segmentation in Complex Acquisition Scenarios" (2606.04888) addresses pixel-wise scleral surface anomaly segmentation as a technical foundation for intelligent Traditional Chinese Medicine (TCM) ocular inspection. The work is motivated by unmet needs for robust multi-class segmentation under highly heterogeneous acquisition conditions—particularly those involving cross-source discrepancies, anomaly morphological diversity, and pronounced specular reflection artifacts. The authors introduce a full-stack pipeline, TAO, and focus on a class-aware hierarchical dual Mixture-of-Experts (MoE) network, validated on a new multi-label scleral dataset (ML-SASD) constructed by the authors.
Motivation and Problem Framing
Recent advances in computer vision for ophthalmic images have predominantly focused on fundus-based analysis or macro-anatomical segmentation, neglecting the fine-grained parsing of heterogeneous scleral surface anomalies consequential for both Western and TCM diagnostic paradigms. The pipeline deployment context is characterized by multi-source (clinical/prospective–wild/mobile) data, varying imaging artifacts (including strong SSR), and complex anomaly entities—Vessels (Ve), Yellow and Black Spots (YBS), and Blood Spots (BS)—with overlapping spatial distributions and ambiguous class semantics. The three primary challenges explicitly targeted are: (1) data scarcity for dense, multi-label anomaly annotations; (2) cross-distributional domain shift and generalization collapse; and (3) artifact-induced leakage, particularly from SSR occlusion.
Methodological Framework
Data: ML-SASD Construction
The paper presents ML-SASD, a comprehensive multi-label dataset with three anomaly categories and explicit SSR region annotations, covering both curated clinical images (controlled exposure, multi-gaze protocol) and wild/mobile acquisitions (unconstrained ambient conditions, device variety, participant diversity). The annotation protocol systematically supports overlapping semantic regions, enabling fine-grained, composite labeling appropriate for both clinical and TCM informatics research.
Model: HD-DinoMoE Design
The HD-DinoMoE is architected as a hierarchically class-aware dual-branch mixture-of-experts segmentation network, with four main innovations:
- Class-Aware Dual-Stream Gated Fusion Encoder (CA-DSGF): Dual parallel DINOv3-L backbones (SAT and LVD pretrainings) are fused at the feature level using MLP-driven, class-specific gating. This design selectively leverages strong topological encoding or texture-adaptive representation according to class, addressing both cross-source and class-specific visual heterogeneity.
- Three-Stage Backbone-Frozen Routing Alignment (TS-BFRA): Training is staged to sequentially adapt each DINOv3 backbone to the scleral anomaly domain, freezing the foundation weights and finally aligning gating and expert routing post-adaptation. This design is critical for stabilizing both dual-backbone fusion and downstream conditional expert utilization given strong overfitting risk with billions of parameters and limited annotation scales.
- Class-Specific Multi-Expert Decoder (CS-MED): Decoding employs a class-conditioned MoE, aggregating DPT, SAM-MLP, D2S, and LinearAttn experts per class with an MLP-based gating network, fusing 2D spatial reconstructions that emphasize distinct inductive biases (multi-scale structure, semantic context, detail, long-range dependencies). Various class-specialized and shared-projection/gating topologies are explored, revealing that class-specific encodings with shared multi-expert decoding generalize optimally under strong cross-image/class variation.
- Progressive Confidence Penalty (PCP) Loss and Class-Aware Adaptive Sample Weighting (CA-ASW): SSR regions are dynamically re-weighted in the loss function to penalize high-confidence false positives, shape gradient flow for artifacts in ambiguous regions, and suppress risky leakage. A sample-by-class historical loss matrix enables online adaptive reweighting of classes and cases, targeting hard examples via a global running median and further modulated by the entropy of expert gating.
Experimental Results
On ML-SASD-Mix, HD-DinoMoE achieved mDice 72.11%, mIoU 58.44%, and mBF1 41.40%. The mGFPR (specular reflection-induced false positive rate) was tightly controlled at 1.02%. Notably, gains over monolithic DINOv3 and hybrid DINO-U-Net models are systematic: compared to SegDINO-LVD (70.06% mDice), HD-DinoMoE provides a +2% absolute Dice improvement. The increase in mBF1 (boundary F1) (+3.14%) indicates substantially superior anomaly localization accuracy—a key requirement in precise ocular anomaly quantification.
Cross-Dataset and Generalization
Generalization was established on the public SBVPI dataset (Vessels subset), showing that HD-DinoMoE outperforms both classic CNNs and competitive foundation-model-based methods (Dice 49.13% vs. 48.54% for DINOUnet-SAT), with a marked gain in the boundary metric (BF1 +5.51%). These results confirm transferability across acquisition protocols, device types, and annotation styles.
Ablation and Architectural Studies
Comprehensive ablation validates:
- CA-DSGF: Marginal gains under single-source scenarios, but clear necessity for cross-source, class-conditional adaptation in mixed settings; the dynamic gating is critical for anomaly-specific feature fusion.
- CS-MED: Diverse expert ensemble is required to balance multi-scale, semantic, and fine-detail decoding, but fully independent class-specific projections/experts are superseded by shared (but class-aware) structures when coupled with class-specific encoded features.
- PCP Loss: Targeted, confidence-proportional penalization within SSR suppresses artifact-driven leakage with minimal cost to mIoU, optimizing the practical trade-off between anomaly recall and artifact robustness.
- CA-ASW: Sample- and class-adaptive weighting, particularly median-based balancing, enhances overall learning focus, especially effective during backbone-adaptation stages.
Implications and Forward Directions
Practical and Clinical:
The HD-DinoMoE architecture directly addresses the deployment gap in automated, quantifiable scleral surface analysis, a critical element in digital TCM and broader ocular diagnostics. The presented system is robust to distributional shift and real-world imaging artifacts, enabling scalable deployment on both clinical and user-acquired data. The design of ML-SASD as a multi-label framework encourages the community to move beyond simplistic, exclusive class assignments toward composite, TCM-aligned paradigms.
Theoretical and Algorithmic:
The model instantiates a fully class-aware, hierarchically modular vision pipeline, combining state-of-the-art foundation model representations, conditional computation, and adaptive optimization—demonstrating that multi-expert and multi-backbone approaches retain practical utility under real, small-to-medium-scale medical training regimes, provided careful staged adaptation and class-specific fusion mechanisms are employed.
Future AI Directions:
The results support further scaling of class-aware and sample-wise MoE architectures in medical vision, especially under resource constraints and annotation ambiguity. The explicit coupling of expert routing entropy and adaptive sample weighting is a technically promising mechanism for iterative, intrinsically explainable learning. Extending lightweight or distillation-based variants for mobile/edge deployment will enable widespread screening use cases. Additionally, the integration of segmentation outputs with downstream structured scoring and retrieval-augmented reporting modules (as in the TAO system) paves the way for interpretable, knowledge-augmented clinical decision support in TCM and beyond.
Conclusion
HD-DinoMoE advances the state-of-the-art for scleral surface anomaly segmentation through its class-aware dual-backbone/dual-expert design, targeted artifact handling, and robust adaptive optimization. The model yields significant gains in both quantitative and boundary-level metrics under complex, multi-source scenarios and demonstrates strong generalization to public benchmarks. Critically, the released ML-SASD dataset and accompanying codebase underpin transparent evaluation and future development in intelligent TCM ocular analysis. The methodological approach is broadly applicable to heterogeneous, artifact-prone medical vision tasks requiring robust, interpretable, and class-aware segmentation pipelines.
Reference:
"HD-DinoMoE: A Class-Aware Hierarchical Dual Mixture-of-Experts Network for Scleral Anomaly Segmentation in Complex Acquisition Scenarios" (2606.04888)
