DualGate-Net: A Prior-Gated Dual-Encoder Framework for Histopathology Cell Detection

Abstract: Cell detection in histopathology images strongly depends on surrounding tissue context, where visually similar cells may belong to different classes under different microenvironments. Recent tissue-aware methods incorporate contextual priors, but often rely on static fusion strategies that may propagate noisy information. In this work, we propose DualGate-Net, a prior-aware dual-encoder framework that combines a ConvNeXtV2-based local encoder and a SegFormer-based global encoder through a learnable prior-gated fusion mechanism. The proposed module adaptively regulates the influence of tissue priors across spatial locations, while an auxiliary foreground reconstruction branch preserves high-frequency cellular structures during training. In addition, auxiliary cellness-guided cues are incorporated to further improve localization robustness. Experiments on the OCELOT benchmark demonstrate consistent improvements, achieving macro F1-scores of 0.7722 on the validation set and 0.7345 on the test set, highlighting the effectiveness of adaptive prior integration for robust histopathology cell detection.

• The paper introduces a dual-encoder framework that adaptively integrates local (ConvNeXtV2) and global (SegFormer) representations using a novel prior-gated fusion module.
• It employs a spatially adaptive prior gating mechanism and an auxiliary foreground reconstruction branch to preserve cellular morphology and improve detection in complex histopathology images.
• Experimental results on the OCELOT and BRCA datasets demonstrate significant performance gains, validating the method’s robustness and its potential for clinical digital pathology.

DualGate-Net: A Prior-Gated Dual-Encoder Approach for Histopathology Cell Detection

Introduction and Motivation

Accurate cell detection in histopathology remains a central challenge in computational pathology, especially in contexts where cellular phenotype depends on surrounding tissue microenvironment. Existing deep learning approaches either emphasize local morphology (CNNs) or global context (transformers), but these are often combined via static fusion schemes that fail to account for the unreliability and spatial heterogeneity of contextual tissue priors. "DualGate-Net: A Prior-Gated Dual-Encoder Framework for Histopathology Cell Detection" (2606.07222) introduces a novel framework that adaptively fuses local and global representations under the influence of dynamically weighted tissue priors, addressing these methodological limitations.

DualGate-Net Architecture

DualGate-Net is characterized by a dual-encoder structure: a ConvNeXtV2-based local encoder for high-fidelity semantic detail, and a SegFormer-based global encoder capable of modeling long-range contextual dependencies. These two streams are merged via a parameterized, spatially adaptive prior-gated fusion mechanism that modulates tissue prior influence at each spatial location and across feature scales.

Figure 1: The DualGate-Net architecture integrates ConvNeXtV2 (local) and SegFormer (global) encoders and adaptively fuses them via multi-scale prior-gated modules, with an auxiliary branch for high-frequency foreground reconstruction.

Prior-Gated Fusion Module

The prior-gated fusion module operates at multiple scales. It generates a spatial reliability gate using local features, global features, and tissue prior maps to measure the utility of priors at each spatial coordinate. These gates are then used to modulate the injection of priors into both the local and global feature spaces. The gated features are subsequently combined using Convolutional Block Attention Modules (CBAM) to realize a robust, context-aware feature representation.

Figure 2: The prior-gated fusion module—left, (B1)—estimates a spatial gate from concatenated feature sets and priors; right, (B2)—projected priors are gated and injected into each encoder pathway, followed by attention-based fusion.

Auxiliary Foreground Reconstruction

To counteract the degradation of cellular structures during deep feature abstraction and improve localization in densely packed or ambiguous regions, DualGate-Net incorporates an auxiliary decoder branch. This branch reconstructs a foreground-enhanced, high-frequency representation (the difference between the input and a Gaussian-smoothed version), regularizing learning and preserving morphological detail. The branch is utilized only during training and incurs no inference-time penalty.

Figure 3: The auxiliary foreground reconstruction branch promotes recovery of fine structural detail from decoder features by reconstructing enhanced foreground representations.

Prior and Cellness Signals

DualGate-Net leverages both tissue-derived priors and a class-agnostic "cellness" prior. Tissue priors are generated from a SegFormer tissue segmentation network and retain multi-channel (cancer/background) spatial probability maps. The cellness prior is produced by a teacher network that detects regions likely to contain cells, furnishing additional localization cues, especially in ambiguous microenvironments. Both signals are incorporated as separate input channels rather than being multiplicatively fused, preserving the integrity of contextual information.

Experimental Results

DualGate-Net is evaluated on the OCELOT benchmark, which includes both cell centroids and fine-grained tissue region annotations. When compared to both CNN-based (e.g., DeepLabV3+, U-Net, ACS-SegNet) and transformer/hybrid methods (CellViT, context-aware detectors), DualGate-Net achieves the strongest reported results, with macro F1-scores of 0.7722 on the validation set and 0.7345 on the test set. Notably, it consistently surpasses prior state of the art on both tumor cell and background cell classes.

Figure 4: Example from the OCELOT dataset showing ground-truth cell and tissue labels (a, b), generated cellness prior (c), predicted tissue priors (d, e), and final prediction overlays (f) with false positives/negatives.

Ablation studies confirm that each individual component—prior-gated fusion, auxiliary foreground reconstruction, and the incorporation of the cellness prior—incrementally contributes to overall detection performance and robustness. The full model, with all these elements, yields the best tradeoff between localization and classification accuracy.

Cross-Dataset Generalization

When evaluated on the BRCA dataset, which lacks explicit tissue contextual supervision, DualGate-Net's adaptive cellness gating retains its efficacy, slightly improving both Macro F1 and Detection F1 scores over leading context-aware baselines. This indicates the generalizability of adaptive, reliability-weighted prior integration mechanisms even outside explicit tissue-aware settings.

Practical and Theoretical Implications

The study evidences that spatially adaptive, learned prior integration yields more robust cell detection, particularly in histology domains characterized by context-dependency, label ambiguity, and annotation noise. The modularity of the prior-gated fusion mechanism suggests extensibility to other domains where contextual priors are noisy or heterogeneous (e.g., other forms of microscopic, satellite, or multimodal medical imaging).

Practically, this framework promises improved cell detection accuracy in clinical digital pathology pipelines, potentially improving diagnostic, prognostic, or therapeutic decision support. Theoretically, it advocates for architectures that treat contextual cues as learnable, spatially weighted signals rather than static conditioning inputs, a paradigm relevant for any context-sensitive computer vision task.

Future Directions

Potential future research avenues include explicit modeling or estimation of prior uncertainty, integration of more diverse prior signals (e.g., omics data, spatial transcriptomics), and evaluation in transfer and domain adaptation contexts across tissue types and staining modalities. Exploring uncertainty-aware gating and the integration of uncertainty quantification within the gating mechanism could further enhance performance and reliability.

Conclusion

DualGate-Net introduces a dual-encoder, prior-gated framework that adaptively fuses contextual priors into multi-scale cell detection, outperforming previous methods in dense and complex histopathological imagery. The approach demonstrates improved generalization and robustness via spatially adaptive integration of noisy priors, auxiliary regularization of fine structural information, and complementary cellness guidance. The findings underscore the broader importance of dynamic, reliability-weighted prior integration for context-dependent vision tasks in biomedical AI.