LGBP-OrgaNet: Dual-branch Organoid Analysis

• The paper introduces LGBP-OrgaNet, a dual-branch framework combining ResNet50 and Swin Transformer for robust organoid segmentation and tracking.
• It employs Learnable Gaussian Band Pass Fusion to merge spectral features, enhancing local boundary details and global semantic context.
• The approach integrates multi-scale bidirectional fusion with tracking via connected components, the Hungarian algorithm, and Kalman filtering for improved performance.

Searching arXiv for the primary paper and a few related baselines for citation metadata. LGBP-OrgaNet is a dual-branch deep learning framework for non-destructive organoid segmentation and tracking from bright-field microscopy. It is designed for automated, label-free analysis in settings where manual analysis is difficult because organoid shapes are irregular, sizes vary widely, and images contain bubbles, shadows, and blurred boundaries. The method combines a ResNet50 branch for local texture and edge details with a Swin Transformer branch for global context and long-range dependencies, and introduces Learnable Gaussian Band Pass Fusion to merge the two representations in a frequency-aware manner. In the decoder, it uses a Bidirectional Cross Fusion Block and a progressive concatenation and upsampling strategy to integrate multi-scale features for segmentation, followed by a tracking pipeline based on connected components, the Hungarian algorithm, and Kalman filtering (Zhang et al., 3 Sep 2025).

1. Problem formulation and biological motivation

LGBP-OrgaNet addresses two related tasks: segmentation of organoids in bright-field images and tracking organoids across consecutive frames by matching segmented regions over time (Zhang et al., 3 Sep 2025). The method is explicitly motivated by the need for non-destructive, automated, and robust organoid analysis.

The biological context is central to the formulation. Organoids replicate organ structure and function and are used in tumor treatment, drug screening, growth monitoring, and tumor–immune co-culture studies. In this setting, fluorescence labeling can improve visibility, but it may also compromise organoid integrity, which is undesirable for downstream biological experiments. The framework therefore targets automated, label-free analysis that can segment organoids accurately and then track them over time without disturbing the sample (Zhang et al., 3 Sep 2025).

The core imaging difficulty is not merely low contrast. The paper emphasizes irregular morphologies, large inter-object scale variation, and structured nuisances such as bubbles, shadows, reflections, and blurred boundaries. This suggests that conventional single-backbone architectures may be insufficient when both local boundary fidelity and global semantic consistency must be preserved simultaneously.

2. Data regime and evaluation setting

To validate robustness, the study uses three datasets: a self-constructed mice bladder organoid dataset with complex backgrounds, a public mammary epithelial organoid dataset, and a public brain organoid dataset with imaging distractions such as shadows and reflections (Zhang et al., 3 Sep 2025). The bladder dataset is described as the most challenging and as the main contribution of the study.

The self-constructed bladder organoid dataset contains 772 RGB images of size $648 \times 486$ with 2407 organoid regions. The tracking set includes 3 time-lapse sequences (Zhang et al., 3 Sep 2025). These data are used to test whether the model can segment and track organoids under realistic experimental variability.

The evaluation protocol reports Accuracy, Precision, Recall, IoU, F1-score, and Mean Dice (Zhang et al., 3 Sep 2025). The comparative baselines are OrganoID, TransOrga, U-Net, U-Net++, and TransUNet. A plausible implication is that the authors position LGBP-OrgaNet against both classical encoder–decoder CNNs and more recent hybrid or Transformer-based segmentation systems, but the article’s quantitative discussion remains confined to the metrics explicitly reported in the source.

3. Architecture: dual encoding and multi-stage decoding

The architecture consists of a dual-encoding pathway plus a multi-stage decoder (Zhang et al., 3 Sep 2025). The encoder uses a ResNet50 branch and a Swin Transformer branch. The architectural motivation is that the two backbones capture complementary information: CNN features are strong at local spatial structure, edges, textures, and fine details, whereas Transformer features are better at global semantic consistency and broader context.

The encoder is hierarchical. ResNet has an initial block and three convolutional stages. The Swin Transformer has three stages with 2, 2, and 6 encoding layers, using 3, 6, and 12 attention heads and a window size of 7. Between Swin stages, Patch Merging downsamples features. After each ResNet stage, a $1 \times 1$ convolution aligns channel dimensions with the Transformer branch so that cross-branch fusion is possible (Zhang et al., 3 Sep 2025).

The decoder refines multi-scale encoder features using a Bidirectional Cross Fusion Block, then combines them through progressive concatenation and upsampling. Adjacent fused features are upsampled, concatenated layer by layer, processed by convolutional modules, and finally sent to the segmentation head. This design retains and aggregates shallow features, intermediate features, and deep semantic features (Zhang et al., 3 Sep 2025).

The resulting architecture can be understood as a structured attempt to preserve three properties simultaneously: local boundary precision, scale robustness, and semantic coherence. This interpretation follows directly from the stated role of the CNN branch, Transformer branch, BCF module, and progressive decoder.

4. Learnable Gaussian Band Pass Fusion

The principal technical novelty is Learnable Gaussian Band Pass Fusion, a module for merging CNN and Transformer representations in a frequency-aware way (Zhang et al., 3 Sep 2025). The paper frames the two encoders as having different spectral tendencies: CNNs tend to emphasize high-frequency components such as edges and texture, while Transformers tend to emphasize low-frequency components such as global semantics and structural layout.

Given CNN features $F_c$ and Transformer features $F_t$, both are transformed into the frequency domain via the 2D Fourier transform: $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$

The model then applies a bank of $K$ learnable Gaussian band-pass filters: $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$ where $r$ is the frequency radius, $\mu_k$ is the learnable center frequency, $\sigma_k$ is the learnable bandwidth, and $1 \times 1$0 is a learnable amplitude/exponent factor that modulates the filter shape (Zhang et al., 3 Sep 2025).

The filtered sub-band features are

$1 \times 1$1

and are then transformed back to the spatial domain: $1 \times 1$2

Within each frequency band, the model performs cross attention using Transformer features as queries and CNN features as keys and values: $1 \times 1$3

$1 \times 1$4

The band-wise outputs are recombined as

$1 \times 1$5

The final stage balances the frequency-aware fused representation with direct concatenation of the original inputs: $1 \times 1$6

$1 \times 1$7

According to the paper, this makes fusion frequency-aware, adaptive, interpretable, and less prone to destructive mixing of incompatible features (Zhang et al., 3 Sep 2025). A plausible implication is that the module serves not only as a fusion operator but also as an inductive bias about how local-detail and global-semantic representations should interact.

5. Multi-scale decoding: Bidirectional Cross Fusion Block and progressive upsampling

The second key fusion module is the Bidirectional Cross Fusion Block, used to combine multi-scale encoder features in the decoder (Zhang et al., 3 Sep 2025). Its motivation is the large variation in organoid scale: fine boundaries and small protrusions require high-resolution details, whereas shape completion and contextual coherence require lower-resolution semantic information.

BCF fuses information in both directions: high-to-low injects detailed information into coarse features, and low-to-high injects context into detailed features. The block takes adjacent-scale features, such as $1 \times 1$8 and an upsampled $1 \times 1$9, and processes them with LayerNorm, patch expansion or interpolation for low-resolution features, a pre-fusion MLP that concatenates features along channels, bidirectional multi-head attention, learnable relative positional bias, adaptive gating, residual connections, and a final linear projection (Zhang et al., 3 Sep 2025).

For the high-to-low direction, the attention is given as

$F_c$0

where $F_c$1 comes from high-resolution features $F_c$2, $F_c$3 and $F_c$4 come from upsampled low-resolution features, and $F_c$5 is a learnable relative position bias.

For the low-to-high direction,

$F_c$6

where the query comes from the low-resolution side and the key/value are built from the high-resolution plus pre-fused representation. The outputs are projected and scaled by learnable parameters $F_c$7 and $F_c$8, then fused through an MLP-based gate: $F_c$9

After BCF, the decoder uses a layer-by-layer progressive concatenation strategy. The paper describes $F_t$0, $F_t$1, and $F_t$2 as fused encoder features at different scales; $F_t$3 is upsampled and concatenated with $F_t$4 to form $F_t$5, $F_t$6 is upsampled and concatenated with $F_t$7 to form $F_t$8, and $F_t$9 is further upsampled and concatenated with $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$0 and $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$1, followed by a final convolution and segmentation head (Zhang et al., 3 Sep 2025).

The paper refers to this as fully connected progressive upsampling. Its stated intention is to preserve edge detail, shape continuity, and large-scale organoid structure. This suggests that the decoder is designed to avoid the limitations of a single skip connection by repeatedly reintroducing complementary information across scales.

6. Tracking, optimization, and empirical results

After segmentation, tracking is performed by extracting connected components, matching regions between adjacent frames using the Hungarian algorithm, and using Kalman filtering for unmatched targets (Zhang et al., 3 Sep 2025). The matching cost combines Euclidean distance and region similarity: $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$2 with

$$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$3

and

$$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$4

The maximum allowable matching cost is set to 20, and unmatched targets are propagated using Kalman prediction (Zhang et al., 3 Sep 2025). The authors note a limitation: tracking only compares adjacent frames, so robustness could be improved by multi-frame association.

Training uses PyTorch with input size $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$5, batch size 4, SGD, initial learning rate 0.01, learning rate decay by multiplication with 0.1 every 10 epochs, and 300 epochs on an NVIDIA RTX 3070 with 12 GB VRAM. The total loss is

$$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$6

with $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$7, $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$8, and $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t)$$9 (Zhang et al., 3 Sep 2025).

The component losses are defined as follows. The isoperimetric quotient loss is

$K$0

where $K$1 is perimeter, $K$2 is area, $K$3 are gradients, $K$4 is the segmented region, and $K$5 denotes pixel values in the organoid region. Dice loss is

$K$6

and focal loss is

$K$7

This combination is intended to address shape compactness, class imbalance, and hard-example emphasis (Zhang et al., 3 Sep 2025).

The reported segmentation results are summarized below.

Dataset	IoU	Precision	Recall
Mice bladder organoids	80.91%	90.75%	88.56%
Mammary epithelial organoids	85.89%	91.24%	93.15%
Brain organoids	94.49%	97.38%	96.99%

On mice bladder organoids, the method reports F1 $K$8, Accuracy $K$9, and Mean Dice $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$0. The paper states that the gain over OrganoID is $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$1 IoU and $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$2 Dice. On mammary epithelial organoids, it reports F1 $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$3, Accuracy $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$4, and Mean Dice $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$5, and is described as slightly better than TransUNet. On brain organoids, it reports F1 $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$6, Accuracy $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$7, and Mean Dice $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K$$8, with strong and stable performance under lighting variation and blur (Zhang et al., 3 Sep 2025).

The paper also evaluates IoU over different organoid area ranges. The model remains above 80% IoU on the challenging bladder dataset across scales and above 90% IoU on brain organoids, suggesting good scale robustness. Qualitatively, it is reported to better separate overlapping organoids, background bubbles, impurities, blurred boundaries, and lighting artifacts, particularly on the bladder dataset where competing methods often over-segment or fail to distinguish organoid boundaries from clutter (Zhang et al., 3 Sep 2025).

7. Ablation, interpretation, and practical significance

The ablation study removes four components: the ResNet branch, LGBP-Fusion, BCF, and the decoder design (Zhang et al., 3 Sep 2025). On the bladder organoid dataset, the full model achieves IoU 80.91, Precision 90.75, Recall 88.56, F1 88.53, Acc 97.17, and Mean Dice 93.37. Removing the ResNet branch yields IoU 72.51, removing LGBP-Fusion yields 75.78, removing BCF yields 65.82, and replacing the decoder design yields 76.18. The most severe drop comes from removing BCF, and removing LGBP-Fusion also reduces performance substantially (Zhang et al., 3 Sep 2025).

These ablations clarify the division of labor among modules. BCF is presented as especially critical for multi-scale bidirectional fusion; LGBP-Fusion is important for frequency-aware CNN–Transformer fusion; the ResNet branch contributes local detail; and the progressive decoder contributes to feature aggregation. A plausible implication is that the model’s performance depends less on any single backbone than on the coordinated interaction between spectral fusion and multi-scale decoding.

In practical terms, the paper’s stated message is that LGBP-OrgaNet provides a reliable way to segment organoids without fluorescent labels, track them over time, measure growth dynamically, and analyze morphology in difficult imaging conditions (Zhang et al., 3 Sep 2025). The method is presented as relevant to drug screening, cancer modeling, organoid growth analysis, and organoid–immune cell co-culture studies. The authors also show application in a co-culture setting where bright-field segmentation avoids the need for fluorophores that could alter biological behavior. This suggests a role for the framework in non-invasive, high-throughput organoid phenotyping while preserving sample integrity.

A common misconception in this domain is that label-free bright-field analysis is principally a visualization problem. The paper instead frames it as a coupled representation-learning and fusion problem involving heterogeneous spatial frequencies, multi-scale morphology, and temporal association under realistic experimental variability. Within that framing, LGBP-OrgaNet is distinguished by the combination of frequency-domain CNN–Transformer fusion, bidirectional multi-scale decoding, and a simple practical tracking pipeline (Zhang et al., 3 Sep 2025).