Learnable Gaussian Band Pass Fusion

• The paper introduces LGBP-Fusion, a module that decomposes CNN and Transformer features into Gaussian band-pass sub-bands to reduce cross-modal interference.
• It employs band-wise cross-attention and a gated fusion mechanism to balance high-frequency details and low-frequency global context effectively.
• Empirical results show improved IoU and mean Dice scores in organoid segmentation, demonstrating its robustness to segmentation challenges like noisy and complex imaging.

Searching arXiv for the cited paper and closely related work to ground the article. Learnable Gaussian Band Pass Fusion is a frequency-aware, learnable fusion mechanism introduced as the core feature fusion block of LGBP-OrgaNet, also referred to as SROrga, for organoid segmentation and tracking. In that formulation, CNN and Transformer feature maps are transformed to the frequency domain, decomposed into multiple Gaussian band-pass sub-bands by a learnable filter bank, fused by cross-attention within each band, recombined, and then balanced against direct spatial fusion through a gate (Zhang et al., 3 Sep 2025). The method is designed to address the representational discrepancy between CNN features, which tend to emphasize high-frequency local details, and Transformer features, which emphasize low-frequency global context and semantics; in organoid imaging, that discrepancy is consequential because precise segmentation depends simultaneously on fine boundaries, internal textures, global shape, and robustness to clutter such as bubbles and shadows (Zhang et al., 3 Sep 2025). In a broader multiscale context, the mechanism also sits near classical Gaussian scale-space reasoning, in which Gaussian filtering parameterizes controllable low-pass structure and band-pass behavior emerges from differences across scales (Mujkanovic et al., 2024).

1. Definition and motivating problem

Learnable Gaussian Band Pass Fusion, abbreviated LGBP-Fusion, is defined as a frequency-aware, learnable fusion block for combining CNN and Transformer features within LGBP-OrgaNet (Zhang et al., 3 Sep 2025). Its defining workflow has four stages: transformation of feature maps to the frequency domain, decomposition into Gaussian band-pass sub-bands using a learnable filter bank, cross-attention performed separately for each band, and recombination of the band-wise fused features together with a gated spatial fusion of the original features (Zhang et al., 3 Sep 2025).

The immediate motivation is the “inherent representational discrepancy” between the two encoder branches. In the formulation used by LGBP-OrgaNet, the ResNet branch emphasizes high-frequency local details such as edges, textures, and fine boundaries, whereas the Swin Transformer branch emphasizes low-frequency global context, long-range dependencies, and semantic structure (Zhang et al., 3 Sep 2025). Simple addition or concatenation treats all frequencies and channels equally, and pure attention-based fusion in the spatial domain still mixes all frequency components together; the stated consequence is cross-modal interference, where global Transformer activations may suppress local CNN details or noisy high-frequency CNN responses may distort semantic Transformer cues (Zhang et al., 3 Sep 2025).

For organoid images, the paper grounds this need in application-specific image characteristics. Organoids exhibit irregular, non-smooth boundaries, complex internal textures, and significant background noise, including bubbles and shadows. The paper states that correct segmentation often depends on high-frequency information for precise boundaries and fine interior or edge texture, together with low-frequency information for global organoid shape and separation from large-scale background fluctuations (Zhang et al., 3 Sep 2025). Frequency-agnostic fusion can therefore either overfit to noise in high frequencies or oversmooth fine structures if low-frequency semantics dominate (Zhang et al., 3 Sep 2025).

The authors summarize the intended function of the module as follows: “LGBP-Fusion decomposes features into adaptive frequency bands and performs band-wise self-attention, integrating CNN textures with Transformer semantics while reducing cross-modal interference” (Zhang et al., 3 Sep 2025). This description positions the module not merely as a feature mixer but as a structured band-selective interface between two complementary representation families.

2. Mathematical formulation

The module operates on a CNN feature map $F_c$ and a Transformer feature map $F_t$, conceptually with $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$ (Zhang et al., 3 Sep 2025). Both are first mapped into the frequency domain through a 2D Fourier transform: $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$ This establishes the domain in which the Gaussian band-pass filter bank is applied (Zhang et al., 3 Sep 2025).

The learnable filter bank consists of $K$ Gaussian band-pass filters indexed by frequency radius $r$: $$B_k(r) = \exp\left(-\frac{(r - \mu_k)^2}{2\sigma_k^2}\right)^{\max(0, \alpha_k)}, \quad k=1,\dots,K.$$ Here, $\mu_k$ is the learnable center frequency, $\sigma_k$ is the learnable bandwidth, and $\alpha_k$ is the learnable amplitude factor, passed through $F_t$0 to ensure non-negative power for stability (Zhang et al., 3 Sep 2025). The filters are applied by element-wise multiplication in the frequency domain: $F_t$1 Each filtered feature is then returned to the spatial domain: $F_t$2 These steps produce paired CNN and Transformer features that are restricted to the same frequency band (Zhang et al., 3 Sep 2025).

For each band $F_t$3, the method performs cross-attention using Transformer-band features as queries and CNN-band features as keys and values. After learned projections

$F_t$4

the paper uses scaled dot-product attention: $F_t$5 Spatial dimensions are flattened into sequence length, and the operation is conceptually standard Transformer attention applied separately within each frequency band (Zhang et al., 3 Sep 2025). This band-wise factorization is the central mechanism by which the fusion becomes frequency-selective rather than full-spectrum.

The outputs of all band-wise attention blocks are then transformed back into the frequency domain, summed, and returned to the spatial domain: $F_t$6 The resulting $F_t$7 is the frequency-aware fused feature (Zhang et al., 3 Sep 2025).

To prevent the model from relying exclusively on the frequency-domain branch, the module introduces a gate: $F_t$8 followed by the final output fusion

$F_t$9

The gate is obtained through a convolutional projection and a Sigmoid, so that $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$0 (Zhang et al., 3 Sep 2025). The stated interpretation is adaptive control: where the frequency-aware fusion is informative, the model can emphasize it; elsewhere it can fall back to direct spatial concatenation.

3. Architectural role in LGBP-OrgaNet

LGBP-Fusion is embedded in a dual-branch encoder composed of a ResNet50 branch for local spatial features and a Swin Transformer branch for global context (Zhang et al., 3 Sep 2025). The feature pyramid spans three scales in each branch: the ResNet pathway uses an initialization block plus three levels, and the Swin Transformer pathway uses three modules with 2, 2, and 6 layers, with patch merging between modules (Zhang et al., 3 Sep 2025).

At each matching encoder scale, the outputs from the ResNet and Swin Transformer branches are fused. The paper states: “The outputs of each ResNet module and Swin Transformer module are fused using the CorrelationBlock and then transmitted to the next SwinTransformer module and decoder. The LGBP-Fusion module effectively combines features from different domains, outputting the fused features to the subsequent Swin Transformer module and decoder module” (Zhang et al., 3 Sep 2025). The nomenclature is explicitly noted as slightly inconsistent in the source text, but the described function corresponds to the Learnable Gaussian Band Pass Fusion block (Zhang et al., 3 Sep 2025).

This placement is structurally significant. The fused representation is not used only as a skip connection; it is also passed forward into the subsequent Swin Transformer module (Zhang et al., 3 Sep 2025). This means the Transformer branch itself is updated on fused representations rather than on an isolated global-context stream. A plausible implication is that the architecture treats LGBP-Fusion as an inter-branch alignment mechanism throughout the encoder, rather than as a terminal late-fusion operation.

The decoder uses a distinct mechanism, the Bidirectional Cross Fusion Block, or BCF, which fuses multi-scale features across resolution levels rather than across modalities at the same scale (Zhang et al., 3 Sep 2025). In the decoder, $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$1, $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$2, and $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$3 are encoder outputs already fused by LGBP, and BCF fuses $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$4 with $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$5 and $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$6 with $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$7 to produce refined features before progressive concatenation and upsampling (Zhang et al., 3 Sep 2025). The division of labor is therefore explicit: LGBP-Fusion performs within-scale CNN–Transformer fusion, whereas BCF performs across-scale decoder fusion.

4. Functional properties and signal-processing interpretation

The paper attributes five principal properties to LGBP-Fusion: frequency-aware fusion, bridging of CNN–Transformer discrepancy, reduced cross-modal interference, adaptivity and interpretability, and robustness to noise and complex backgrounds (Zhang et al., 3 Sep 2025). These are all framed as consequences of decomposing features into adaptive frequency bands and fusing the modalities separately within each band.

The frequency-aware aspect is the most direct. Low frequencies are associated with global semantic structure, high frequencies with local textures and edges, and mid frequencies with structural details (Zhang et al., 3 Sep 2025). Because each band is processed separately, the model can preserve Transformer-dominant global information in low-frequency regions while retaining CNN-dominant boundary and texture information in high-frequency regions (Zhang et al., 3 Sep 2025). This is more structured than full-band attention or channel-wise fusion.

The paper explicitly frames this as a way of bridging the discrepancy between CNN and Transformer representations. In high-frequency bands, cross-attention allows Transformer context to modulate CNN details; in low-frequency bands, CNN coarse structure and Transformer global context interact (Zhang et al., 3 Sep 2025). This suggests a modality-specific division of spectral labor, though the mechanism remains learnable because the band centers $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$8, bandwidths $$F_c, F_t \in \mathbb{R}^{B \times C \times H \times W}$$9, and amplitude factors $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$0 are all trainable (Zhang et al., 3 Sep 2025).

Interpretability is grounded in signal-processing terminology. The learned parameters correspond to center frequency, bandwidth, and gain or sharpness in a Gaussian-like filter family (Zhang et al., 3 Sep 2025). The source does not provide explicit learned frequency-response plots, but the parameterization itself is designed to remain legible in classical terms. This distinguishes the module from generic attention-based fusion, where spectral behavior is usually implicit.

In relation to broader Gaussian scale-space reasoning, the mechanism is narrower than a full continuous scale-space field but conceptually aligned with the use of Gaussian filtering to organize multiscale information. Neural Gaussian Scale-Space Fields represent a continuous anisotropic Gaussian low-pass family $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$1, from which Gaussian band-pass filters can be formed by differences across scales, such as

$$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$2

(Mujkanovic et al., 2024). LGBP-Fusion does not implement that formulation; instead, it learns explicit band-pass filters directly in the frequency domain of encoder features (Zhang et al., 3 Sep 2025). A plausible implication is that LGBP-Fusion occupies a middle ground between classical multiband signal decomposition and end-to-end attention-based feature fusion.

5. Empirical effects in organoid segmentation

The paper reports an ablation that isolates the contribution of LGBP-Fusion, referred to in the ablation text as CorrelationBlock/LGBP-Fusion (Zhang et al., 3 Sep 2025). The relevant variants are summarized below.

Variant	IOU (%)	Mean Dice (%)
Ours (full)	80.91	93.37
Ours_LGBP-Fusion	75.78	91.01

The source states that “Ours_LGBP-Fusion” means the model without LGBP-Fusion, using simple addition of features instead of the fusion block (Zhang et al., 3 Sep 2025). Removing the module causes an IoU drop of $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$3 and a Mean Dice drop of about $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$4 on the Mice Bladder dataset with complex backgrounds (Zhang et al., 3 Sep 2025). The paper comments: “After removing the Correlation Block, the model's segmentation accuracy also decreased, highlighting the importance of the Correlation module in feature fusion” (Zhang et al., 3 Sep 2025).

In broader model comparison on Mice Bladder Organoids, LGBP-OrgaNet achieves IoU 80.91% versus 68.31% for the second-best OrganoID, and Mean Dice 93.37% versus 87.93% (Zhang et al., 3 Sep 2025). The source associates this improvement with the new fusion design together with BCF, rather than with a single isolated component (Zhang et al., 3 Sep 2025). Accordingly, any attribution of the full performance gap specifically to LGBP-Fusion should remain qualified.

The qualitative results reported in the source connect the module to robustness across several failure modes. LGBP-OrgaNet is described as handling overlapping organoids in the Mice Bladder dataset, background impurities in the Mammary Epithelial dataset, and blurred contours with low contrast in the Brain Organoid dataset better than competing models (Zhang et al., 3 Sep 2025). The qualitative descriptions state that where other models merge overlapping organoids, confuse bubbles or debris with organoids, or lose organoid boundaries in low-contrast images, LGBP-OrgaNet keeps precise boundaries, correctly distinguishes organoid from background, and is less affected by illumination variations and imaging artifacts (Zhang et al., 3 Sep 2025). These behaviors are explicitly attributed to the frequency-aware CNN–Transformer fusion of LGBP (Zhang et al., 3 Sep 2025).

The source also states that the method delivers “stable performance across size scales: average IoU above 80% across organoid area ranges in the most challenging dataset” and robust segmentation in co-culture systems with immune cells, where bright-field organoid images are noisy and cluttered (Zhang et al., 3 Sep 2025). Within the argument of the paper, this supports the claim that frequency-selective fusion is useful precisely because organoid morphology is distributed across multiple spatial frequency regimes.

6. Parameterization, optimization, and implementation constraints

LGBP-Fusion introduces three classes of learnable parameters: the Gaussian filter parameters $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$5, $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$6, and $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$7 for each band; the attention projections $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$8, $$\hat{F}_c = \mathcal{F}(F_c), \quad \hat{F}_t = \mathcal{F}(F_t).$$9, and $K$0; and the gate projection $K$1 (Zhang et al., 3 Sep 2025). The paper describes these additions as relatively lightweight compared to the full ResNet50 and Swin Transformer backbones (Zhang et al., 3 Sep 2025).

The source identifies several stability-related constraints. The exponent uses $K$2, which ensures non-negative power and is stated to prevent numerical issues from negative exponents on the Gaussian (Zhang et al., 3 Sep 2025). The gate uses a Sigmoid so that $K$3, stabilizing the balance between the frequency-aware fusion $K$4 and the direct spatial concatenation $K$5 (Zhang et al., 3 Sep 2025). The positivity of $K$6 is not explicitly specified in the paper; the provided details note that this is typically implemented by parameterizing a raw variable and applying a softplus or absolute value (Zhang et al., 3 Sep 2025). Because that wording is itself interpretive in the source detail, it should be read as implementation expectation rather than as a stated paper-level algorithmic commitment.

The training configuration reported for the overall model is standard and end-to-end. The optimizer is SGD; the initial learning rate is 0.01, decayed by 0.1 every 10 epochs; training runs for 300 epochs with batch size 4 and input size $K$7; and the loss is a combination of Isoperimetric Quotient Loss, Dice Loss, and Focal Loss (Zhang et al., 3 Sep 2025). LGBP-Fusion is fully differentiable and trained jointly with the rest of the network (Zhang et al., 3 Sep 2025).

The paper does not report special learning-rate tricks or dedicated regularization for the fusion parameters (Zhang et al., 3 Sep 2025). A plausible implication is that the module is intended to integrate cleanly into standard end-to-end optimization without an auxiliary training schedule, which is relevant for reproducing it as part of larger hybrid CNN–Transformer segmentation systems.

7. Relation to standard fusion mechanisms, adjacent frequency-domain work, and limitations

The paper contrasts LGBP-Fusion with common CNN–Transformer fusion strategies: concatenation followed by $K$8 convolution, element-wise addition, and attention-based fusion in the spatial domain (Zhang et al., 3 Sep 2025). The stated limitation of these methods is that they ignore the frequency structure of features, potentially causing loss of fine local details, distortion of global semantic cues, and reduced interpretability with respect to which spatial frequencies are amplified or suppressed (Zhang et al., 3 Sep 2025).

LGBP-Fusion differs in four specific respects: explicit frequency-band decomposition via Gaussian band-pass filters, band-wise cross-attention rather than single full-spectrum attention, learnable filter centers and bandwidths, and a gated combination with spatial features to avoid over-commitment to the frequency domain (Zhang et al., 3 Sep 2025). The paper therefore positions it closer to classical multi-band fusion or multi-scale wavelet decomposition, but implemented with differentiable Fourier filtering, learnable parameters, and integration with modern attention mechanisms (Zhang et al., 3 Sep 2025).

The source also discusses TransOrga as an earlier organoid segmentation method that uses Fourier filtering and Vision Transformer to combine spatial and frequency-domain features (Zhang et al., 3 Sep 2025). The distinction drawn is that TransOrga uses frequency filtering mainly as a preprocessing or enhancement step, whereas LGBP-Fusion integrates Fourier filtering directly inside the fusion module, decomposes both CNN and Transformer features into multiple learned bands, and applies cross-attention per band with recombination (Zhang et al., 3 Sep 2025). This comparative framing is important because it identifies LGBP-Fusion not simply as “using Fourier features,” but as structuring multimodal interaction explicitly in the spectral domain.

The paper’s explicit limitation discussion focuses more on tracking than on LGBP-Fusion itself, but several limitations are identified or inferred in relation to the fusion block (Zhang et al., 3 Sep 2025). The first is computational overhead: the module requires 2D Fourier and inverse Fourier transforms together with multiple band-wise attention operations, making it more expensive than simple addition or concatenation, and the paper does not quantify runtime or memory overhead (Zhang et al., 3 Sep 2025). The second concerns the number of bands $K$9, which is not empirically analyzed; the source notes that more bands give more flexibility but also more computation (Zhang et al., 3 Sep 2025). The third is the absence of explicit frequency visualization: although the design is interpretable in principle, the paper does not show learned $r$0, $r$1, band energy distributions, or frequency response curves (Zhang et al., 3 Sep 2025). The fourth concerns external generality, since validation is reported on bright-field organoid images from several sources, while extension to fluorescence, phase contrast, or other medical image segmentation settings is posed as future work (Zhang et al., 3 Sep 2025).

Taken together, these points define Learnable Gaussian Band Pass Fusion as a specific answer to a recurrent problem in hybrid CNN–Transformer vision architectures: how to combine local detail and global context without collapsing their spectral distinctions. In LGBP-OrgaNet, the answer is to make the fusion itself frequency-selective, trainable, and interpretable in Gaussian band-pass terms, with measurable improvements on challenging organoid segmentation benchmarks (Zhang et al., 3 Sep 2025).