Complex Convolutional Block Attention Module

• CCBAM is a family of attention mechanisms that enhances CNNs by integrating joint channel and spatial weighting for both complex-valued and real-valued tasks.
• It employs sequential complex channel and spatial modules to reweight feature maps, improving performance in applications like speech enhancement with notable PESQ and SI-SNR gains.
• The Cross-CBAM variant fuses high- and low-level features in semantic segmentation, boosting mIoU while maintaining real-time efficiency.

The Complex Convolutional Block Attention Module (CCBAM) refers to a family of architectural attention mechanisms designed to augment the representational capacity of convolutional neural networks (CNNs) by integrating channel- and spatial-wise weighting strategies. There are two primary CCBAM lineages documented in the literature: the “Complex Convolutional Block Attention Module” for complex-valued speech enhancement systems (Zhao et al., 2021) and the “Cross Convolutional Block Attention Module” for lightweight scene segmentation (Zhang et al., 2023). Both are structurally related to classical CBAM, but each introduces tailored attention operations—either for complex-valued signals or for cross-stream feature fusion in multi-scale real-valued tasks.

1. Architectural Overview and Motivations

CCBAM, in its original complex-valued form, is a plug-and-play attention block that can be interposed after any complex convolutional (or deconvolutional) layer in complex-valued CNNs. Its principal function is to enable fine-grained, joint channel–spatial attention in Short-Time Fourier Transform (STFT) or similar domains by constructing real-valued gates that selectively rescale the real and imaginary branches of feature maps. The module is fully differentiable, lightweight in parameter cost, and has negligible impact on the computational complexity of canonical encoder–decoder architectures such as DCUnet and DCCRN (Zhao et al., 2021).

The Cross-CBAM variant for semantic segmentation applies a two-stream, cross-attention mechanism at decoder fusion points within an FPN-style architecture, using channel and spatial attention gates to modulate low- and high-level feature streams. This structure aims to enhance semantic consistency and spatial detail retention during feature pyramid fusion, with minimal overhead (Zhang et al., 2023).

2. Complex-Valued CCBAM Structure and Mathematical Formulation

The complex-valued CCBAM consists of two sequential sub-modules:

1. Complex Channel-Attention Module
2. Complex Spatial-Attention Module

Let $$U = U_r + j U_i \in \mathbb{C}^{H \times W \times C}$$ denote an input feature map. The module proceeds as follows:

2.1 Complex Channel-Attention

• Squeeze:
  • Apply global average- and max-pooling independently to real and imaginary channels, forming complex pooled vectors:
  • $$U^{\mathrm{avg}} = U_r^{\mathrm{avg}} + j U_i^{\mathrm{avg}}$$, $U^{\max} = U_r^{\max} + j U_i^{\max}$, each $1 \times 1 \times C$
• Excitation:
  • Pass through two shared complex-valued fully-connected layers with reduction ratio $r$:
  • $$Z^{\mathrm{avg}} = W_2^{\mathrm{FC}} \delta( W_1^{\mathrm{FC}} U^{\mathrm{avg}} )$$
  • $$Z^{\max} = W_2^{\mathrm{FC}} \delta( W_1^{\mathrm{FC}} U^{\max} )$$
  • $W_1^{\mathrm{FC}} \in \mathbb{C}^{C/r \times C}$, $W_2^{\mathrm{FC}} \in \mathbb{C}^{C \times C/r}$, $\delta(\cdot)$ denotes elementwise complex ReLU.
  • Output combines both branches with a complex sigmoid, yielding a real-valued channel attention map:
  • $G_c = \sigma(Z^{\mathrm{avg}}) + \sigma(Z^{\max})$, $G_c \in \mathbb{R}^{1 \times 1 \times C}$
• Re-scaling:
  • The original feature map is reweighted channel-wise: $U' = G_c \odot U$, broadcasted over $H \times W$.

2.2 Complex Spatial-Attention

• Squeeze:
  • Pool across channels at each spatial location (separately for real/imag):
  • $$S_r^{\mathrm{avg}}(h,w) = \tfrac{1}{C}\sum_{c}U_r(h,w,c)$$, $S_r^{\max}(h,w) = \max_{c}U_r(h,w,c)$
  • Analogous expressions for $S_i$.
  • Form complex average and max maps, concatenate:

  $$S = [ S^{\mathrm{avg}}; S^{\max} ] \in \mathbb{C}^{H \times W \times 2}$$

• Excitation:

  • Apply a single complex 2D convolution $F = F_r + j F_i$ (kernel size 7x7), followed by a (complex) sigmoid:
  • $G_s = \sigma(F * S)$, $G_s \in \mathbb{R}^{H \times W \times 1}$
• Re-scaling:
  • Results are spatially reweighted: $U'' = G_s \odot U'$, broadcast along $C$.

3. Cross-CBAM in Real-Valued Multi-Stream Decoders

Cross-CBAM is deployed at FPN-style feature fusions in semantic segmentation networks, facilitating two-way attention between encoder-derived (low-level, detailed) and decoder/ASPP-derived (high-level, semantic) features (Zhang et al., 2023):

• Inputs:
  • $$\mathrm{Input}_{\mathrm{high}}, \mathrm{Input}_{\mathrm{low}} \in \mathbb{R}^{B \times C' \times H \times W}$$
• Channel Attention:
  • Compute channel gates $C_{\mathrm{high}}, C_{\mathrm{low}}$ as in standard CBAM.
• Cross Multiply (Step 1):
  • $$F_{\mathrm{high}} = \mathrm{Input}_{\mathrm{low}} \odot C_{\mathrm{high}}$$
  • $$F_{\mathrm{low}} = \mathrm{Input}_{\mathrm{high}} \odot C_{\mathrm{low}}$$
• Spatial Attention:
  • For each $F$, pool across channels, concatenate avg/max, apply a (usually 1×1) convolution, and sigmoid to obtain $S_{\mathrm{high}}, S_{\mathrm{low}}$.
• Cross Multiply (Step 2) and Final Sum:
  • $$O_{\mathrm{high}} = F_{\mathrm{low}} \odot S_{\mathrm{high}}$$
  • $$O_{\mathrm{low}} = F_{\mathrm{high}} \odot S_{\mathrm{low}}$$
  • Final output: $$\mathrm{Output} = O_{\mathrm{high}} + O_{\mathrm{low}}$$

This design guides feature fusion with both “what” (channel) and “where” (spatial) cues, using semantic information from high-level features to weight low-level details and vice versa.

4. Integration into Target Architectures

4.1 Speech Enhancement Networks

• In both deep complex U-Net (DCUnet) and convolutional recurrent network (DCCRN), CCBAM modules are injected:
  • On skip-connection feature maps after encoder–decoder concatenation.
  • On decoder outputs post up-convolution, batch normalization, and activation.
• The injection order is always channel-attention then spatial-attention blocks.
• No modifications to batch normalization or activation layers; CCBAM is fully modular (Zhao et al., 2021).

4.2 Scene Segmentation Networks

• Each decoder fusion stage fuses an upsampled semantic feature with an encoder detail feature by matching channel counts and passing both streams through a CCBAM block.
• Operations are organized for computational efficiency: channel reduction via 1×1 convolutions, minimal convolutional kernels for attention maps, and bilinear upsampling preceding segmentation head output (Zhang et al., 2023).

5. Loss Functions and Training Protocols

5.1 Joint Time–Frequency Loss (Speech Enhancement)

The CCBAM-enhanced speech enhancement framework employs a total loss as a weighted sum of time-domain scale-invariant signal-to-noise ratio (SI-SNR) and time–frequency-domain mask mean squared error (MSE):

$$\mathcal{L}(y, \hat{y}) = \lambda_{\mathrm{SI\text{-}SNR}} \, \mathcal{L}_{\mathrm{SI\text{-}SNR}}(y, \hat{y}) + \lambda_{\mathrm{Mask}} \, \mathcal{L}_{\mathrm{Mask}}(M, \hat{M})$$

where both $\lambda$ are set to 0.5.

• SI-SNR Definition:

$$\mathcal{L}_{\mathrm{SI\text{-}SNR}}(y, \hat{y}) = 10\log_{10} \frac{\|y_{\mathrm{target}}\|_2^2}{\|e_{\mathrm{noise}}\|_2^2}$$

• Mask MSE (Complex Ratio Mask):

$$\mathcal{L}_{\mathrm{Mask}}(M, \hat{M}) = \sum_{t,f} \left[ (\hat{M}_r(t,f) - M_r(t,f))^2 + (\hat{M}_i(t,f) - M_i(t,f))^2 \right]$$

5.2 Segmentation Optimization

Segmentation networks utilize a combination of cross-entropy and focal loss, with an auxiliary supervision head weighted for improved convergence. Training follows standard SGD with “poly” learning rate scheduling, large minibatch sizes, and extensive random augmentation of input crops and resolutions (Zhang et al., 2023).

6. Empirical Performance and Complexity Analysis

CCBAM Variant	Application	Topline Metric Gain	Params/Block	Computation
Complex-valued	Speech enhancement	+0.05–0.15 PESQ, +0.4–0.8 dB SI-SNR	— (lightweight)	Negligible
Cross-CBAM	Real-time segmentation	+3.9 % mIoU, ~20 FPS drop	~16.5K	<0.001 GFLOPs/block

• In speech enhancement (WSJ0+DEMAND, DNS-challenge), replacing SI-SNR loss with mixed loss yields +0.1–0.2 PESQ and +0.2–0.3 dB SI-SNR; adding CCBAM provides a further +0.05–0.15 PESQ and +0.4–0.8 dB SI-SNR improvement (Zhao et al., 2021).
• In Cross-CBAM, the standalone addition of CCBAM to a baseline STDC1 segmentation model boosts Cityscapes val mIoU from 52.26% to 72.63%, with total inference speed of 245 FPS on 1080Ti. Full Cross-CBAM-M1 (SE-ASPP + CCBAM + Aux Loss) reaches 74.19% mIoU at 240.9 FPS (Zhang et al., 2023).

7. Comparative Perspectives and Design Implications

CCBAM extends classical attention in CNNs by addressing deficiencies in detail–semantics fusion and complex-valued feature manipulation. In the complex-valued variant, CCBAM achieves shared gating over both real and imaginary spectral domains, yielding more expressive feature selection in time-frequency processing. In the cross-attention variant, bidirectional gating enforces semantic consistency and enhances boundary delineation without undermining real-time performance requirements.

A plausible implication is that the general CCBAM design—sequential channel-then-spatial attention with lightweight, differentiable gates—provides a principled template for attention integration in both complex- and real-valued domains, and could be adapted to other architectures with minimal parameter and computation expansion. Empirical results consistently demonstrate considerable improvements in task accuracy for modest computational cost across diverse vision and speech modalities (Zhao et al., 2021, Zhang et al., 2023).