Wavelet-Based Directional Attention Module

• WDAM is a neural network component that uses multi-scale DWT to decompose features into directional subbands, capturing edges and textures.
• The module employs learnable attention gates with Haar wavelet transforms to selectively reweight high-frequency details for improved segmentation, classification, and restoration.
• Empirical results show WDAM boosts performance metrics in medical imaging, flicker removal, and anomaly detection while maintaining low computational overhead.

A Wavelet-Based Directional Attention Module (WDAM) is a neural network component that utilizes multi-scale discrete wavelet transforms (DWTs) in conjunction with learnable, orientation-aware attention mechanisms to enhance feature representations in convolutional and transformer architectures. WDAMs integrate directionally decomposed subbands—typically extracted with the Haar wavelet—into trainable attention or gating modules, allowing networks to selectively emphasize or attenuate features such as edges and textures along specific orientations. This approach has been systematically applied in domains including medical image segmentation, burst artifact removal, image classification, polyp detection, and industrial anomaly detection, where spatially structured high-frequency signals are critical for performance.

1. Core Principles and Transform Foundations

At its core, WDAM leverages the 2D discrete wavelet transform to separate input feature maps into four non-redundant, orthogonally oriented subbands: low–low (LL), low–high (LH), high–low (HL), and high–high (HH). The LL subband captures coarse approximations; LH and HL capture horizontal and vertical details, respectively; HH encodes diagonal/anti-diagonal fluctuations. The Haar wavelet is predominant due to its orthogonality and computational efficiency, as well as its direct association with local directional edges. Formally, for an input $X \in \mathbb{R}^{C \times H \times W}$, the DWT outputs

$$(X_{LL},\, X_{LH},\, X_{HL},\, X_{HH}) = DWT_{\text{Haar}}(X)$$

where the filtering and downsampling are implemented as channel-wise convolutions followed by subsampling (Zhang et al., 5 Dec 2025, Wu et al., 3 Mar 2026, Qu et al., 24 Mar 2026, Xiangyu, 2022, Tan, 3 Jul 2025).

2. Directional Attention Formulations

WDAM selectively re-weights directional subbands by learning distinct attention gates for each high-frequency branch (LH, HL, and sometimes HH). These gates are typically derived through channel-wise or spatial pooling followed by a multi-layer perceptron (MLP) or convolutional attention mechanism. For example, channel attention can be computed as

$$s_{\text{lh}} = \sigma(\mathbf{W}_2\,\text{ReLU}(\mathbf{W}_1\,\text{GAP}(X_{LH})))$$

with analogous computations for $s_{\text{hl}}$ and $s_{\text{hh}}$, where σ denotes the sigmoid and GAP is global average pooling. Alternative approaches concatenate high-frequency bands and apply joint 3×3 convolutions and sigmoids to produce normalized attention maps for each orientation (Xiangyu, 2022, Zhang et al., 5 Dec 2025, Qu et al., 24 Mar 2026).

A cross-fusion with spatial and channel-gated variants of the input feature map is often incorporated (e.g., following ACFA-style gates) to further reinforce multi-aspect feature selection. Attention-modulated subbands are then broadcast and fused, followed by inverse DWT to reconstruct the attended feature (Zhang et al., 5 Dec 2025, Wu et al., 3 Mar 2026, Xiangyu, 2022).

3. Architectures and Integration Strategies

WDAMs are modular and designed as plug-in blocks in various network stages:

• Medical decoding: WDAM is inserted parallel to spatial, Fourier, and cross-fusion modules within decoder blocks, as in U-Net variants. Multi-level (typically bi-level) wavelet decompositions enable the capture of fine-grained and coarse structures. The output is fused via $1 \times 1$ convolution and residual connections with the original feature stream (Zhang et al., 5 Dec 2025).
• Transformer architectures: In Flickerformer, WDAM replaces conventional self-attention in decoder stages, achieving substantial FLOP reduction while preserving or improving restoration quality (Qu et al., 24 Mar 2026).
• CNN backbones: WDAM replaces stride 2 convolutions or pooling layers, introducing essentially no extra learnable parameters beyond the small gating MLP (or convolutional blocks) (Xiangyu, 2022, Wu et al., 3 Mar 2026).
• Edge-guided segmentation: In frameworks such as MEGANet-W, wavelet-guided attention operates on decoder features, recalibrating them using fixed wavelet responses fused with reverse- and input-branch cues for weak boundary enhancement (Tan, 3 Jul 2025).

A tabulation of parameter costs and integration points across select architectures:

Application Domain	WDAM Insertion Point	Learnable Parameters
Medical Segmentation	Decoder (per block)	Conv (1×1, 3×3), BN, LN
Flicker Removal	Decoder (attention modules)	Convs, window projection
Image Classification	Downsample block in backbone	Small MLP per α
Anomaly Detection	Layer1 (residual block)	2-layer MLP (per WDAM)
Polyp Segmentation	Decoder (all stages)	Only in downstream fusion

4. Mathematical Formulations

Typical WDAM instantiation involves two principal steps: decomposition and attention-weighted synthesis. For a two-level variant (Zhang et al., 5 Dec 2025):

1. Decomposition: For $l=1,2$,

$$(LL^l, LH^l, HL^l, HH^l) = DWT(LL^{l-1}), \quad LL^0 = X$$

At the deepest scale, channel-reduced features $$S_{\bullet}^2 = \text{ReLU}(\text{Conv}_{1\times1}(X_{\bullet}^2))$$ are used to generate normalized attention maps:

$$A_p = \sigma(\text{Conv}_{3\times3}([S_{LL}^2; S_{HH}^2]))$$

$$A_h = \sigma(\text{Conv}_{3\times3}(S_{HL}^2)),\quad A_v = \sigma(\text{Conv}_{3\times3}(S_{LH}^2))$$

Attention-modulated subbands:

$$\tilde{LH}^2 = A_v \odot LH^2,\ \tilde{HL}^2 = A_h \odot HL^2,\ \tilde{HH}^2 = A_p \odot HH^2$$

2. Reconstruction: Attended details are synthesized via inverse DWT:

$$X_\text{wave}^{l-1} = IDWT(LL^l, \tilde{LH}^l, \tilde{HL}^l, \tilde{HH}^l)$$

At level 0, $X_{out} = IDWT(X_{\text{wave}}^1, 0, 0, 0)$

In both single- and multi-level designs, attention weights adaptively recalibrate the contribution of frequency- and orientation-specific components to the aggregate feature representation (Zhang et al., 5 Dec 2025, Wu et al., 3 Mar 2026, Xiangyu, 2022).

5. Empirical Efficacy and Computational Characteristics

Empirical studies consistently demonstrate that WDAM enhances task performance with minimal computational or parameter overhead. In segmentation, WDAM increases Dice score (DSC) and reduces boundary error (HD95) over baselines with ACFA, TFFA, and SMMM, yielding up to +0.9% DSC and +1.2 M parameters for medical segmentation (Zhang et al., 5 Dec 2025). In image restoration (e.g., flicker removal), WDAM improves PSNR by +0.33 dB while reducing FLOPs by ≈75% relative to global self-attention, and by ≈10G compared to window SA (Qu et al., 24 Mar 2026).

In image classification, ablating each subband's attention in WDAM reveals that concurrent attention to LH and HL yields the largest gains over the base MobileNetV2 (CIFAR-10 Top-1: 93.14% vs. 91.88%) (Xiangyu, 2022). In anomaly detection, WDAM confers improvements in image/pixel-level AUROC and PRO by up to +2.97%, +2.57%, and +8.84% respectively, while model size overhead is negligible (≈0.2 MB on WideResNet50-2) (Wu et al., 3 Mar 2026).

6. Application Domains and Contextual Efficacy

• Medical Image Segmentation: WDAM is integral within segmentation decoders for increased edge and boundary fidelity, particularly in blurred or complex boundary scenarios (organs, tumors) (Zhang et al., 5 Dec 2025).
• Burst Flicker Removal: WDAM, as part of Flickerformer, directly modulates restoration by targeting orientation-aligned flicker artifacts, outperforming Swin/ASSA attention modules and delivering improved perceptual quality at reduced cost (Qu et al., 24 Mar 2026).
• Polyp Boundary Detection: Multi-scale, parameter-free wavelet heads and directional wavelet attention modules deliver up to +2.3% mIoU and +1.2% mDice, focusing on weak and variable-contrast edges (Tan, 3 Jul 2025).
• Anomaly Detection: WDAM, via adaptive spectral subband re-weighting, highlights anomalies and suppresses irrelevant background, facilitating sensitive and efficient industrial visual inspection (Wu et al., 3 Mar 2026).
• General Classification: Extension to lightweight CNN backbones shows WDAM's utility in noise-suppressed, detail-enhanced classification (Xiangyu, 2022).

7. Implementation Considerations and Parameterization

Wavelet filters in WDAM are typically fixed (non-learnable), ensuring orthogonality and computational simplicity (2×2 Haar kernels). Trainable parameters reside only in the attention/convolutional layers and MLPs for subband weighting. Most WDAM variants are end-to-end trainable, with residual or additive fusion to maintain shape compatibility throughout the network. The number of levels ($L$), receptive field of attention convolutions, and choice of learned vs. fixed weighting determine WDAM's expressive and computational scope (Zhang et al., 5 Dec 2025, Qu et al., 24 Mar 2026, Xiangyu, 2022, Wu et al., 3 Mar 2026).

A plausible implication is that WDAM modules can serve as lightweight but potent augmentations for a range of architectures, delivering orientation-aware frequency selectivity without substantial training or inference penalties.

---

Key References:

• "Decoding with Structured Awareness: Integrating Directional, Frequency-Spatial, and Structural Attention for Medical Image Segmentation" (Zhang et al., 5 Dec 2025)
• "It Takes Two: A Duet of Periodicity and Directionality for Burst Flicker Removal" (Qu et al., 24 Mar 2026)
• "Wavelet-Attention CNN for Image Classification" (Xiangyu, 2022)
• "MEGANet-W: A Wavelet-Driven Edge-Guided Attention Framework for Weak Boundary Polyp Detection" (Tan, 3 Jul 2025)
• "Improving Anomaly Detection with Foundation-Model Synthesis and Wavelet-Domain Attention" (Wu et al., 3 Mar 2026)