PW-FNet: Pyramid Wavelet-Fourier Network

• The paper’s main contribution is an attention-free restoration architecture that isolates image degradations using multi-scale wavelet and Fourier processing.
• The methodology employs a 3-level encoder-decoder with trainable wavelet transforms and FFT-based token mixing to achieve global modeling at O(N log N) complexity.
• Practical results demonstrate competitive PSNR/SSIM on tasks like deraining, super-resolution, and deblurring with lower computational cost and latency.

Searching arXiv for the cited paper and closely related restoration baselines to ground the article in current literature. Pyramid Wavelet-Fourier Network (PW-FNet) is an image restoration architecture introduced in "Global Modeling Matters: A Fast, Lightweight and Effective Baseline for Efficient Image Restoration" (Jiang et al., 18 Jul 2025). It is designed as a fast, lightweight restoration baseline that preserves global modeling while removing the deployment barriers associated with self-attention. The model combines multi-scale wavelet decomposition at the inter-block level with global Fourier transforms at the intra-block level, with the stated objective of isolating degradation compactly in frequency space and processing it efficiently. Within the paper’s framing, the central claim is that image restoration benefits primarily from global modeling rather than from attention as a mechanism per se, and that wavelet-Fourier processing provides a more efficient route to that capability for deraining, raindrop removal, super-resolution, motion deblurring, dehazing, desnowing, and underwater or low-light enhancement (Jiang et al., 18 Jul 2025).

1. Motivation and problem formulation

PW-FNet is motivated by the observation that recent transformer-based restorers, including Restormer, Uformer, and DRSformer, derive strong restoration quality from long-range dependency modeling but incur substantial computational cost because self-attention scales quadratically, $O(N^2)$, in the number of spatial tokens. The paper further positions channel self-attention, windowed attention, anchor attention, Metaformer-style convolutional simplifications, NAFNet, and Mamba-like state-space models such as MaIR as attempts to reduce this cost, while arguing that such approaches often sacrifice either global context or practical efficiency (Jiang et al., 18 Jul 2025).

The underlying restoration-specific premise is that degradations such as haze veils, blur kernels, and rain streaks exhibit non-local structure across the image. According to the paper, restoration therefore requires long-range dependencies that conventional CNNs struggle to capture. The conceptual shift in PW-FNet is to treat the advantage of modern attention-based restorers as a consequence of global modeling rather than of attention itself. This yields an attention-free design in which wavelets first narrow the feature search space by decomposing the image into multi-scale, multi-frequency subbands, and Fourier transforms then provide global token mixing at $O(N \log N)$ cost (Jiang et al., 18 Jul 2025).

A key empirical insight supporting this design is the paper’s pyramid wavelet-Fourier iterative pipeline. Multi-scale wavelet decomposition reveals that degradation localizes into specific subbands, while subsequent Fourier analysis further concentrates those degradations into compact high-frequency regions. For deraining, the reported finding is that differences between rainy and clean images concentrate in high-frequency regions of the LL and HL bands across levels. This suggests that frequency-domain processing is not merely an efficiency device but also a way of aligning the architecture with the structure of restoration degradations.

2. Architectural organization

PW-FNet uses a 3-level hierarchical encoder-decoder with a pyramid wavelet-based multi-input multi-output structure, denoted PW-MIMO (Jiang et al., 18 Jul 2025). Given an image $I$, multiple wavelet transforms $W$ generate multi-scale inputs $\{I^i, i=1,2,4\}$. These branches correspond to pyramid levels or scales and preserve lossless multi-frequency components from the separable 2D discrete wavelet transform. Shallow features $\{f^i\}$ are extracted from each branch via a $3\times 3$ convolution.

Within each level, wavelet downsampling reduces resolution in the encoder and inverse wavelet upsampling increases resolution in the decoder. The wavelet transform produces the standard subbands LL, LH, HL, and HH by separable low-pass and high-pass filtering along rows and columns. LL captures coarse content, while LH, HL, and HH capture horizontal, vertical, and diagonal details. The paper does not add bespoke subband-specific heads; instead, the decomposition itself is treated as the mechanism that segregates content and detail frequencies. Downsampling and upsampling are implemented by trainable filters, and ablations compare Haar, Daubechies, Symlets, Coiflets, and Biorthogonal wavelet families.

After $N$ PW-FNet blocks per level, a $3\times 3$ convolution maps latent features to residual images $\{r^i\}$, and the outputs are computed by residual addition:

$O(N \log N)$0

This produces small (S), medium (M), and large (L) outputs from a single trained model by using the outputs of the last three stages as nodes. No separate training is required. In the paper’s practical interpretation, this dynamic multi-output design allows one backbone to serve multiple efficiency-quality operating points.

The intra-block component replaces self-attention with a global 2D FFT-based token mixer. For an input feature $O(N \log N)$1, the block first applies a pointwise $O(N \log N)$2 convolution to expand channels, then a global 2D FFT to map features into the frequency domain, then a frequency-domain $O(N \log N)$3 convolution and GELU for lightweight filtering or modulation, followed by inverse FFT and a final $O(N \log N)$4 convolution to restore the original channel dimension. The formalization given in the paper is

$O(N \log N)$5

where $O(N \log N)$6 denotes convolution, implemented here as pointwise convolutions (Jiang et al., 18 Jul 2025).

PW-FNet couples this global token mixer with an FFN derived from Uformer’s FFN, augmented by a depthwise $O(N \log N)$7 convolution for stronger local representation:

$O(N \log N)$8

The resulting block composition is therefore explicitly hybrid in scope: FFT provides global context, while depthwise convolution preserves local detail. Activation is GELU. Normalization layers are not reported, which the paper notes is consistent with lightweight Metaformer- and NAFNet-style blocks.

3. Frequency-domain foundations

PW-FNet is built on the combination of discrete Fourier analysis and wavelet multiresolution analysis (Jiang et al., 18 Jul 2025). The 2D DFT and its inverse are stated as

$O(N \log N)$9

$I$0

The convolution theorem,

$I$1

provides the rationale for frequency-domain filtering: pointwise convolutions in frequency space act as global filters while retaining negligible spatial footprint.

For the wavelet component, the 1D discrete wavelet transform uses low-pass $I$2 and high-pass $I$3 filters:

$I$4

The 2D extension is separable along rows and columns, producing

$I$5

where $I$6 denotes separable application.

The training objective is likewise frequency-aligned. The paper defines a Fourier-domain multi-output loss

$I$7

with ablations reporting that Fourier loss alone performs best. In the paper’s interpretation, this objective aligns supervision with the model’s core operating domain and improves restoration relative to spatial and wavelet losses, as well as their combinations.

4. Training protocol and empirical performance

PW-FNet is trained with AdamW using $I$8 and $I$9, batch size 24, patch size $W$0, and 500K iterations. The learning rate is $W$1 cosine annealed to $W$2. Reported data augmentation consists of random crop, rotation, and flipping. No task-specific heads or task-specific architectural adjustments are required; the network is trained per dataset, while the S/M/L outputs are derived from the same backbone via PW-MIMO (Jiang et al., 18 Jul 2025).

The reported benchmark coverage is broad. For deraining, the datasets include Rain200L/H, DDN-Data, DID-Data, SPA-Data, 4K-Rain13K, and Raindrop (A/B). Super-resolution uses DIV2K for training and Set5, Set14, BSD100, Manga109, and Urban100 for testing at scales $W$3, $W$4, and $W$5. Motion deblurring uses GoPro; dehazing uses NH-HAZE and DENSE-HAZE; desnowing uses SRRS and Snow100K; underwater enhancement uses LSUI; and low-light enhancement uses LOL-v2 in real and synthetic variants.

On Rain200L at $W$6, PW-FNet-L reports 1.442M parameters, 33.56G FLOPs, and 42.23/0.9915 PSNR/SSIM; PW-FNet-M reports 1.196M, 22.38G, and 41.82/0.9904; PW-FNet-S reports 0.719M, 16.64G, and 41.64/0.9900. The same comparison lists Restormer at 26.10M parameters, 141.0G FLOPs, and 40.99 PSNR. On Rain200H, PW-FNet-L reports 32.88/0.9413; on DDN-Data, 34.48/0.9606; on DID-Data, 35.54/0.9663. On real SPA-Data, PW-FNet-L reports 49.79/0.9936.

For high-resolution deraining on 4K-Rain13K at $W$7, the S, M, and L outputs report 39 ms with 35.02/0.9565, 50 ms with 35.77/0.9588, and 81 ms with 35.93/0.9607, respectively. UDR-Mixer is listed at 77 ms with 34.30/0.9505, while Restormer is reported at 1145 ms and DRSformer at 2682 ms. The paper also states that PW-FNet-S achieves 25 FPS at $W$8 for 4K deraining.

On GoPro motion deblurring, PW-FNet-L reports 34.03/0.968 with 5.98M parameters and 12.71G FLOPs at $W$9; M reports 33.72/0.966; S reports 33.30/0.964. In the same comparison, MaIR is listed at 26.29M parameters, 49.29G FLOPs, and 33.69/0.966, while Restormer is listed at 26.13M, 35.31G, and 32.92/0.961. At $\{I^i, i=1,2,4\}$0, the paper states that PW-FNet achieves +0.34 dB PSNR over MaIR with only 18.5% memory and 2.9% latency, corresponding to 259 MB versus 1400 MB and 28.6 ms versus 996.8 ms.

For super-resolution on Urban100, PW-FNet reports 215 MB memory and 29.7 ms latency at $\{I^i, i=1,2,4\}$1. The $\{I^i, i=1,2,4\}$2 setting reports an average of 35.05 dB/0.9398 on BSD100, Urban100, and Manga109, with 624K parameters and 118.9G MACs. The $\{I^i, i=1,2,4\}$3 setting reports 30.98 dB/0.8768 with 648K parameters and 56.0G MACs. The $\{I^i, i=1,2,4\}$4 setting reports 28.69 dB/0.8231 with 644K parameters and 35.2G MACs.

Additional reported task results are 20.81/0.82 on NH-HAZE and 17.95/0.66 on DENSE-HAZE for dehazing, 32.74/0.98 on SRRS and 34.50/0.95 on Snow100K for desnowing, 23.32/0.873 on LOL-v2-real and 26.02/0.941 on LOL-v2-syn for low-light enhancement, and 28.44/0.9489 on LSUI for underwater enhancement.

5. Ablations, qualitative analysis, and interpretive significance

The paper’s ablation study attributes the final performance profile to the joint action of PW-MIMO, wavelet selection, global Fourier mixing, and Fourier-domain supervision (Jiang et al., 18 Jul 2025). On Rain200L, PW-MIMO yields +0.12 dB over MIMO and +0.37 dB over SISO with minimal overhead. Among wavelet families, Biorthogonal is reported as best at 42.43/0.9917, while Daubechies is selected for the speed-performance balance at 42.23/0.9915. Windowed FFT ablations show that larger windows improve PSNR and that global FFT is best at 42.23/0.9915. The Fourier-only loss also outperforms spatial loss, wavelet loss, and combinations, again at 42.23/0.9915. Against wavelet- and Fourier-based baselines, the paper states that PW-FNet tops DAWN, WaveMamba, FADformer, and DMSR with the fewest parameters, lowest FLOPs, and fastest latency.

The qualitative analysis emphasizes that multi-frequency decomposition isolates degradation in specific subbands. For deraining, the HL subband is reported to carry prominent streak differences, and swapping HL across pyramid levels in the iterative pipeline is said to clarify this behavior. Fourier global modeling is then described as sharpening and compacting degradation regions, enabling precise filtering in frequency space.

The visual comparisons summarized in the paper assign distinct roles to the global and local branches of the architecture. For rain streak removal, PW-FNet is reported to preserve fine textures, such as clothing straps in Rain200H, while avoiding the over-removal observed in other models. On SPA-Data, it is reported to maintain better global consistency and fewer artifacts than NeRD-Rain, DRSformer, and Restormer. For raindrop removal, the paper notes smoother structures and more consistent restoration around edge transitions. On motion deblurring, it emphasizes stronger recovery of high-frequency details, including license plate characters, with fewer ghosts. On super-resolution, it reports stronger texture and edge fidelity and better high-frequency consistency across Urban100 scenes.

A common misconception addressed by the paper is that restoration improvements in recent high-performing models necessarily depend on self-attention. The stated interpretation is narrower: self-attention is useful because it provides global receptive fields, and FFT-based token mixing can supply that same global scope at lower cost. This does not imply that Fourier transforms subsume all relational modeling needs; it means only that, for the tasks tested, global modeling rather than attention per se is the decisive factor.

6. Relation to prior paradigms, deployment, and open directions

PW-FNet is positioned against three restoration lineages: attention-based transformers, state-space models, and earlier wavelet- or FFT-based networks (Jiang et al., 18 Jul 2025). Relative to Restormer and Uformer, it discards self-attention entirely and replaces quadratic token mixing with global FFT at $\{I^i, i=1,2,4\}$5 while maintaining global receptive fields. Relative to Mamba and MaIR, it avoids sequence recurrences and instead centers the design on frequency decomposition and global transforms. Relative to FFT-based models such as Fourmer, FFTformer, FFC, and GFNet, the paper claims novelty in combining a principled multi-scale, multi-frequency inter-block decomposition with dynamic multi-output inference. Relative to wavelet-based restorers such as DAWN and WaveMamba, the distinctive combination is trainable wavelet downsampling and upsampling, an intra-block global FFT token mixer, and a Fourier-domain loss.

The deployment profile is one of the paper’s central practical claims. At $\{I^i, i=1,2,4\}$6, reported latencies are around 27–30 ms and memory is approximately 214–259 MB across tasks. For deraining variants, the parameter footprint is at most 1.5M. The S/M/L variants are derived from the same training through PW-MIMO dynamic outputs, so device-specific quality-efficiency trade-offs can be selected without retraining. The paper reports that code is released at https://github.com/deng-ai-lab/PW-FNet. ONNX and TensorRT support are not reported; however, the FFT-heavy core is noted to map well to vendor libraries such as cuFFT.

The paper also identifies limitations and future directions. Performance is sensitive to the wavelet family, with Haar underperforming and Biorthogonal performing best in the reported ablation. Fourier windowing matters as well: global FFT performs best, and smaller windows reduce performance. Multi-output selection varies by degradation type; raindrop removal, for example, favors smaller-scale outputs, which indicates that task-adaptive output routing may be beneficial. More generally, the paper notes that FFT-based processing can exhibit ringing or boundary effects if frequency filters are not well regularized, although it suggests that the Fourier-only loss helps mitigate mismatch. Proposed extensions include learned or adaptive wavelets, subband-aware fusion or gating for LL/LH/HL/HH, and hybrid designs that combine attention with Fourier transforms for cases that require explicit relational modeling.

In aggregate, PW-FNet defines a restoration paradigm in which multi-scale wavelet decomposition is used to place degradations into structured frequency subspaces and global FFT token mixing is used to process those subspaces efficiently. Its reported contribution is therefore not only an attention-free replacement block, but a restoration-specific architecture in which decomposition strategy, global modeling mechanism, and training objective are all aligned in the frequency domain.