WeatherRemover: Multi-Weather Image Restoration

• WeatherRemover is an image restoration framework that removes multiple weather degradations like rain, snow, and fog using a UNet-based encoder–decoder and multi-scale transformer modules.
• It leverages multi-scale pyramid vision transformers with selective gating and channel-wise attention to compress features and reduce computational costs while preserving details.
• Experimental results reveal superior PSNR, SSIM, and real-time performance compared to baseline models, making it effective for multi-weather restoration in practical systems.

WeatherRemover

WeatherRemover refers both to a general class of image restoration models designed for the removal of multiple adverse weather-induced degradations (rain, snow, raindrop, fog) and to a specific model architecture, “WeatherRemover: All-in-one Adverse Weather Removal with Multi-scale Feature Map Compression” (Qu et al., 8 Apr 2026), which exemplifies this all-in-one design philosophy. Such models target single-image or video restoration and aim to maximize restoration quality, parameter efficiency, computational efficiency, and memory usage in practical computer vision systems subject to diverse and stochastic weather artifacts.

1. Architectural Overview

WeatherRemover (Qu et al., 8 Apr 2026) adopts a UNet-like encoder–decoder backbone combined with a multi-scale pyramid vision Transformer (MS-PVT), unified by selective gating mechanisms and multi-scale feature map compression. The principal architectural objectives are robust restoration under variable weather, compact model size, and real-time inference. The overall dataflow is as follows:

• The input image $I \in \mathbb{R}^{3 \times H \times W}$ is projected via a $1 \times 1$ convolution to an embedding $F_0$.
• The encoder consists of three downsampling stages (D1–D3); each stage comprises a main path (stacked MS-PVT blocks) and a bypass path, merged by a gating unit, followed by downsampling (pixel-unshuffle).
• A bottleneck stage applies further MS-PVT blocks.
• The decoder mirrors the encoder, employing pixel-shuffle upsampling and skip connections from the encoder outputs.
• Final refinement blocks further process feature maps; the network predicts a residual $\hat{R}$, added to the input $I$ to yield the restored output $\hat{I}$.

Distinctive design elements are as follows:

• Channel-wise attention modules: strong local channel selection using Squeeze-and-Excitation-type mechanisms at selected network points to enhance restoration of fine detail.
• Multi-scale spatial reduction attention (SRA): keys and values for self-attention are adaptively pooled, drastically reducing the computational and memory cost of attention from $O((HW)^2)$ to $O(HW \cdot P^2)$.
• Gating mechanisms: element-wise and channel-wise gates inserted both in the downsampling modules and within a custom gating feed-forward network (GFN), enabling dynamic selection of informative features and suppression of redundancy.

2. Component Modules and Information Flow

2.1 Encoder–Decoder and Gating Details

Each encoder stage processes feature map $F_k$ via:

$$\begin{aligned} P_m &= \mathrm{MSPVT}_n(\cdots(\mathrm{MSPVT}_1(F_k))) \ P_b &= \mathrm{MSPVT}(F_k) \ F_{k+1} &= \mathrm{Down}(\phi(P_m) \odot P_b) \end{aligned}$$

where $1 \times 1$0 is GELU and $1 \times 1$1 denotes element-wise product.

2.2 Channel-wise Attention

At specific locations, channel attention is computed as:

$1 \times 1$2

Here, $1 \times 1$3, $1 \times 1$4, $1 \times 1$5, $1 \times 1$6 are learnable, $1 \times 1$7 is the reduction ratio, $1 \times 1$8 is ReLU, $1 \times 1$9 is sigmoid.

2.3 Multi-scale Pyramid Vision Transformer Blocks

The MS-PVT blocks use a linear Spatial Reduction Attention module:

• Given $F_0$0, features for keys and values are pooled down to $F_0$1 spatial size prior to attention:

$F_0$2

• Attention is then computed as:

$F_0$3

with $F_0$4 derived from the unpooled $F_0$5, $F_0$6 from pooled representations.

2.4 Gating Feed-Forward Network (GFN)

For the gating unit in the FFN:

$F_0$7

Finally: $F_0$8

3. Training Procedures and Loss Functions

Training is conducted via the Pseudo-Huber loss for robust convergence:

$F_0$9

with $\hat{R}$0. Training employs Adam optimizer, a starting learning rate of $\hat{R}$1 annealed to $\hat{R}$2 over 800,000 iterations. Data augmentation includes random crops and progressive batch size scaling.

4. Efficiency and Complexity Analysis

WeatherRemover’s principal computational efficiency stems from two sources:

• Spatial Reduction in Attention: SRA reduces keys/values’ spatial dimensionality, lowering compute and memory costs for attention.
• Selective Gating: GFN and D-stage gating together boost PSNR by $\hat{R}$3 dB (Snow100K), with only moderate MACs increase ($\hat{R}$427% for GFN, $\hat{R}$55% for D-stage).

Empirically:

Model	Parameters (M)	MACs (G)	Inference Time (s, 640×480)	Memory (MiB)
WeatherRemover	24.3	301.8–377	0.11 – 0.16	109.6
Restormer	26.5	743.5	0.28	114.2
DRSformer	33.6	805.3	0.25	147.8
TransWeather	38.1	32.6	0.20	161.6

WeatherRemover achieves state-of-the-art or superior accuracy at a considerably reduced computational and memory cost compared to baseline multi-weather and single-weather models.

5. Experimental Results

Comprehensive evaluations are performed on Snow100K (desnowing), RainDrop (raindrop removal), and Outdoor-Rain (deraining+dehazing):

Task	Dataset	Mode	PSNR (dB)	SSIM	Time (s)	Params (M)
Desnowing	Snow100K-L	Single/Multi	32.26/30.87	0.927/0.912	0.11	24.3
Raindrop	RainDrop-A	Single/Multi	32.99/31.51	0.943/0.931	0.16	24.3
Derain+Haze	Outdoor-Rain	Single/Multi	32.56/31.52	0.933/0.917	0.13	24.3

Ablation studies confirm the synergistic effects of SRA and GFN (+0.52 dB), and channel-wise attention and gating provide significant increments in restoration accuracy. Visual analysis shows preservation of small details, removal of both fine and large artifacts (e.g., snowflakes, droplets), and retainment of shadow/texture signals on diverse weather.

6. Comparative Positioning Among Multi-Weather Restorers

Relative to representative all-in-one architectures:

• WeatherRemover demonstrates $\hat{R}$650% lower MACs than Restormer/DRSformer, with comparable or better accuracy and $\hat{R}$710–20% faster inference.
• Against TransWeather, WeatherRemover achieves higher PSNR for rain, snow, and raindrop test sets, despite a slightly slower inference speed, offset by much smaller parameter count.
• Compared to WeatherDiffusion, WeatherRemover infers orders of magnitude faster (one pass vs. iterative sampling) and with less memory/compute.

7. Limitations and Application Considerations

While WeatherRemover efficiently addresses multi-weather removal with strong generalization, certain limitations persist:

• Performance under extremely low-resolution or very heavy mixed snow-fog can degrade, as with most non-specialized models.
• High-resolution inference is memory-bound for large batch sizes or extremely high-resolution images.
• Model scalability is $\hat{R}$8 in input size, so resource-constrained deployment should consider cropping/tiling strategies.

Deployed on hardware such as RTX 3090, WeatherRemover achieves 7–9 FPS at 320×320 pixels, corresponding to real-time or near real-time processing for many vision pipelines.

---

WeatherRemover epitomizes the contemporary approach to all-in-one weather degradation removal via efficient Transformer-compressed feature processing, attention mechanisms aggressively optimized for locality/globality trade-offs, and careful gating—achieving robustness and efficiency demanded by practical multi-weather restoration tasks in computer vision (Qu et al., 8 Apr 2026).