WFANet: Weather-Aware Saliency Detection

• The paper introduces WFANet, a dual-branch network that fuses weather prediction and saliency detection to address adverse weather degradations.
• It uses a dedicated weather branch to extract degradation-specific features and a saliency branch to aggregate multi-scale semantic cues, achieving superior results on the WXSOD benchmark.
• Quantitative results demonstrate significant improvements in MAE and S-measure over baselines, validating the effectiveness of weather-aware feature fusion under various conditions.

Searching arXiv for the specified paper and closely related sources. {"query":"arXiv (Chen et al., 17 Aug 2025) WXSOD Weather-aware Feature Aggregation Network WFANet", "max_results": 5} Weather-aware Feature Aggregation Network (WFANet) is a fully supervised, two-branch baseline for salient object detection (SOD) under adverse weather, introduced with the Weather-eXtended Salient Object Detection (WXSOD) benchmark in "WXSOD: A Benchmark for Robust Salient Object Detection in Adverse Weather Conditions" (Chen et al., 17 Aug 2025). It is designed for RGB-only SOD in scenes affected by fog, rain, snow, dark or low-light conditions, over-exposure, and several mixed degradations. The central premise is that weather noise should be modeled inside the detection architecture rather than handled by a separate restoration stage: WFANet therefore combines a weather prediction branch that mines weather-related deep features with a saliency detection branch that fuses semantic and weather features for weather-aware saliency inference.

1. Problem setting and motivation

Adverse weather degrades SOD in qualitatively different ways. Rain streaks and snow particles introduce spatially sparse but pervasive high-frequency artifacts that corrupt local details; fog causes large-area scattering and attenuation, washing out contrast and texture; low-light and over-exposure push content into extreme intensity ranges, collapsing discriminative cues (Chen et al., 17 Aug 2025). Under these conditions, standard SOD models trained on clean RGB datasets frequently miss boundaries, fragment the object through holes or broken contours, over-segment by confusing noise with structure, and mislocalize under global contrast collapse.

Within this framing, WFANet is positioned against two-stage "restore-then-detect" pipelines. The paper states that such pipelines add latency and are brittle because imperfect restoration can hallucinate or erase content, which can further degrade downstream segmentation. WFANet instead incorporates weather-aware representation learning inside the SOD architecture itself. A plausible implication is that robustness is treated as a representation-learning problem rather than as a preprocessing problem.

The network is therefore organized around an explicit decomposition. Branch-2 performs supervised multi-class weather recognition to mine degradation-specific features, while Branch-1 performs saliency detection by combining multi-scale semantic features with weather-conditioned information. This design aligns the representation space to the weather regime present in the input rather than assuming a single clean-image distribution.

2. WXSOD benchmark and task formulation

WXSOD comprises 14,945 RGB images with pixel-wise saliency masks and per-image weather labels (Chen et al., 17 Aug 2025). It is divided into a synthesized training set of 12,891 images, a synthesized test set of 1,500 images, and a real test set of 554 images. The paper does not specify a separate validation split.

Split	Images	Characteristics
Synthesized training set	12,891	Clean images processed into degraded variants; original may be retained
Synthesized test set	1,500	Each image unique, either degraded or clean
Real test set	554	Real-world weather noise curated from web images

The dataset uses nine weather labels, with $M=9$: five single-weather types—fog, rain, snow, dark/low-light, and over-exposure—three mixed types—fog+rain, fog+snow, and rain+snow—and one clean or no-noise category. Outdoor scenes were selected from DUTS, DUT-O, ECSSD, HKU-IS, and VT5000. Pixel-wise saliency masks were drawn by professional annotators. For the real test set, more than 1,500 web images were curated and filtered to 554 for balanced weather types and quality; masks were decided by majority vote with at least $4/7$ annotators and refined in Photoshop.

Synthetic degradations are generated through the imgaug library to add rain streaks, fog scattering, snow overlays, low-light, and over-exposure. Mixed types apply compound transforms. Each training RGB image is randomly processed into $2$ to $5$ degraded variants, and the original may be retained. Noise intensity is multi-level. No explicit physical scattering or streak-formation equations are provided; synthesis relies on image-level style transforms.

WXSOD covers more than 5,000 scenes with varying object sizes and counts. In all three splits, most images contain one salient object, approximately $90.7\%$, $95.6\%$, and $97.2\%$ for train, synthesized test, and real test, respectively. Object sizes skew small and middle, with the detailed small/middle/large breakdown reported in Table 1 of the paper. This benchmark structure is intended to test both in-distribution behavior on synthesized degradations and out-of-distribution generalization on real weather noise.

3. Architectural design of WFANet

WFANet is an end-to-end dual-branch network composed of a weather prediction branch and a saliency detection branch (Chen et al., 17 Aug 2025). The weather branch supplies weather-aware features at multiple scales, and the saliency branch performs cross-scale aggregation and final prediction.

Weather prediction branch

Branch-2 uses ResNet-18 as its backbone. Given an RGB input $I_{in} \in \mathbb{R}^{3\times H\times W}$, the backbone extracts a high-level feature:

$F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$

A Weather Prediction Module (WPM) with three stacked $CNR^1$ blocks, each defined as Conv $4/7$0 BN $4/7$1 ReLU, produces

$4/7$2

The resulting feature has shape $4/7$3.

A classifier defined as FC $4/7$4 BN $4/7$5 Dropout $4/7$6 FC produces logits $4/7$7, where $4/7$8. The corresponding cross-entropy supervision is

$4/7$9

with $2$0 the one-hot ground-truth weather label. The feature $2$1 is then upsampled by bi-cubic interpolation to $2$2, yielding weather-aware representations at spatial scales $2$3, $2$4, $2$5, and $2$6.

Saliency detection branch

Branch-1 uses PVTv2-b to extract four semantic feature maps $2$7 at resolutions $2$8, $2$9, $5$0, and $5$1. These are refined by a Cross Fusion Module (CFM), which progressively fuses adjacent scales:

$5$2

and, for $5$3,

$5$4

while $5$5. The element-wise multiplication $5$6 acts as feature gating. According to the paper, CFM strengthens cross-scale spatial dependencies and suppresses redundancy.

At each scale, a Multi-Source Fusion Module (MSFM) merges semantic features $5$7 with weather features $5$8. In schematic form,

$5$9

followed by

$90.7\%$0

Each $90.7\%$1, and concatenation over four scales gives $90.7\%$2.

The final predictor is channel-attentive. It applies global average pooling over $90.7\%$3, then $90.7\%$4 to generate four channel weights, followed by channel-wise modulation, a $90.7\%$5 convolution, and a sigmoid:

$90.7\%$6

Here $90.7\%$7 has 256 output neurons and $90.7\%$8 outputs 4 channel weights. The paper describes this predictor as squeeze-and-excitation-like.

4. Joint learning, optimization, and implementation

WFANet is trained end-to-end with a multi-task objective in which the saliency branch is supervised by a hybrid BASNet-style loss and the weather branch by cross-entropy (Chen et al., 17 Aug 2025). The paper presents the total objective in the form

$90.7\%$9

It states that $95.6\%$0 comprises BCE, SSIM, and IoU losses, citing Qin et al., but does not specify a weighting coefficient $95.6\%$1 or any scheduling. The text notes that, because $95.6\%$2 are consumed by the MSFM in the saliency branch, gradients from $95.6\%$3 can flow into the WPM, while $95.6\%$4 keeps the weather features discriminative for weather classes. This creates a synergy between weather supervision and saliency supervision.

The implementation is in PyTorch on a single NVIDIA RTX 3090. Optimization uses Adam with an initial learning rate of $95.6\%$5, halved every 30 epochs. Training uses batch size 6, input resolution $95.6\%$6, and 50 epochs. The backbones are PVTv2-b for Branch-1 and ResNet-18 for Branch-2. Data augmentation consists of random horizontal flip, rotation, and boundary clipping. No additional weather-aware augmentation beyond the dataset itself is described. Metrics are reported on original-resolution synthesized and real test images during inference.

The evaluation protocol uses S-measure, MAE, E-measure $95.6\%$7, F-measure $95.6\%$8, and PR and F-measure curves. Computational costs are reported as Params (M) and MACs (G) measured at $95.6\%$9.

5. Quantitative results and ablation evidence

The paper reports comprehensive comparison against 17 SOD methods and states that WFANet achieves superior performance on WXSOD (Chen et al., 17 Aug 2025). On the synthesized test set of 1,500 images, WFANet achieves MAE $97.2\%$0, S $97.2\%$1, $97.2\%$2, $97.2\%$3, $97.2\%$4, $97.2\%$5, $97.2\%$6, and $97.2\%$7. Compared with GPONet, the reported MAE reduction is approximately $97.2\%$8, computed as $97.2\%$9.

On the real test set of 554 images, WFANet again leads, with MAE $I_{in} \in \mathbb{R}^{3\times H\times W}$0, S $I_{in} \in \mathbb{R}^{3\times H\times W}$1, $I_{in} \in \mathbb{R}^{3\times H\times W}$2, $I_{in} \in \mathbb{R}^{3\times H\times W}$3, $I_{in} \in \mathbb{R}^{3\times H\times W}$4, $I_{in} \in \mathbb{R}^{3\times H\times W}$5, $I_{in} \in \mathbb{R}^{3\times H\times W}$6, and $I_{in} \in \mathbb{R}^{3\times H\times W}$7. The paper reports MAE improvements of approximately $I_{in} \in \mathbb{R}^{3\times H\times W}$8 and $I_{in} \in \mathbb{R}^{3\times H\times W}$9 relative to GeleNet and GPONet, respectively. PR and F-measure curves show the largest area under curve and the curves closest to the upper-right corner.

Test set	MAE	S
Synthesized test set	0.0229	0.9051
Real test set	0.0159	0.9248

The ablation studies identify the role of each main component. On the real test set, a variant without the weather branch achieves MAE $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$0, S $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$1, and $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$2, which is worse than the full model and therefore supports the importance of weather-aware features. A variant with the weather branch but without classification loss obtains MAE $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$3 and S $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$4, indicating that even unsupervised auxiliary features help; adding $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$5 further improves all metrics. Removing the CFM also reduces performance, which the paper interprets as evidence that cross-scale fusion and gating reduce redundancy and stabilize boundaries.

Additional ablations examine WPM and MSFM depth. Increasing the number of $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$6 and $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$7 units from 1 to 3 yields consistent gains; beyond 3, performance saturates while Params and MACs grow. The default design therefore uses three $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$8 units in WPM and three $F^{res}=\mathcal{T}^{resnet}_{extractor}(I_{in}).$9 units in MSFM as the reported best trade-off. For backbone selection, the paper states that Branch-1 is the critical driver and that PVTv2-b and Swin-B provide the strongest feature extraction under weather noise, outperforming ResNet-50 and VGG-16. The recommended combination is PVTv2-b for Branch-1 and ResNet-18 for Branch-2, balancing robustness and cost. It also notes that using very heavy backbones simultaneously, such as PVTv2-b with Swin-B, can hamper effective fusion under a unified training strategy.

6. Complexity, qualitative behavior, limitations, and significance

WFANet has 50.87M parameters and 112.63G MACs at $CNR^1$0 (Chen et al., 17 Aug 2025). Runtime and memory footprint are not reported. The dual-branch design and multi-stage fusion therefore impose substantially higher cost than lightweight baselines such as MINet at $CNR^1$1M parameters and $CNR^1$2G MACs, although the paper associates this increase with substantially higher accuracy. It suggests pruning, quantization, or knowledge distillation as possible deployment-oriented directions.

Qualitatively, the reported success cases show that under snow+fog and intense rain, WFANet preserves object integrity and complete contours where baselines produce holes or leak background. In high-exposure scenes, it isolates targets such as traffic cones cleanly while other methods wash out boundaries. Relative to PVT-based GeleNet and GPONet, the paper attributes improved results to weather-conditioned fusion that avoids misclassifying streak and flake artifacts as object edges and retains finer boundary fidelity.

The reported failure cases are also explicit. Dense rain and snow can still induce boundary errors, and extreme fog or low-light can cause missing parts in highly attenuated regions. The paper illustrates four such failures and points to future spatial-domain and frequency-domain analysis, as well as multi-modal inputs, as possible remedies. This suggests that the nine-class weather labeling scheme and imgaug-based synthesis do not exhaust the space of adverse-weather degradations, especially under extreme intensity or unusual mixed-weather statistics.

Within the paper’s contribution claims, WFANet serves as a benchmark baseline as much as a standalone model. WXSOD is presented as the first large-scale RGB SOD dataset explicitly targeting adverse weather, with pixel-wise masks, weather labels, and both synthesized and real test sets. WFANet contributes a weather prediction branch for explicit weather-specific feature mining, a Cross Fusion Module for gated cross-scale refinement, and a Multi-Source Fusion Module for integrating semantic and weather features at every scale. The code and benchmark results are stated to be publicly available at the project repository, and the recommended settings are input size $CNR^1$3, Adam with learning rate $CNR^1$4 and decay every 30 epochs, batch size 6, 50 epochs, PVTv2-b in the saliency branch, ResNet-18 in the weather branch, standard augmentations, and inference at original test resolution.