Beyond Model Design: Data-Centric Training and Self-Ensemble for Gaussian Color Image Denoising

Abstract: This paper presents our solution to the NTIRE 2026 Image Denoising Challenge (Gaussian color image denoising at fixed noise level $σ= 50$). Rather than proposing a new restoration backbone, we revisit the performance boundary of the mature Restormer architecture from two complementary directions: stronger data-centric training and more complete Test-Time capability release. Starting from the public Restormer $σ!=!50$ baseline, we expand the standard multi-dataset training recipe with larger and more diverse public image corpora and organize optimization into two stages. At inference, we apply $\times 8$ geometric self-ensemble to further release model capacity. A TLC-style local inference wrapper is retained for implementation consistency; however, systematic ablation reveals its quantitative contribution to be negligible in this setting. On the challenge validation set of 100 images, our final submission achieves 30.762 dB PSNR and 0.861 SSIM, improving over the public Restormer $σ!=!50$ pretrained baseline by up to 3.366 dB PSNR. Ablation studies show that the dominant gain originates from the expanded training corpus and the two-stage optimization schedule, and self-ensemble provides marginal but consistent improvement.

• The paper demonstrates that augmenting training data with a two-stage optimization yields a +3.37 dB PSNR gain over the baseline without changing the model architecture.
• It employs an eight-fold geometric self-ensemble during inference to enhance restoration quality, particularly in complex, texture-rich regions, despite increased computational cost.
• Experimental results highlight that focusing on data diversity and optimized inference strategies can outperform mere model modifications in Gaussian image denoising.

Data-Centric Training and Self-Ensemble for Gaussian Color Image Denoising with Restormer

Introduction

"Beyond Model Design: Data-Centric Training and Self-Ensemble for Gaussian Color Image Denoising" (2604.11468) systematically investigates the upper-bound performance of the Restormer backbone for fixed-level Gaussian color denoising (σ=50), emphasizing the roles of data-centric training strategies and test-time enhancement via geometric self-ensemble. The work builds upon the observation that, after the introduction of deep transformer-based architectures such as Restormer, the main performance constraint for image denoising shifts from model capacity to training data diversity and optimized inference protocols.

Methodology

Expanded Data-Centric Training and Two-Stage Optimization

The paper employs the unmodified Restormer architecture, deliberately shifting attention away from backbone modifications toward data and optimization. The standard multi-dataset training pipeline—usually confined to DIV2K, Flickr2K, BSD500, and WED—is significantly extended. The final corpus aggregates seven sources (up to 143,679 images), ensuring maximized visual diversity and coverage of natural texture statistics. Ultra-high-resolution images are pre-cropped and then patch-sampled.

The training is structured in two successive stages:

1. Stage I: Continues optimization from the public Restormer σ=50 checkpoint using DIV2K, Flickr2K, OST, and LSDIR, stabilizing restoration priors.
2. Stage II: Further augments with LIU4K-v2, NKUSR8K, and DIV8K, broadening the representational range and generalization.

Both stages use AdamW, MSE loss, and progressive patch size scheduling, leveraging multi-GPU setups.

Figure 1: Overview of the final denoising pipeline. A noisy image is restored by Restormer, the TLC-style wrapper is retained for inference consistency, and times 8 self-ensemble is used to refine the final denoised output.

Inference-Time Enhancement

At inference, the pipeline incorporates $\times 8$ geometric self-ensemble, in which all dihedral group symmetries (eight flip/rotation combinations) are applied to each input; outputs are inversely transformed and averaged. Additionally, a TLC-style local wrapper is retained, but analysis confirms its quantitative effect is negligible for this task.

Experimental Results

Quantitative Performance

The denoising pipeline reaches a PSNR of 30.76 dB and SSIM of 0.861 on the unified 100-image validation set at noise level σ=50, representing a +3.37 dB PSNR gain over the public Restormer baseline—without architectural changes. Most of this improvement is attributed to enhanced training data and two-stage optimization. $\times 8$ self-ensemble delivers an additional, stable 0.03 dB (PSNR) and 0.0004 (SSIM) boost but increases inference time by ∼8×.

Figure 2: Qualitative comparison among the noisy input, single-pass Restormer, Restormer with times 8 self-ensemble, final submission, and ground truth. The largest visual gains appear in line-rich, structured, and texture-dense regions.

Qualitative analyses indicate that self-ensemble primarily benefits visually complex, structured regions, with almost negligible improvement in low-texture areas.

Inference Time and Ablation

The only meaningful inference-time improvement stems from self-ensemble; the TLC-style wrapper does not contribute to performance at σ=50 under rigorous ablation. Self-ensemble entails substantial computational cost, raising peak GPU memory (∼36–37 GB) and wall-clock inference (from ∼1 s to ∼8.8 s per image on H200), but is justified if absolute restoration quality is the primary metric.

Figure 3: Same-protocol PSNR comparison with the public Restormer sigma = 50 pretrained baseline. The gap remains large in both the single-pass and times 8 self-ensemble settings; the corresponding SSIM gains are +0.0737 in both cases.

Ablation and Leaderboard Validation

Intermediate checkpoints reveal that the final gain after the introduction of even more diverse datasets is marginal, affirming that the majority of learning occurs soon after substantial data coverage is achieved. Public leaderboard submission (NTIRE 2026 challenge) confirms external validity, ranking 2nd with 29.89 dB PSNR and 0.87 SSIM.

Implications and Future Directions

This work demonstrates that backbone-agnostic improvements—specifically, data-centric corpus expansion and judicious test-time augmentation strategies—can yield substantial performance gains on challenging Gaussian denoising tasks, even with mature architectures. The near-plateau of improvement from model modifications (at least for tasks like additive white Gaussian noise denoising) suggests that future work should prioritize:

• Curating and scaling training corpora: Ensuring coverage of visual diversity is more rewarding than incremental model tweaks.
• Test-time inference strategies: Self-ensemble remains a reliable tool for robustifying prediction, although practical deployments should balance accuracy and efficiency.
• Transferability and domain generalization: As improvements saturate for fixed Gaussian noise, assessing the same strategies on real-world, non-additive, or spatially varying noise distributions is essential.
• Memory and speed trade-offs: Since gains from self-ensemble are modest, careful analysis is necessary for resource-constrained settings.

Conclusion

By decoupling architectural innovation from training and inference protocol optimization, this paper provides clear evidence that substantially strengthening training data and leveraging systematic self-ensemble at inference enables large improvements over previously accepted baselines for Gaussian color denoising. The results imply that future progress in image denoising—and potentially other restoration tasks—will depend less on novel backbones and more on how they are trained and deployed (2604.11468).