MSSIDD: A Benchmark for Multi-Sensor Denoising

Abstract: The cameras equipped on mobile terminals employ different sensors in different photograph modes, and the transferability of raw domain denoising models between these sensors is significant but remains sufficient exploration. Industrial solutions either develop distinct training strategies and models for different sensors or ignore the differences between sensors and simply extend existing models to new sensors, which leads to tedious training or unsatisfactory performance. In this paper, we introduce a new benchmark, the Multi-Sensor SIDD (MSSIDD) dataset, which is the first raw-domain dataset designed to evaluate the sensor transferability of denoising models. The MSSIDD dataset consists of 60,000 raw images of six distinct sensors, derived through the degeneration of sRGB images via different camera sensor parameters. Furthermore, we propose a sensor consistency training framework that enables denoising models to learn the sensor-invariant features, thereby facilitating the generalization of the consistent model to unseen sensors. We evaluate previous arts on the newly proposed MSSIDD dataset, and the experimental results validate the effectiveness of our proposed method. Our dataset is available at https://www.kaggle.com/datasets/sjtuwh/mssidd.

• The paper introduces MSSIDD, a new benchmark dataset and sensor consistency training framework for evaluating and improving multi-sensor raw-domain denoising models.
• MSSIDD contains 60,000 simulated raw images from six distinct sensors, designed to rigorously test the sensor transferability of denoising methods.
• The proposed training framework uses intra/inter-image consistency and adversarial learning to train models to extract sensor-invariant features, enhancing generalization to unseen sensors.

The paper "MSSIDD: A Benchmark for Multi-Sensor Denoising" introduces the Multi-Sensor SIDD (MSSIDD) dataset, a new benchmark designed to evaluate the sensor transferability of raw-domain denoising models across different sensors in mobile camera applications. This research addresses a gap in the current literature where existing industrial solutions either develop unique models for each sensor, leading to burdensome training processes, or simply apply existing models to new sensors without considering sensor disparities, resulting in suboptimal performance.

The MSSIDD dataset includes 60,000 raw images from six distinct sensors, derived by transforming sRGB images using various camera sensor parameters to simulate differences similar to those found in the real-world multi-sensor scenarios. An inverse transformation of the image signal processing (ISP) pipeline reverts sRGB images back to simulated raw images, replicating the degeneration processes of real cameras to include noise representative of different sensors.

Additionally, the authors propose a sensor consistency training framework that allows denoising models to learn sensor-invariant features, enhancing their ability to generalize to unseen sensors. This framework is reinforced through intra-image and inter-image consistency supervision by encouraging the network to extract similar features for the same images captured by different sensors. They complement this with an adversarial training mechanism that leverages a gradient reversal layer to minimize sensor-specific information in the extracted features.

The authors provide comprehensive evaluations using several state-of-the-art denoising models on this new dataset, demonstrating the efficacy of their approach and achieving notable improvements in image quality metrics like PSNR and SSIM over existing methods. They present these results under both Raw-to-Raw and Raw-to-RGB evaluation settings, showing that their proposed method, when integrated into models like NAFNet and Restormer (yielding MS-NAFNet and MS-Restormer), outperforms baseline methods and offers robust sensor generalization capabilities.

In conclusion, the MSSIDD dataset serves as a rigorous benchmark for future research focusing on multi-sensor denoising. The authors provide this resource for free, encouraging further exploration in sensor transferability research and offering their dataset as a vital tool for evaluating and developing advanced denoising methodologies.