Auto White-Balance Correction for Mixed-Illuminant Scenes (2109.08750v2)

Abstract: Auto white balance (AWB) is applied by camera hardware at capture time to remove the color cast caused by the scene illumination. The vast majority of white-balance algorithms assume a single light source illuminates the scene; however, real scenes often have mixed lighting conditions. This paper presents an effective AWB method to deal with such mixed-illuminant scenes. A unique departure from conventional AWB, our method does not require illuminant estimation, as is the case in traditional camera AWB modules. Instead, our method proposes to render the captured scene with a small set of predefined white-balance settings. Given this set of rendered images, our method learns to estimate weighting maps that are used to blend the rendered images to generate the final corrected image. Through extensive experiments, we show this proposed method produces promising results compared to other alternatives for single- and mixed-illuminant scene color correction. Our source code and trained models are available at https://github.com/mahmoudnafifi/mixedillWB.

• The paper introduces a novel AWB approach that bypasses explicit illuminant estimation by blending multiple white balance settings using deep learning.
• It employs a neural network to predict pixel-wise weighting maps, achieving adaptive correction in scenes with mixed illumination.
• Evaluations on synthetic and standard datasets demonstrate improved accuracy with metrics like MSE, MAE, and ΔE 2000, emphasizing real-time potential.

Overview of "Auto White-Balance Correction for Mixed-Illuminant Scenes"

The paper "Auto White-Balance Correction for Mixed-Illuminant Scenes" by Mahmoud Afifi and colleagues presents a novel approach to auto white balance (AWB), specifically addressing challenges posed by mixed-illuminant scenes. Conventional AWB methods typically assume a single illuminant, which often results in suboptimal corrections when scenes are illuminated by multiple light sources. This research provides a method that circumvents the need for traditional illuminant estimation, instead using a machine learning framework to blend multiple predefined white-balance-rendered images of a scene.

Methodology and Contributions

The approach introduced in the paper refines the white balance correction process without explicit illuminant estimation. Here, images are rendered using a small set of predefined WB settings. These image versions are then used as input for a deep neural network that predicts pixel-wise weighting maps to blend the different WB settings, thereby generating the final corrected image. This method operates within a modified image signal processor (ISP) pipeline, offering significant advantages in mixed-illuminant contexts.

Key contributions of this method include:

1. Elimination of Canonical Illuminant Estimation: The method does not depend upon estimating the scene's global illuminant. Removing this requirement allows for more flexibility and accuracy in dynamically-lit environments.
2. Learning Weighting Maps: A neural network predicts local weighting maps, enabling personalized WB correction per pixel, which is particularly useful for scenes with varying lighting conditions.
3. Synthetic Test Set: The paper proposes a synthetic dataset with mixed-illuminant scenes. This test set provides a valuable resource for evaluating WB methods, featuring pixel-wise ground truth corrections.

Technical Evaluation

The authors conducted extensive evaluations using their synthetic test set, as well as established datasets like the Cube+ and MIT-Adobe 5K. The results from these evaluations demonstrate that their method performs competitively against both traditional and state-of-the-art AWB methods, particularly in scenes with mixed lighting. Notably, quantitative metrics such as mean square error (MSE), mean angular error (MAE), and ΔE 2000 showed favorable outcomes, indicating enhanced color correction capabilities of the proposed method.

Implications and Future Directions

The paper's method bears several implications for the development of ISP systems:

• Enhanced Mixed-Illuminant Support: By effectively managing scenes with multiple light sources, this approach supports richer photographic outputs in real-world conditions, where mixed lighting is common.
• Potential for Real-Time Processing: Given the modest overhead introduced by rendering additional small images, the approach is feasible for real-time applications, making it suitable for integration into consumer cameras and mobile devices.

Future developments may explore extending this methodology to handle even more complex lighting environments or incorporating additional scene context into the correction process. Furthermore, there is potential to investigate the application of this methodology to video processing, where temporal coherence of WB correction would present additional challenges and opportunities.

In summary, this research provides a substantial advancement in the field of auto white-balance correction, particularly benefitting scenes with varying light sources. The innovative use of machine learning to blend multiple WB settings positions this method as a valuable tool for enhancing image quality in diverse lighting conditions.