Goal Conditioned Reinforcement Learning for Photo Finishing Tuning (2503.07300v1)

Abstract: Photo finishing tuning aims to automate the manual tuning process of the photo finishing pipeline, like Adobe Lightroom or Darktable. Previous works either use zeroth-order optimization, which is slow when the set of parameters increases, or rely on a differentiable proxy of the target finishing pipeline, which is hard to train. To overcome these challenges, we propose a novel goal-conditioned reinforcement learning framework for efficiently tuning parameters using a goal image as a condition. Unlike previous approaches, our tuning framework does not rely on any proxy and treats the photo finishing pipeline as a black box. Utilizing a trained reinforcement learning policy, it can efficiently find the desired set of parameters within just 10 queries, while optimization based approaches normally take 200 queries. Furthermore, our architecture utilizes a goal image to guide the iterative tuning of pipeline parameters, allowing for flexible conditioning on pixel-aligned target images, style images, or any other visually representable goals. We conduct detailed experiments on photo finishing tuning and photo stylization tuning tasks, demonstrating the advantages of our method. Project website: https://openimaginglab.github.io/RLPixTuner/.

Overview of "Goal Conditioned Reinforcement Learning for Photo Finishing Tuning"

The paper introduces a novel approach to automate the photo finishing tuning process using Goal Conditioned Reinforcement Learning (RL). This work addresses the inherent challenges in photo processing pipelines, such as those encountered in applications like Adobe Lightroom. The authors propose an RL framework capable of efficiently tuning photo parameters by treating the photo finishing pipeline as a black box, eliminating the need for differentiable proxies.

Introduction and Motivation

Photo processing pipelines have traditionally relied heavily on manual tuning, making the process time-consuming and often cumbersome. Recent approaches have attempted to automate this process either through zeroth-order optimization or differentiable proxies. However, these methods exhibit significant limitations when the parameter set is large or the target pipeline is non-differentiable.

The authors aim to overcome these challenges by employing a goal-conditioned RL framework. This approach is designed to interactively and efficiently find the desired set of parameters by conditioning on a goal image. The RL framework proposed shows substantial improvements, achieving optimal tuning with as few as 10 queries compared to the 200 required by traditional optimization methods.

Methodology

The RL-based approach is distinct from prior works due to its application of goal-conditioned RL to the photo tuning task. It utilizes state representation techniques specifically designed for photo finishing. The state representation consists of:

• Dual-path Feature Representation: This extracts both global and local features, which are crucial for tackling the tuning task. The dual-path includes a pair of local and global path convolutional encoders.
• Photo Statistics Representation: This provides traditional image statistics like histograms to aid in the representation of invariant features over different styles and content.
• Historical Action Embedding: Incorporates the history of actions taken, aiding the RL policy's decision-making process.

The process is modeled as a goal-conditioned Partially-Observable Markov Decision Process, and the policy is learned using the TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm. The reward functions are carefully crafted for both photo finishing tuning and photo stylization tasks.

Experimental Analysis

The authors evaluate their framework on standard datasets such as the MIT-Adobe FiveK and HDR+. The results demonstrate that their method significantly outperforms existing optimization and proxy-based approaches both in terms of efficiency and quality. For instance, in the FiveK-Target evaluation, the RL-based method achieves higher PSNR and SSIM values, clearly outperforming other techniques.

Furthermore, the paper includes extensive user studies indicating a preference for images processed by the proposed method, especially in stylization tasks. These results highlight the generalization capability of the proposed framework to unseen datasets and photo-finishing styles.

Implications and Future Work

The paper sets a solid foundation for utilizing reinforcement learning in the domain of photo processing. This framework not only advances the state of automation in photo finishing but also opens up opportunities for integrating similar approaches in other non-differentiable tasks.

Future research may explore expanding the RL framework to handle high-dimensional input variations and possibly integrate multi-modal inputs, such as natural language instructions for style goals. This holds promise for making photo editing more accessible and intuitive for users with varying expertise levels.

In summary, this work presents a robust step towards significantly reducing the manual effort required in photo finishing, leveraging reinforcement learning's capability to optimize complex and dynamic systems without explicit gradient information.