PID: Physics-Informed Diffusion Model for Infrared Image Generation

Abstract: Infrared imaging technology has gained significant attention for its reliable sensing ability in low visibility conditions, prompting many studies to convert the abundant RGB images to infrared images. However, most existing image translation methods treat infrared images as a stylistic variation, neglecting the underlying physical laws, which limits their practical application. To address these issues, we propose a Physics-Informed Diffusion (PID) model for translating RGB images to infrared images that adhere to physical laws. Our method leverages the iterative optimization of the diffusion model and incorporates strong physical constraints based on prior knowledge of infrared laws during training. This approach enhances the similarity between translated infrared images and the real infrared domain without increasing extra training parameters. Experimental results demonstrate that PID significantly outperforms existing state-of-the-art methods. Our code is available at https://github.com/fangyuanmao/PID.

• The paper introduces the PID model that integrates physical infrared laws to translate RGB images into authentic infrared representations.
• It leverages a latent diffusion framework with TeV decomposition and augmented physical losses to enforce thermal properties.
• Experiments on KAIST and FLIR datasets show significant improvements in SSIM, PSNR, and LPIPS compared to conventional methods.

PID: Physics-Informed Diffusion Model for Infrared Image Generation

Introduction to the Concept

The paper presents the Physics-Informed Diffusion (PID) model, a novel approach for translating RGB images into infrared images through a physics-informed diffusion model. This method addresses the limitations of conventional RGB-to-infrared translation techniques that often overlook the physical laws governing infrared imaging. By integrating infrared physical laws into the training process of diffusion models, PID ensures adherence to real-world physical characteristics, significantly enhancing the quality and authenticity of the translated infrared images.

Theoretical Framework

Infrared image generation traditionally suffers from the lack of explicit consideration of physical principles. The PID model proposes a methodology that incorporates these principles into the generative process. Specifically, the latent diffusion model is leveraged for iterative optimization, introducing physical constraints using a self-supervised TeV decomposition network. The fundamental physical components—Temperature ($T$), Emissivity ($e$), and Thermal texture ($X$)—are computed, forming the backbone of the imposed physical constraints during the diffusion model's training.

Methodology

Diffusion Model Overview

The core of the PID model relies on the key concept of diffusion models, which employ a stochastic process to iteratively refine noisy images back to their clean state. The process is governed by the following sequence:

• Noising Process: Starts with a clean image $\boldsymbol{x}_0$ and successively adds Gaussian noise controlled by a parameter $\alpha_t$, following a Markov chain.
• Denoising Process: Commences with a pure noise vector, iteratively refining it through a learned variance-based schedule to revert it to a high-quality image.

Introduction of Physical Losses

The integration of physical laws is achieved by:

• TeV Decomposition: A custom decomposition mechanism designed to handle superimposed spectra from standard infrared cameras. This facilitates the prediction of thermal characteristics by transforming RGB images into conformant infrared representations.
• Loss Augmentation: Two additional losses are introduced:
  • Physical Reconstruction Loss ($\mathcal{L}_{\text{Rec}$): Ensures the refinements generate physically plausible images.
  • TeV Space Loss ($\mathcal{L}_{\text{TeV}$): Aligns generated images with real-world infrared thermal properties.
  • 

  • Figure 1: The overview of our proposed PID. (a) Firstly, the PID trains a $\mathcal{N}_{\text{TeV}$ with self-supervised loss on infrared dataset.

Experimental Evaluation

The PID model was rigorously tested on prevalent datasets like KAIST and FLIR to demonstrate its efficacy. Key performance metrics utilized include SSIM, PSNR, LPIPS, and FID, comparing PID against various baseline models, including GANs and conventional diffusion models.

Results on KAIST and FLIR

• KAIST Dataset: Positioned in diverse environments such as campus and city roads, the PID model displayed superior performance, particularly in generating accurate thermal properties at night.

  Figure 2: Qualitative results on KAIST dataset showcasing superior robustness of PID.

• FLIR Dataset: Illustrated PID's proficiency in translating intricate thermal features, especially under challenging low-light scenarios.

  Figure 3: Qualitative results on FLIR dataset with PID accurately depicting thermal components.

Conclusion and Future Work

The PID model significantly advances the translation of RGB images to infrared imagery by embedding physical laws into diffusion processes, resulting in more genuine infrared representations. The practical implications are expansive, promising better data augmentation methodologies for autonomous systems. Future endeavors will aim to refine the accuracy of the physical decomposition and enhance the diffusion model's generative capabilities, paving the way for more realistic synthesis of infrared imagery.