RAW Domain Diffusion Model

• RDDM is a generative diffusion framework that directly models RAW sensor data, offering enhanced fidelity over traditional post-processed methods.
• It employs specialized modules such as a RAW-domain VAE, Differentiable Post Tone Processing, and Configurable Multi-Bayer LoRA to capture device-specific details.
• The approach achieves improved restoration and generation metrics across imaging, audio, and biosignal domains through advanced stochastic processes and tailored loss functions.

The RAW Domain Diffusion Model (RDDM) is a class of diffusion-based generative frameworks that extend the denoising diffusion paradigm into the RAW data domain—whether the target is sensor RAW images, unprocessed audio waveforms, or low-level signal spaces. By operating at the signal source level and directly modeling the geometry and statistics of RAW data, RDDMs overcome the inherent limitations of conventional approaches that are constrained to post-processed or lossy data representations. These models are characterized by technical innovations in their noising processes, representation learning strategies, and specialized architectural components that adapt the diffusion framework for varied and challenging RAW data modalities.

1. Foundations and Modeling Rationale

The core motivation for RDDM arises from the inadequacy of sRGB-domain (or processed domain) restoration and generation models to recover fine-grained information that is either compressed or irretrievably lost in the raw-to-sRGB conversion pipeline. In image restoration, traditional pipelines typically decouple the processing into two distinct stages: an Image Signal Processing (ISP) module that demosaics and color-corrects the RAW sensor data, and a subsequent image restoration (IR) module that attempts denoising or super-resolution in the sRGB domain. This double-stage approach can lead to sub-optimal fidelity, increased artifacts, and a perpetual trade-off between perceptual quality and signal accuracy (Chen et al., 26 Aug 2025).

RDDM aims to bypass these limitations by defining both forward (noising) and reverse (denoising) stochastic processes in the high-dimensional RAW domain. By anchoring the generative process at the sensor's data acquisition level (e.g., Bayer pattern images for digital cameras), the model has access to statistically richer and less degraded signals, enabling superior restoration and generative fidelity. In addition to imaging, similar motivations exist in audio (where working in raw waveform space captures nuances lost in spectral or perceptual transforms) (Pavlova, 2023), and biosignal translation (where preserving temporal microstructure is essential) (Shome et al., 2023).

2. Architecture and Key Technical Components

Modern RDDMs are composed of several tightly integrated modules developed to address the unique properties of RAW data:

• RAW-domain Variational Autoencoder (RVAE): An encoder–decoder pair trained to produce robust, normalized latent representations for sensor RAW data. This ensures that the diffusion model operates within a tractable latent space, mitigating out-of-distribution (OOD) effects triggered by the non-Gaussian, device-dependent distribution of RAW signals. The RVAE is fine-tuned using domain-specific data, with latent normalization via scaling $\sigma^2 = (1/n)\sum_i(z_i-\mu)^2$, leading to robust whitening and regularization (Chen et al., 26 Aug 2025).
• Differentiable Post Tone Processing (PTP) Module: Designed to approximate the ISP and sRGB mapping as a differentiable layer, this module enables joint supervision in both the RAW (linear) and sRGB (perceptual) domains during training. The pipeline computes loss in both output spaces, ensuring that reconstructions maintain linear fidelity and also yield perceptually optimized sRGB outputs, thereby decreasing color deviations and mismatches (Chen et al., 26 Aug 2025).
• Configurable Multi-Bayer (CMB) LoRA: A parameter-efficient adaptation layer (Low-Rank Adaptation) that injects trainable groups for each Bayer filter configuration (e.g., RGGB, BGGR) into the encoder and/or diffusion U-Net. This enables the model to generalize across device-specific mosaic patterns, handling sensor heterogeneity in real-world deployments (Chen et al., 26 Aug 2025).
• Data Synthesis Pipeline: Since large, curated RAW datasets remain scarce, a scalable degradation process synthesizes low-high quality RAW image pairs by inverting tone mapping of sRGB images, applying synthetic sensor noise, and mosaicing with realistic Bayer patterns. This approach enables large-scale, domain-aligned training for RDDMs without extensive manual RAW data collection (Chen et al., 26 Aug 2025).

The architecture is extensible to audio (specialized 1D U-Nets (Pavlova, 2023)), signals with region-specific structures (ROI masks for biosignals (Shome et al., 2023)), and can incorporate modular guidance and regularization for further domain adaptation.

3. Forward and Reverse Diffusion Processes

RDDM generalizes the stochastic differential equation (SDE) framework prevalent in generative diffusion models by introducing parameterizations for spatial and/or structural components. In contrast to fixed, isotropic noise schedules (as in standard DDPM/VE SDEs), the forward noising process in an RDDM is often flexibly formulated as:

$dX_t = f(X_t)\,dt + \sqrt{2R(X_t)}\,dW_t$

where $R(x)$ is a Riemannian metric tensor accommodating the local geometric structure of the data manifold, and $f(x)$ is a drift term potentially crafted from the symplectic/Hamiltonian perspective (Du et al., 2022). Through constrained design (e.g., by requiring the stationary distribution to be standard Gaussian), this flexibility allows the noise schedule to be adapted or optimized for RAW data.

In practice, bespoke forward SDEs are complemented by specialized reverse processes; for example, in biosignal translation, forward noise is selectively injected via binary masks into ROI regions (such as QRS complexes in ECGs), and two separate denoising networks $\varepsilon_\theta, \rho_\phi$ are trained for ROI and background, respectively (Shome et al., 2023). In image applications, residual and noise diffusion branches are decoupled: residual diffusion deterministically encodes high-level structural consistency, while noise diffusion introduces texture diversity (Liu et al., 17 Apr 2025).

4. Training Objectives and Optimization

The RDDM learning objective typically combines variational score matching (e.g., minimizing the squared difference between the model's estimated score and the true data score) with additional regularization tailored to RAW data:

$$\mathcal{L}_{\text{ESM}} = \int_0^T \mathbb{E}_{X_s}\left[\frac{1}{2}\|s_\theta(X_s,s) - \nabla \log p_s(X_s)\|^2_{\Lambda(s)}\right]\,ds$$

For multi-domain or dual-space supervision, loss functions interleave MSE, perceptual (LPIPS), and adversarial terms over both the linear (RAW) and sRGB outputs after mapping through the PTP module. For example, the composite loss may take the form

$$L = L_{\text{VSD}}(\hat{X}_H^{\text{lin}}, X_H^{\text{lin}}) + \lambda_1 L_{\text{RAW}}(\hat{X}_H^{\text{lin}}, X_H^{\text{lin}}) + \lambda_2 L_{\text{rgb}}(F_{\text{PTP}}(\hat{X}_H^{\text{lin}}), F_{\text{PTP}}(X_H^{\text{lin}}))$$

where $F_{\text{PTP}}$ is the differentiable ISP mapping (Chen et al., 26 Aug 2025). Regularization terms can penalize excessive curvature in the drift vector field or impose smoothness across Bayer channel mappings.

5. Empirical Performance and Evaluation

RDDMs have demonstrated superior empirical performance in both reference-based (PSNR, SSIM, FID, LPIPS) and non-reference (NIQE, MUSIQ, CLIP-IQA) image restoration metrics. On real-world RAW-to-sRGB restoration benchmarks:

• RDDM attains lower LPIPS and higher perceived fidelity than sRGB-based diffusion competitors, with fewer perceptual artifacts and color mismatches, as supported by both objective metrics and user studies (Chen et al., 26 Aug 2025).
• The scalable data synthesis pipeline ensures the trained models generalize robustly to diverse sensor patterns and are competitive in terms of parameter efficiency and FLOPs.
• In audio, RDDMs equipped with progressive distillation achieve low Fréchet Audio Distance (FAD) and kernel inception distances, generating coherent, continuous musical outputs (Pavlova, 2023).
• In bio-signal translation, region-disentangled RDDMs reconstruct high-fidelity signals (e.g., reducing RMSE for ECG waveform synthesis) with a fraction of the original diffusion steps, yielding increased clinical utility (Shome et al., 2023).

The table below summarizes selected empirical properties:

Domain	Specialized Module	Dataset/Metric	RDDM Result
Image	RVAE + PTP + CMB LoRA	FID, LPIPS, NIQE	Outperforms sRGB DM
Audio	1D U-Net + Distillation	FAD, PKID/IKID	Lower FAD, higher IIS
Bio-signal	ROI Mask + Dual NNs	RMSE, CardioBench	Lower RMSE, SOTA

6. Comparison to Related Diffusion Models

RDDM generalizes and unifies several preceding architectures:

• It subsumes variance-preserving (VP), variance-exploding (VE), sub-VP, and critically-damped Langevin SDE models as special cases, by parameterizing the spatial and metric components in the forward SDE (Du et al., 2022).
• Integrating Riemannian geometry enables modeling of data lying on submanifolds or constrained domains, aligning with recent advances in generative modeling for structured data (Liu et al., 7 May 2025).
• The dual-branch (residual + noise) diffusion process is an extension over standard DDPM/DDIM, enhancing both certainty (deterministic restoration) and diversity (high-frequency texture synthesis) (Liu et al., 17 Apr 2025).
• RDDM's explicit adaptation to RAW data distinguishes it from methods like Relay Diffusion, which address frequency-domain SNR and conditioning issues across image resolutions but do not operate at the sensor domain (Teng et al., 2023).

7. Applications, Limitations, and Future Prospects

RDDM's primary impact lies in real-world signal restoration, imaging pipelines in edge and professional devices, and any domain where access to unprocessed sensor data is available and fidelity constraints are dominant. This includes photographic RAW restoration, medical and scientific imaging, low-level audio synthesis, and biosignal generation for clinical monitoring.

Potential limitations include dependence on RAW data access, the requirement for domain-aligned data synthesis when large RAW datasets are not available, and computational intensity in joint-domain supervision, particularly for high-resolution inputs. Future directions may include:

• Extension to video and burst RAW restoration via temporal-aware architectures.
• Integration with advanced ISP modeling for more fine-grained colorimetric and perceptual optimization.
• Automated adaptation to new sensor patterns and modalities via modular LoRA-style parameter sharing.
• Deployment in data-free synthesis settings using guidance layers informed by classifier statistics (as in DDIS) (Kim et al., 18 Jun 2025).

Arrayed against these challenges, the technical and empirical advances of RDDM frameworks mark them as a central technology in next-generation restoration and generative signal modeling, with a growing set of domain-specific derivatives across imaging, audio, biosignals, and more.