RawGen: Diffusion-based Raw Image Generation

• RawGen is a diffusion-based framework that transforms text prompts and sRGB inputs into physically meaningful, scene-referred raw outputs.
• It combines conditional denoising with a specialized decoder and deterministic camera calibration to produce precise, camera-specific raw representations.
• The framework enables sRGB inversion and many-to-one supervision to robustly suppress photo-finishing variability for improved downstream low-level vision tasks.

Searching arXiv for “RawGen” and closely related papers to ground the article with current citations. {"^^^^^^^^7query7 OR all:RawGen7^^^^^^^^", "^^^^^^^^7max_results7 ^^^^7ti:RawGen OR all:RawGen7query7^^^^} Searching for inverse-ISP and raw image generation context to support positioning and related-work framing. {"^^^^^^^^7query7 ISP\" OR ^^^^^^^^7all:\7 image generation\" OR ^^^^^^^^7all:\7 raw\"", "^^^^^^^^7max_results7 ^^^^7ti:RawGen OR all:RawGen7query7^^^^} RawGen is a diffusion-based framework for learning camera raw image generation from either text prompts or sRGB inputs, producing scene-referred linear outputs in CIE XYZ and camera-specific raw representations for arbitrary target cameras (&&&^^^^7query7^^^^&&&). It is presented as, to the authors’ knowledge, the first diffusion-based framework enabling text-to-raw generation for arbitrary target cameras, alongside sRGB-to-raw inversion. Its central premise is that large-scale sRGB diffusion priors can be repurposed into physically meaningful linear image generation by combining conditional denoising, a specialized decoder, and deterministic camera calibration derived from DNG metadata (&&&^^^^7query7^^^^&&&).

^^^^7ti:RawGen OR all:RawGen7^^^^. Problem setting and nomenclature

RawGen is motivated by the distinction between scene-referred raw images and display-referred sRGB images. Scene-referred raw data retains linear radiometric information about the scene and is the appropriate domain for denoising, demosaicing, HDR reconstruction, illuminant estimation, and learning ISPs, because white balance, exposure scaling, and tone mapping have physically meaningful interpretations only on linear signals. By contrast, sRGB outputs are 8-bit, nonlinear, and already entangled with gamma, tone curves, contrast, color styling, and other photo-finishing operations. Those properties make sRGB a poor substrate for physics-aware image analysis and restoration (&&&^^^^7query7^^^^&&&).

The practical obstacle is dataset scarcity. Large-scale raw datasets are limited, camera-specific, and coupled to proprietary in-camera ISP behavior. As a result, each new sensor or ISP variant typically requires new capture and relabeling, while existing diffusion models remain optimized for photo-finished sRGB imagery rather than scene-referred linear representations. Classical inverse-ISP methods assume a fixed pipeline and therefore struggle when the input has passed through diverse, unknown, and nonlinear photo-finishing stages (&&&^^^^7query7^^^^&&&).

The name “RawGen” also appears in unrelated literature. In one large-systems paper, “RawGen” is interpreted as Retrieval-Augmented Generation rather than raw-image generation, so the term is not unique across arXiv usage (&&&^^^^7all:\7^^^^&&&).

^^^^7max_results7^^^^. Core objectives and conceptual contributions

RawGen repurposes a large sRGB text-to-image prior to produce physically meaningful scene-referred outputs. Its first contribution is text-to-raw generation for arbitrary cameras: the framework generates linear CIE XYZ first, then deterministically maps that output to a target camera raw-RGB space using calibration metadata such as ForwardMatrix and ColorMatrix from DNG files. This design makes the linear synthesis camera-agnostic while still allowing camera-specific raw outputs without retraining (&&&^^^^7query7^^^^&&&).

Its second contribution is sRGB-to-raw inversion. Given an sRGB image containing unknown photo-finishing, RawGen uses image conditioning to invert the input to a canonical linear XYZ anchor, after which the result can be mapped to a device raw space if desired. The framework therefore addresses both generative and inverse formulations within the same latent-space pipeline (&&&^^^^7query7^^^^&&&).

A third contribution is the many-to-one inverse-ISP dataset. For each underlying scene, multiple sRGB renditions generated with diverse ISP and photo-finishing parameters are anchored to a single scene-referred target. This supervision explicitly teaches the model to suppress stylistic variability and recover the common linear representation. The authors describe the resulting outputs as camera-centric linear reconstructions that outperform fixed-ISP inverse methods on heterogeneous inputs (&&&^^^^7query7^^^^&&&).

A fourth contribution is methodological rather than purely architectural: RawGen is designed as a data source for downstream low-level vision. Text-driven synthetic raw generation is used to scale training data for illuminant estimation, raw denoising, and neural ISP training, with reported gains over prior synthetic sources (&&&^^^^7query7^^^^&&&).

^^^^7query7^^^^. Architecture and learning formulation

RawGen builds on a pretrained rectified-flow DiT with native image conditioning, specifically the FLUX.^^^^7ti:RawGen OR all:RawGen7^^^^-Kontext backbone. The framework operates in latent space. A frozen VAE encoder PRESERVED_PLACEHOLDER_^^^^7query7^^^^ encodes sRGB inputs into latents, while a specialized VAE decoder PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7^^^^ is fine-tuned so that it decodes latents into linear CIE XYZ images rather than sRGB (&&&^^^^7query7^^^^&&&).

For sRGB-to-raw inversion, the input image is encoded into PRESERVED_PLACEHOLDER_^^^^7max_results7^^^^, and those conditioning tokens are concatenated with noisy target tokens. The combined sequence is then jointly processed through the DiT with LoRA adapters on attention projections; only the target tokens are supervised. For text-to-raw generation, the framework uses the base model’s text-to-latent path to obtain PRESERVED_PLACEHOLDER_^^^^7query7^^^^, and then runs the same conditional generation procedure to produce an XYZ latent. In both modes, the latent prediction stage is followed by XYZ decoding and then by deterministic camera-specific mapping (&&&^^^^7query7^^^^&&&).

The denoiser is trained with a rectified-flow PRESERVED_PLACEHOLDER_^^^^7all:\7^^^^-prediction objective. Let PRESERVED_PLACEHOLDER_^^^^7 OR all:\7^^^^ and define the straight-line noising path

PRESERVED_PLACEHOLDER_^^^^7 OR all:\7^^^^

The ground-truth velocity is

$v_{\mathrm{gt}} = \epsilon - z_{\mathrm{XYZ}},$

and the denoising loss is

$$\mathcal{L}_{\mathrm{denoise}} = \mathbb{E}_{k,\, t,\, \epsilon} \left\| v_{\mathrm{gt}} - v_{\theta}(z_t,\, t;\, z_{\mathrm{sRGB}^{(k)}}) \right\|_2^2.$$

The decoder is fine-tuned with an $L_1$ reconstruction objective,

PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7query7^^^^

which retargets the decoder from the sRGB domain to the XYZ domain while preserving spatial representation capacity (&&&^^^^7query7^^^^&&&).

^^^^7all:\7^^^^. Many-to-one inverse-ISP dataset and forward imaging model

The many-to-one dataset is central to RawGen’s invariance claim. For each raw scene, a canonical scene-referred target PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7ti:RawGen OR all:RawGen7^^^^ in CIE XYZ is derived by applying per-scene white balance and camera-to-XYZ conversion before any sRGB rendering. Multiple sRGB variants are then produced by varying white balance gains, tone mapping curves, and contrast around realistic ranges. The training set is therefore

PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7max_results7^^^^

where each PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7query7^^^^ is a different photo-finished rendition of the same underlying scene (&&&^^^^7query7^^^^&&&).

The representative forward ISP used to motivate the inverse problem begins from camera raw with black level PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7all:\7^^^^ and white balance gains PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7 OR all:\7^^^^:

PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7 OR all:\7^^^^

where PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen77^^^^ is demosaicing, PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7^^^^8 is a tone map, and PRESERVED_PLACEHOLDER_^^^^7ti:RawGen OR all:RawGen7^^^^9 is the sRGB gamma or OETF. Optional modules include denoising, sharpening, gamut mapping, and contrast or saturation adjustments (&&&^^^^7query7^^^^&&&).

In the reproducibility details, the randomized photo-finishing process is specified more concretely. White balance uses red and blue gains sampled as PRESERVED_PLACEHOLDER_^^^^7max_results7query7^^^^, with green fixed at PRESERVED_PLACEHOLDER_^^^^7max_results7ti:RawGen OR all:RawGen7^^^^. Tone mapping is applied per channel as

PRESERVED_PLACEHOLDER_^^^^7max_results7max_results7^^^^

and global contrast uses

PRESERVED_PLACEHOLDER_^^^^7max_results7query7^^^^

This construction explicitly exposes the model to ISP and photo-finishing diversity while keeping the target scene-referred representation fixed (&&&^^^^7query7^^^^&&&).

^^^^7 OR all:\7^^^^. Camera-specific mapping and inference modes

RawGen separates canonical linear generation from device-specific rendering by decoding into CIE XYZ first and only then mapping into the target camera’s raw-RGB space. The mapping uses DNG calibration metadata and interpolates matrices between two reference illuminants:

PRESERVED_PLACEHOLDER_^^^^7max_results7all:\7^^^^

PRESERVED_PLACEHOLDER_^^^^7max_results7 OR all:\7^^^^

The decoded XYZ image is mapped to white-balanced camera RGB by

PRESERVED_PLACEHOLDER_^^^^7max_results7 OR all:\7^^^^

the illuminant is converted as

PRESERVED_PLACEHOLDER_^^^^7max_results77^^^^

and device raw-RGB is obtained through

PRESERVED_PLACEHOLDER_^^^^7max_results7^^^^8

Optional heteroscedastic noise and CFA remosaicing can then be applied for more realistic raw outputs (&&&^^^^7query7^^^^&&&).

The sRGB-to-raw inversion mode takes an input image PRESERVED_PLACEHOLDER_^^^^7max_results7^^^^9, target camera PRESERVED_PLACEHOLDER_^^^^7query7query7^^^^, and optional text prompt PRESERVED_PLACEHOLDER_^^^^7query7ti:RawGen OR all:RawGen7^^^^, and first encodes the sRGB image as

PRESERVED_PLACEHOLDER_^^^^7query7max_results7^^^^

It then integrates the reverse rectified-flow ODE conditioned on PRESERVED_PLACEHOLDER_^^^^7query7query7^^^^:

PRESERVED_PLACEHOLDER_^^^^7query7all:\7^^^^

decodes the scene-referred XYZ image, and maps it to the target camera’s raw-RGB. The text-to-raw mode replaces the image encoder stage with the base model’s text-to-latent path,

PRESERVED_PLACEHOLDER_^^^^7query7 OR all:\7^^^^

after which the same conditional generation, decoding, and XYZ-to-raw mapping are used (&&&^^^^7query7^^^^&&&).

The implementation is intentionally lightweight in calibration requirements. The reported inputs are an sRGB image or a text prompt together with the target camera’s DNG metadata, and the paper states that using a single DNG file of the target camera suffices to obtain the required matrices (&&&^^^^7query7^^^^&&&).

^^^^7 OR all:\7^^^^. Training setup and empirical evaluation

The training data combines the MIT-Adobe FiveK and RAISE raw DNG collections. XYZ anchors are computed using DNG AsShotNeutral and ForwardMatrix, while multiple sRGB variants are rendered with a physically grounded software ISP and randomized photo-finishing. The denoiser uses the FLUX.^^^^7ti:RawGen OR all:RawGen7^^^^-Kontext DiT backbone with LoRA of rank PRESERVED_PLACEHOLDER_^^^^7query7 OR all:\7^^^^ and PRESERVED_PLACEHOLDER_^^^^7query77^^^^ on attention projections; the decoder is fine-tuned to XYZ using PRESERVED_PLACEHOLDER_^^^^7query7^^^^8 loss. Training uses PRESERVED_PLACEHOLDER_^^^^7query7^^^^9 crops, with XYZ anchors stored as ^^^^7ti:RawGen OR all:RawGen7 OR all:\7^^^^-bit PNG and sRGB variants as 8-bit PNG (&&&^^^^7query7^^^^&&&).

In the many-to-one invertability evaluation based on FiveK expert-retouched variations from editors A–E, RawGen is reported to achieve the best CIE XYZ reconstruction across all five styles against CIE XYZ Net, InvISP, and Raw-Diffusion. The reported PSNR/SSIM examples are A: ^^^^7max_results7query7^^^^.^^^^7max_results7query7 B: ^^^^7max_results7all:\7^^^^.^^^^7query7 OR all:\7^^^^/^^^^7query7^^^^.8 OR all:\7max_results7ti:RawGen OR all:RawGen7^^^^, C: ^^^^7max_results7query7^^^^.^^^^7query77 D: ^^^^7max_results7query7^^^^.^^^^7 OR all:\7ti:RawGen OR all:RawGen7^^^^/^^^^7query7^^^^.8 OR all:\7query7ti:RawGen OR all:RawGen7^^^^, and E: ^^^^7max_results7query7^^^^.89/^^^^7query7 OR all:\7query7query7^^^^. Typical baselines are summarized as approximately ^^^^7ti:RawGen OR all:RawGen7^^^^9–^^^^7max_results7ti:RawGen OR all:RawGen7^^^^ dB PSNR and approximately ^^^^7query7^^^^.78–^^^^7query7 SSIM, and an ablated one-to-one RawGen is described as substantially worse, which the authors use to highlight the role of many-to-one training (&&&^^^^7query7^^^^&&&).

The suppression of photo-finishing variability is also evaluated in latent space using PCA, t-SNE, and UMAP compactness over ^^^^7ti:RawGen OR all:RawGen7query7query7^^^^ graded variants per prompt. Mean distance to centroid is reported as PCA ^^^^7ti:RawGen OR all:RawGen7 OR all:\7query7^^^^.^^^^7query7^^^^, t-SNE ^^^^7ti:RawGen OR all:RawGen7query7^^^^.^^^^7 OR all:\7^^^^9, and UMAP ^^^^7ti:RawGen OR all:RawGen7^^^^.^^^^7query7 OR all:\77^^^^ for RawGen, compared with larger values for alternatives such as XYZNet and Raw-Diffusion. The paper interprets this as evidence that RawGen more strongly suppresses photo-finishing variability while preserving a common scene-referred anchor (&&&^^^^7query7^^^^&&&).

For device-specific synthesis, the decoded XYZ outputs are mapped to the Samsung Galaxy S^^^^7max_results7all:\7^^^^ main camera raw-RGB space with optional noise. Pre-trained neural ISPs trained only on real S^^^^7max_results7all:\7^^^^ data produce plausible sRGB from RawGen raw inputs without retraining, which the paper presents as an indicator of distribution alignment between the synthetic raw outputs and real-device raw data (&&&^^^^7query7^^^^&&&).

7. Downstream use, limitations, and significance

A major claim of RawGen is that synthetic raw data can improve downstream low-level vision systems. Using ^^^^7query7^^^^K generated samples and evaluating on real test splits, the paper reports gains over Graphics^^^^7max_results7^^^^RAW in three tasks. For illuminant estimation on NUS-8 with nine cameras, Graphics^^^^7max_results7^^^^RAW yields mean ^^^^7all:\7^^^^.^^^^7max_results7ti:RawGen OR all:RawGen7^^^^°, median ^^^^7query7^^^^.^^^^7query7 and worst ^^^^7max_results7 OR all:\7^^^^% 8.^^^^7 OR all:\77^^^^°, whereas RawGen yields mean ^^^^7query7^^^^.^^^^7ti:RawGen OR all:RawGen7all:\7^^^^°, median ^^^^7max_results7^^^^.^^^^7ti:RawGen OR all:RawGen7ti:RawGen OR all:RawGen7^^^^°, and worst ^^^^7max_results7 OR all:\7^^^^% 7.^^^^7query77^^^^°, approaching the real-data model at mean ^^^^7query7^^^^.^^^^7query7max_results7 median ^^^^7max_results7^^^^.^^^^7ti:RawGen OR all:RawGen77^^^^°, and worst ^^^^7max_results7 OR all:\7^^^^% ^^^^7 OR all:\7^^^^.77°. For neural ISP training on a nighttime dataset, Graphics^^^^7max_results7^^^^RAW gives PSNR ^^^^7query7^^^^8.^^^^7ti:RawGen OR all:RawGen7query7^^^^, SSIM ^^^^7query7^^^^.97^^^^7all:\7 and PRESERVED_PLACEHOLDER_^^^^7all:\7query7^^^^ ^^^^7max_results7^^^^.^^^^7query7query7ti:RawGen OR all:RawGen7^^^^, while RawGen gives PSNR ^^^^7query7^^^^8.^^^^7all:\7max_results7 SSIM ^^^^7query7^^^^.97^^^^7query7 and PRESERVED_PLACEHOLDER_^^^^7all:\7ti:RawGen OR all:RawGen7^^^^ ^^^^7max_results7^^^^.^^^^7ti:RawGen OR all:RawGen7max_results7query7^^^^, comparable to training on real raw at PSNR ^^^^7query7^^^^8.^^^^7query7max_results7 SSIM ^^^^7query7^^^^.97^^^^7all:\7 and PRESERVED_PLACEHOLDER_^^^^7all:\7max_results7^^^^ ^^^^7max_results7^^^^.^^^^7ti:RawGen OR all:RawGen7query7query7^^^^. For raw denoising, RawGen reports ^^^^7 OR all:\7query7^^^^.^^^^7 OR all:\7query7^^^^/^^^^7query7^^^^.99 at ISO ^^^^7ti:RawGen OR all:RawGen7 OR all:\7query7query7^^^^ and ^^^^7all:\7^^^^8.^^^^7 OR all:\77^^^^/^^^^7query7^^^^.99 at ISO ^^^^7query7max_results7query7query7^^^^, compared with ^^^^7all:\7^^^^9.^^^^7query77 OR all:RawGen7^^^^ and ^^^^7all:\7^^^^8.^^^^7ti:RawGen OR all:RawGen7 OR all:\7^^^^/^^^^7query7^^^^.989 for Graphics^^^^7max_results7^^^^RAW (&&&^^^^7query7^^^^&&&).

The reported advantages follow directly from the framework’s representation choice. By generating canonical XYZ, RawGen decouples scene synthesis from rendering, so white balance, exposure, and tone mapping remain reliable and camera-agnostic in the linear domain. Deterministic mapping to arbitrary camera raw spaces provides device-specific data without retraining, and optional noise plus CFA remosaicing makes the outputs suitable for raw-domain models. The use of text prompts further scales scene diversity without capture campaigns or graphics asset preparation (&&&^^^^7query7^^^^&&&).

The limitations are also explicit. Device fidelity beyond color remains incomplete, because accurate raw synthesis depends not only on color calibration but also on sensor noise, lens shading, point spread functions, and optical blur. RawGen currently injects heteroscedastic noise but does not model complex spatially varying characteristics. Inversion remains many-to-one, so extreme photo-finishing or heavy local edits can challenge recovery. Mapping quality also depends on correct DNG calibration; inaccurate or incomplete ForwardMatrix or ColorMatrix metadata degrades XYZ-to-raw accuracy. The paper therefore positions future work around richer device priors and learned physics for noise and optics (&&&^^^^7query7^^^^&&&).