---
title: 'RawJPEG Adapter: Invertible Raw-to-JPEG Pipeline'
url: https://www.emergentmind.com/topics/rawjpeg-adapter
type: topic
---

# RawJPEG Adapter: Invertible Raw-to-JPEG Pipeline

RawJPEG Adapter, published as **Raw-JPEG Adapter**, denotes a lightweight, learnable, and invertible preprocessing pipeline that adapts raw images for standard JPEG compression while preserving substantially higher raw reconstruction fidelity than direct raw-to-JPEG storage. In its canonical form, the method applies compact invertible transforms to a black-level-normalized demosaiced raw RGB image before standard JPEG encoding, stores the transform parameters inside the JPEG comment field, and inverts those transforms after JPEG decoding. The result is a standard JPEG file that remains widely interoperable, while a raw-aware decoder can reconstruct a raw-domain image with much higher fidelity than ordinary JPEG applied directly to raw data [2509.19624].

## 1. Definition and representational scope

In the literature, RawJPEG Adapter is primarily a storage-and-reconstruction method rather than a new image codec. The central object is a black-level-normalized demosaiced raw RGB image \(\mathbf{I}\), and the direct-storage baseline is written as
\[
\mathbf{I}' = Dec\left(Enc\left(\mathbf{I}\right)\right) \approx \mathbf{I}.
\]
The adapter inserts an invertible transform \(F\) with parameters \(\theta\):
\[
\mathbf{\hat{I}} = F^{-1}\left(Dec\left(Enc\left(F\left(\mathbf{I}; \theta\right)\right)\right); \theta\right).
\]
This formulation makes the JPEG file itself the transport format, while the learned adapter is an auxiliary reversible map around the codec [2509.19624].

A critical representational choice is that the method operates on **demosaiced raw RGB** rather than the Bayer mosaic directly. The paper states, “Given an input RGB demosaiced raw image \(\mathbf{I} \in \mathbb{R}^{H \times W \times 3}\)...”, and in experiments Bayer raw images are demosaiced using **Menon’s algorithm**. This means the method targets linear raw-domain RGB after sensor normalization, not native CFA storage. It therefore preserves raw-domain linearity and editability better than ordinary JPEG, but it is not a drop-in lossless replacement for Bayer-container formats such as DNG [2509.19624].

The method is also explicitly **not** a new codec standard. It does not alter JPEG syntax, entropy coding, or decoder semantics. A conventional viewer can decode the file as a normal image, but it will display the adapted JPEG image rather than the reconstructed raw. A raw-aware decoder reads the metadata stored in the **JPEG COM segment** and inverts the adapter to recover the raw-domain representation [2509.19624].

## 2. Problem setting: RAW, ISP, and JPEG mismatch

The motivation for RawJPEG Adapter is the structural mismatch between raw sensor data and standard JPEG. Raw data is typically **12–14 bits/channel**, linear, and shaped by sensor and illumination statistics. Standard JPEG stores **8-bit** image data, uses **YCbCr conversion**, usually **4:2:0 chroma subsampling**, **8×8 DCT**, and fixed quantization tables designed for natural sRGB-like images. Direct raw→JPEG therefore causes “banding in dark regions, clipping/highlight distortion, color shifts, and irreversible quantization artifacts,” even at high quality [2509.19624].

Related work sharpens the distinction between compression loss and earlier image-signal-processing loss. In underwater imaging, the main scientific problem with JPEG is stated to be “not due to JPEG compression itself, but to the in-camera processing that precedes it,” because proprietary photofinishing destroys the linear relationship between pixel values and scene radiance before compression occurs. This makes JPEG→RAW recovery fundamentally ill-posed for quantitative color recovery, especially underwater [2603.20823]. A plausible implication is that RawJPEG Adapter addresses a different problem: it does not attempt to invert arbitrary consumer JPEGs after unknown ISP, but instead preserves raw-domain information by controlling the transform **before** JPEG encoding.

A separate but related line of evidence comes from direct RAW vision. “Raw Instinct” reports that RAW-to-RGB conversion “does not add new capture information” for classification, and the total computation time from RAW image data to classification results can be **up to 8.46 times faster than RGB** on its controlled dataset when conversion is bypassed [2403.14439]. This suggests that the RAW/JPEG boundary is not merely a storage issue; it is also an interface problem between sensor-domain data and downstream computation.

## 3. Invertible architecture and transform components

The Raw-JPEG Adapter pre-encoding pipeline consists of three invertible components applied in sequence: **channel-wise 1D look-up tables (LuTs)**, an optional **global blockwise DCT coefficient scaling** transform, and a **pixel-wise gamma map**. These parameters are predicted from a thumbnail of the input image by a compact encoder-decoder network, then serialized into the JPEG comment field [2509.19624].

For each channel \(c \in \{R,G,B\}\), the adapter applies a monotonic 1D LUT with **128 samples**:
\[
\mathbf{I}^{LuT}_{c(x,y)} = LuT_c\!\left(\mathbf{I}_{c(x,y)}\right), \quad c \in \{R,G,B\}.
\]
Monotonicity is enforced through cumulative positive increments:
\[
\mathbf{L}(i) = cumsum\!\left(softplus(h_\theta)\right)_i.
\]
This redistributes raw intensities globally so that dark-region values are less vulnerable to 8-bit JPEG quantization [2509.19624].

The optional frequency-domain stage is aligned with JPEG’s own block structure. The image is divided into non-overlapping \(8\times 8\) blocks; for each block \(b\), the adapter computes a 2D DCT, multiplies by a single global learned **8×8** scaling matrix \(\mathbf{S}\), and reconstructs:
\[
\mathbf{I}^{DCT}_b = DCT^{-1}\!\left(\mathbf{S} \odot DCT(\mathbf{I}^{LuT}_b)\right).
\]
The scaling matrix is constrained as
\[
\mathbf{S} = \exp\left(0.7 \tanh(s_\theta)\right),
\]
which restricts values to approximately \([0.5, 2.0]\). This stage captures camera-specific frequency biases and better aligns raw statistics with JPEG’s fixed quantization tables, although its generalization across unseen cameras is weaker than the spatial-only variant [2509.19624].

The final adaptive stage is a spatially varying gamma field. The network predicts a **100×100 gamma map** \(\boldsymbol{\Gamma}\), upsamples it bilinearly to image resolution, and applies
\[
\mathbf{I}^{\Gamma}_{(x,y)} = {\mathbf{I}^{DCT}_{(x,y)}}^{\,\boldsymbol{\Gamma}_{(x,y)}}.
\]
Gamma values are constrained by
\[
\boldsymbol{\Gamma}_{(x,y)} = \exp\left(2 \tanh(g_{\theta_{(x,y)}})\right),
\]
mapping to approximately \([0.14, 7.4]\). The inverse pipeline performs inverse gamma, inverse DCT scaling if present, and inverse LUT application. Decode-time inversion is lightweight and deterministic; the decoder does **not** run a neural network [2509.19624].

Per image, the stored metadata consists of the **100×100 gamma map**, three **128-entry 1D LUTs**, and optionally one global **8×8 DCT scaling matrix**. These parameters are compressed with **zlib**, Base64-encoded, inserted into the **JPEG COM segment**, and kept **under 64 KB**, typically around **40 KB**. The predictive network itself has about **37K parameters** [2509.19624].

## 4. Training objective and codec coupling

Training is self-supervised: the target is the original demosaiced raw image itself. The pipeline predicts \((\mathbf{S}, \boldsymbol{\Gamma}, LuTs)\), applies the forward adapter, passes the result through a differentiable JPEG simulator, inverts the adapter, and compares the reconstruction \(\hat{\mathbf I}\) to \(\mathbf I\). The loss is
\[
\mathcal{L} = \lambda_{\text{L1}} \mathcal{L}_{\text{L1}} + \lambda_{\text{SSIM}} \mathcal{L}_{\text{SSIM}} + \lambda_{\text{FFT}} \mathcal{L}_{\text{FFT}},
\]
with \(\lambda_{\text{L1}} = 1.0\), \(\lambda_{\text{FFT}} = 0.1\), and \(\lambda_{\text{SSIM}} = 0.1\) [2509.19624].

The differentiable JPEG simulator models the standard JPEG path: RGB to **YCbCr**, **4:2:0 chroma subsampling**, blockwise **8×8 DCT**, quantization using luminance/chroma tables scaled by quality factor \(Q\), and inverse steps back to RGB. The quality scaling is
\[
f(Q) =
\begin{cases}
\dfrac{50}{Q}, & Q < 50,\\[4pt]
\dfrac{200 - 2Q}{100}, & Q \ge 50.
\end{cases}
\]
Coefficient quantization is modeled as
\[
\hat{C}_{u,v} = \mathrm{round}\!\left(\frac{C_{u,v}}{T_{u,v} f(Q)}\right),
\]
and rounding is approximated by a truncated Fourier series:
\[
\mathrm{round}(z) \approx z - \frac{1}{\pi}\sum_{n=1}^{N}\frac{(-1)^{n+1}}{n}\sin(2\pi n z).
\]
An ablation replacing this JPEG simulator with simple RGB 8-bit uniform quantization performs slightly worse, indicating that modeling actual JPEG internals matters [2509.19624].

Training uses the **S24 training set**, comprising **2,619** raw images from the Samsung S24 Ultra main camera, with a **400**-image S24 test set. Training is patch-based on non-overlapping **512×512** patches, for **100 epochs**, with **Adam**, learning rate **0.001**, \((\beta_1,\beta_2)=(0.9,0.999)\), weight decay \(10^{-4}\), and cosine decay to \(10^{-5}\). Separate models are trained for each JPEG quality \(Q \in \{100,95,75,50,25\}\). Augmentations include standard geometry, intensity scaling by a random factor in \([0.85,1.15]\), and a random \(3\times 3\) color transform with a small **+0.05 boost to the green row** before normalization [2509.19624].

## 5. Empirical behavior and performance

On the S24 test set, Raw-JPEG Adapter consistently outperforms direct JPEG-on-raw across all reported qualities. Representative results are shown below [2509.19624].

| JPEG quality | Direct JPEG PSNR | Raw-JPEG Adapter PSNR |
|---|---:|---:|
| 25 | 36.58 | 39.82 |
| 75 | 40.75 | 43.58 |
| 100 | 46.00 | 49.04 |

At **Q=100**, direct JPEG yields **46.00 PSNR**, **98.47 SSIM**, **99.72 MS-SSIM**, **4.36 BPP**, **CR 6.51**, whereas Raw-JPEG Adapter with DCT and augmentation reaches **49.04 PSNR**, **99.35 SSIM**, **99.89 MS-SSIM**, **5.28 BPP**, **CR 5.41**. Across qualities **25, 50, 75, 95**, the gain over direct JPEG is typically about **2.6–3.2 dB** on the training camera. The paper identifies **fixed gamma** as a strong simple baseline, but the full adapter remains better [2509.19624].

Cross-camera evaluation changes the ranking of components. At **Q=75**, the **spatial-only variant (without DCT)** generalizes best:

| Dataset at Q=75 | Direct JPEG PSNR | Best adapter PSNR |
|---|---:|---:|
| S7 | 40.94 | 43.46 |
| MIT-Adobe FiveK | 43.91 | 46.75 |
| NUS | 42.32 | 45.97 |

This pattern supports the paper’s conclusion that the DCT scaling matrix captures camera-dependent biases and helps most in-domain, while the spatial transforms carry better across unseen sensors [2509.19624].

The method also improves downstream rendering. For example, on S24 with **LiteISP** at **Q=75**, direct JPEG raw gives **20.78 PSNR / 78.99 SSIM**, fixed gamma gives **24.00 / 85.80**, and Raw-JPEG Adapter without DCT gives **24.60 / 87.99**. The paper further reports that at **JPEG quality 50**, the method can reduce storage to **under 2 MB** while producing Lightroom-rendered images perceptually similar to those from original DNGs larger than **30 MB** [2509.19624].

The framework is not strictly JPEG-specific. It also improves **LIC-TCM** and **JPEG 2000**, though the paper treats JPEG as the central practical target because of ecosystem compatibility. Conventional BPP rises relative to direct JPEG because of both transformed image statistics and metadata, but the paper introduces a weighted BPP,
\[
\mathrm{wBPP} = \frac{\mathrm{BPP}}{\log_2(1 + N_{\text{unique}})},
\]
to reflect preserved tonal richness [2509.19624].

## 6. Position within related adapter research

RawJPEG Adapter belongs to a broader family of interface methods that bridge raw sensor data, JPEG structure, and pretrained vision systems. One adjacent direction is **invertible ISP**: InvISP learns a bijective mapping between demosaiced RAW and RGB, includes a differentiable JPEG simulator, and is strongest when the forward ISP itself is under control, rather than for arbitrary legacy JPEGs [2103.15061]. Another direction is **RAW-to-pretrained-model adaptation**: RAW-Adapter combines input-level learnable ISP stages with model-level adapters to connect RAW/ISP priors to sRGB-pretrained backbones, addressing recognition rather than storage or reconstruction [2408.14802].

JPEG-specific front ends appear in several forms. JPEG-DL prepends a trainable JPEG layer as the first layer of an underlying DNN, replacing hard quantization with a differentiable soft quantizer; however, the downstream model still consumes a reconstructed RGB image, so it is a pixel-domain adapter rather than a raw-storage mechanism [2410.07081]. By contrast, “RGB no more” trains ViTs directly from minimally decoded JPEG features—specifically dequantized DCT coefficients—and reports up to **39.2% faster training** and **17.9% faster inference** with no accuracy loss, illustrating a compressed-domain adapter that bypasses full RGB decoding [2211.16421].

Backward-compatible JPEG augmentation provides another neighboring line. JPNeO adds learned operators at both encoding and decoding while maintaining full backward compatibility with the current JPEG format, and JPEG XL defines a distinct **losslessly transcoded JPEG** mode that reconstructs the original JPEG1 codestream. These works preserve legacy JPEG interoperability, but they target JPEG-domain fidelity and compatibility rather than raw-domain reconstruction from a JPEG-carried transform [2507.23521] [1908.03565].

## 7. Limitations and implications

Raw-JPEG Adapter is explicitly **not lossless raw compression**. Artifacts remain, especially at low quality. File size increases relative to direct JPEG because of both metadata and transformed-image statistics. The method is camera-sensitive: the optional DCT component helps on the training camera but hurts cross-camera generalization. The published system trains a **separate model for each JPEG quality**, works on **demosaiced raw RGB** rather than native CFA, and requires a **special decoder** to recover the raw-domain representation; generic viewers only show the adapted JPEG image [2509.19624].

These limits clarify its proper scope. The method is best understood as a learned, invertible **front-end/back-end for standard JPEG**, not as a universal inverse-ISP system and not as a raw-native replacement for all camera containers. A plausible implication is that its most durable contribution lies in the adapter principle itself: raw data can be “bent” into JPEG’s operating regime by compact invertible transforms, then “unbent” after decoding, allowing standard JPEG infrastructure to carry substantially more raw-domain information than ordinary direct storage would preserve [2509.19624].

Source: https://www.emergentmind.com/topics/rawjpeg-adapter