JPEG AI: Neural Image Compression

• JPEG AI is a neural network–based image coding standard that replaces traditional block transforms with end-to-end optimized deep models.
• It employs an autoencoder-based variational pipeline with adaptive transforms, hyperpriors, and spatial quality mapping to achieve continuous variable-rate coding.
• The standard supports both human visual tasks and machine-vision applications, ensuring robust interoperability and bit-exact decoding across multiple platforms.

JPEG AI is the first international standard for end-to-end neural-network–based image coding designed to serve both human visual tasks and machine consumption. Developed by the Joint Photographic Experts Group (JPEG), its technical core replaces classical block-based hand-engineered transforms (such as DCT in traditional JPEG) with globally optimized deep neural networks. JPEG AI provides a single-stream, compressed-domain representation and aims for broad interoperability, outperforming prior standards in both perceptual and rate-distortion metrics by leveraging adaptive, learned transforms and probabilistic entropy models (Esenlik et al., 13 Oct 2025).

1. Technical Foundations and Architecture

JPEG AI coding employs an autoencoder-based paradigm structured as a rate-distortion–optimized variational pipeline. The standard distinguishes itself by the following sequence of normative modules (Esenlik et al., 13 Oct 2025, Jia et al., 20 Mar 2025, Jia et al., 2024, Jenadeleh et al., 7 Apr 2025):

• Encoder (Analysis Transform): An initial preprocessing step converts images to YCbCr (4:2:0, 4:2:2, or 4:4:4). The luma component is encoded by a convolutional analysis network to yield a latent tensor. Chroma is processed via a secondary encoder, often concatenated with luma for improved energy compaction.
• Hyperprior Branches: Both primary (luma) and secondary (chroma) latents are fed into hyper-encoders, yielding hyper-latents quantized and entropy coded into a dedicated Z-stream.
• Context and Residual Coding: Hyper-decoders provide distribution parameters for entropy coding of the primary/secondary residual streams via multi-stage context models (MCM). The arithmetic coder leverages tabulated CDFs and supports multithreading and skip modes for computation efficiency.
• Synthesis Transform (Decoder): Three different decoder branches (decoderID=0/1/2) provide trade-offs between quality and complexity, with optional post-processing filters (e.g., EFE, ICCI, LEF).
• 3D Quality Map and Bitrate Adaptation: A 3D quality map M[c, i, j] enables continuous variable-rate coding and region-of-interest (RoI) control by scaling channel/spatial elements of the latent tensors, allowing precise rate allocation up to 2.0 bpp (Jia et al., 20 Mar 2025).
• Output: The decoder reconstructs the RGB image after inverse color-space conversion.

This architecture is encapsulated in a single codestream with segment markers and fully documented in Parts 1–5 of the JPEG AI standard, supporting robust software/hardware deployment and bit-exact interoperability at the entropy-decoder stage (Esenlik et al., 13 Oct 2025).

2. Rate–Distortion Optimization and Training

JPEG AI is trained end-to-end using a rate–distortion Lagrangian objective,

$L = R + \lambda D$

where $R$ is estimated rate from the entropy models, $D$ is distortion (e.g., mean squared error or mixed perceptual loss), and $\lambda$ sets the trade-off. Training proceeds in staged fashion: initial epochs use pure MSE, followed by mixed objectives (MSE + MS-SSIM), decoder/hyperprior/gain tuning, and fine adjustment of the gain unit for continuous variable-rate operation (Jia et al., 20 Mar 2025). Hyperparameter selection is refined through cross-validation on diverse image corpora (CTTC, Kodak, CLIC datasets) (Esenlik et al., 13 Oct 2025).

JPEG AI profiles provide several pretrained model sets (with varying $\lambda$) to span the rate range and enable continuous adaptation via gain modulation at inference (Jia et al., 20 Mar 2025). The entropy coder is trained to maximize compression efficiency under the provided hyperprior and context models.

3. Compression Performance, Adaptivity, and Bit Allocation

Extensive benchmarking against anchor codecs such as VVC Intra (VTM 11.1) demonstrates JPEG AI’s substantial BD-rate gains, summarized in Table 1:

Profile	BD-rate Gain vs. VVC Intra
Main@Simple (decID=0)	–16.2 %
Main@Base (decID=0/1)	–20.2 %
Main@High (decID=0/1/2)	–22.1 %

(Esenlik et al., 13 Oct 2025)

Performance is measured using seven perceptual metrics: MS-SSIM, FSIM, VIF, VMAF, PSNR-HVS, IW-SSIM, NLPD, and confirmed across test sets (Kodak: –21.1 %, CLIC 2024: –24.9 %) (Esenlik et al., 13 Oct 2025).

JPEG AI adapts bit allocation both globally and locally. The standard default pipeline employs per-channel gain adjustment; block-level RoI gains are achieved via the spatial quality map. By synthesizing a quality index map inspired by VVC, it is possible to localize bits to high-saliency regions, producing up to +0.45 dB PSNR-Y improvement without prohibitive rate overhead (Jia et al., 2024, Jia et al., 20 Mar 2025):

Method	PSNR-Y Gain (dB)
JPEG-AI VM base	reference
+ spatial Q-map (VVC)	up to +0.45

4. Subjective Visual Quality and Artifact Analysis

JPEG AI’s subjective visual quality was comprehensively evaluated using the JPEG AIC-3 JND protocol (Jenadeleh et al., 7 Apr 2025), which combines large-scale triplet preference testing and boosted comparison methods. Rate–distortion curves in JND units for JPEG AI exhibit exponential decay with narrow confidence intervals, confirming precise perceptual control in the near-lossless regime.

Objective metrics (e.g., CVVDP, HDR-VDP-3, IW-SSIM) exhibit high linear and rank correlation coefficients (PLCC, SRCC ≥ 0.96) with JND data for JPEG AI (Jenadeleh et al., 7 Apr 2025). Nevertheless, most objective models are over-optimistic, underestimating visibility of certain artifacts unique to neural networks. The Meng–Rosenthal–Rubin test was introduced to formally compare IQA metric performance for significance.

Artifact detection and characterization for JPEG AI are well-advanced. JPEG AI, despite its global transforms, can introduce localized texture, color, boundary, and text corruption that remains undetected by traditional full-reference metrics (Tsereh et al., 2024). A dedicated detection methodology employs:

• Differential MS-SSIM and CIEDE2000 for texture and color artifacts,
• Gradient–cosine distance for boundary artifacts,
• FSIM differences in detected text regions.

A dataset of 46,440 validated artifacts, with bounding boxes and confidence scores, provides a benchmark for further codec development and regression testing (Tsereh et al., 2024).

5. Security, Adversarial Robustness, and Forensic Analysis

JPEG AI’s end-to-end learned structure introduces both new robustness challenges and counter-forensic effects. On one hand, it is notably more robust to white-box adversarial attacks than prior neural codecs, with 30–50% smaller ΔVMAF degradation, and reduced bitrate inflation (ΔBPP ≈ +5–15%) under attack (Kovalev et al., 2024). The standard supports reversible preprocessing (e.g., flip, self-ensemble) for added robustness with negligible compute overhead.

On the other hand, the learned upsampling mechanisms in JPEG AI produce Fourier domain footprints closely resembling those of synthetic generation (GANs, diffusion models), thus confounding classic forensic detectors for deepfakes and manipulation (Cannas et al., 2024). Quantitative results show a marked increase in AUC (up to 0.92 at 0.12 bpp) and TPR (0.56) for pristine images being misclassified as fakes. Pixel-level splicing localizers also lose accuracy, with AUCₛ dropping to 0.57 for aggressively compressed images.

Emerging forensic toolsets exploit JPEG AI–specific cues: color-channel correlations induced by YUV conversion and subsampling, nonlinear recompression signatures in rate–distortion, and latent-space quantization characteristics uniquely found in compressed (not synthetic) images (Bergmann et al., 4 Apr 2025). These analytic features form the basis of new, explainable forensic decision pipelines.

6. Interoperability, Implementation, and Machine–Vision Support

JPEG AI codestreams are designed for multi-branch decoding, covering deployment scenarios from low-memory mobile hardware to high-complexity offline synthesis. Integer-arithmetic in the entropy pipeline ensures bit-exact decoding across CPU/NPU/GPU platforms (Esenlik et al., 13 Oct 2025). Profiles and levels (e.g., Main@Simple/Base/High) provide implementation flexibility, and the file format is ISOBMFF/HEIF compatible (Esenlik et al., 13 Oct 2025).

A central design goal is to enable machine-vision applications to consume the compressed-domain representations directly, bypassing full image reconstruction. Latent tensors are designed to deliver competitive or even superior accuracy in detection, segmentation, and related tasks at low bpp (Esenlik et al., 13 Oct 2025, Deng et al., 2021). The standardization roadmap targets incremental adoption of cv-inference APIs and support for direct semantic decoding.

7. Context, Developments, and Future Directions

JPEG AI emerges from a decade of DNN–centric compression research, uniting standalone neural codecs and JPEG-compatibility efforts (DeepN-JPEG, Neural JPEG, JPNeO) under a normative, interoperable standard (Liu et al., 2018, Mali et al., 2022, Han et al., 31 Jul 2025). While fully end-to-end variable-rate learned codecs (e.g., via 3D quality maps and fast bit-rate matching) represent the current state-of-the-art, ongoing research targets:

• Enhanced robustness (adversarial training, robust entropy models) (Kovalev et al., 2024)
• Tunable spatial/spectral bit allocation (Jia et al., 2024, Jia et al., 20 Mar 2025)
• Dedicated machine-friendly distortion metrics and quantization strategies (Ye, 13 Mar 2025)
• Further forensic/anti-forensic characterization and hybrid physical–digital provenance schemes (Cannas et al., 2024, Bergmann et al., 4 Apr 2025)
• Extensions to video coding and multi-modal pipelines via canonical codec tokens (Han et al., 2024)

The JPEG AI standard thus defines a new computational foundation for visual data interchange, supporting scalable encoding, direct machine consumption, robust perceptual quality, and a formal testing regime, while also introducing new challenges for security, trust, and forensic analysis in the deep learning era.