Variable-Rate Image Compression

• Variable-rate image compression is a technique that enables a single codec to flexibly operate at multiple points along the rate-distortion curve by dynamically controlling bitrate and quality.
• It employs conditional models and modulators—such as CVAEs, modulation networks, and transformer-based architectures—to achieve continuous adaptation and efficient deployment.
• Empirical studies confirm that these methods match fixed-rate performance with negligible RD loss, supporting advanced standards like JPEG AI while reducing storage and training costs.

Variable-rate image compression encompasses algorithmic and architectural approaches in which a single compression model can be flexibly operated at multiple points along the rate–distortion (RD) curve, enabling continuous or adaptive choice of bitrate or output fidelity at inference. This is in contrast to fixed-rate deep learning-based image compression, where separate models are conventionally required for each target trade-off between distortion and compressed size. Recent research has demonstrated numerous methods that resolve the storage, deployment, and efficiency problems of fixed-rate learned codecs while maintaining (or even improving) coding efficiency. Variable-rate image compression is now fundamental to state-of-the-art learned codecs and underpins emerging standards environments such as JPEG AI.

1. Motivation and Problem Setting

Variable-rate image compression is motivated by two core requirements in practical learned codecs:

1. Efficiency in model deployment: Classical VAE-based learned image compression methods optimize a model for a single Lagrange parameter $\lambda$ (controlling $D+\lambda R$), necessitating separately trained encoder-decoder pairs per operating point. This multiplies storage and training cost and is antithetical to software and hardware deployment in bandwidth-variable settings (Jiang et al., 2021).
2. Continuous and precise bitrate control: Real-world applications—such as streaming, mobile photography, and region-of-interest (ROI) coding—demand fine-grained control over compression rates and spatial quality. Solutions must yield dense (potentially continuous) coverage of the RD curve, support “on-the-fly” adaptation, and avoid degradation relative to multiple fixed-rate networks (Yang et al., 2019, Sun et al., 2021).

The main technical challenges arise from the tension between model efficiency (single model) and the need to “navigate” a high-dimensional and nonconvex RD envelope without performance gaps.

2. Methodological Frameworks

2.1 Conditional and Modulated Autoencoders

Architectures such as the conditional variational autoencoder (CVAE) and modulated autoencoder (MAE) adapt the analysis and synthesis transforms (and associated entropy models) based on explicit or learned control parameters:

• Conditional Networks: Expose explicit RD control variables—typically $\lambda$, quantization bin size $\Delta$, or both—as conditioning vectors to every convolution (weights, biases, or scale factors). Each layer receives side information encoding the intended RD tradeoff, enabling channel-wise or pixel-wise adaptation of layer responses (Choi et al., 2019).
• Modulation Networks: Insert lightweight channel-wise modulation (or demodulation) networks (usually small MLPs) at multiple depth points, which take in a normalized RD parameter and output per-channel scaling. This enables soft, continuous morphing of the shared model to new rates without architectural changes (Yang et al., 2019).

Both approaches are trained over sets of discrete or sampled ($\lambda$, $\Delta$) pairs and optimized via the collective multi-rate objective. At inference, fine control is achieved by interpolating within the control space.

2.2 Feature Scaling and Quantization Control

Variable-rate operation can be realized by dynamically scaling the latent representations, thus mimicking the effect of changing $\lambda$:

• Bit-rate Modulator (BM): Learns per-channel scaling factors via small FC layers controlled by $\lambda$, inserted before quantization and reversed prior to decoding. This enables direct manipulation of the distribution of latent codes and, indirectly, rate control (Yin et al., 2021).
• Quantization Regulator: Applies a vector of per-channel quantization step-sizes or scaling factors (‘a’ in QVRF), each coupled to a particular Lagrange multiplier, and enables continuous or interpolated control between trained values (Tong et al., 2023).

These approaches may generalize to control at ROI level or for spatially-varying target rates, provided scaling factors are mapped from spatial maps as well as scalar RD parameters.

2.3 Selective Compression and Content-Guided Masking

Another operational paradigm involves selective encoding of latent features according to content importance and desired quality:

• Selective Compression via 3D Importance Maps: Learns a rate-independent importance map over the latent tensor, then applies adjustable, rate-dependent nonlinearities (“importance adjustment curves”) to select which latent dimensions to encode. This enables adaptive bit-allocation focused on salient features, with mask parameters interpolated for continuous rate control (Lee et al., 2022).
• Spatial Importance Guided Scaling (as in SigVIC): Multiplies spatially-varying scaling factors, learned from per-pixel importance masks and the target RD parameter, onto each encoded feature, achieving both local and global control (Liang et al., 2023).

2.4 Invertible and Multi-scale Transforms

Invertible Neural Networks (INNs) and multi-scale architectures leverage bijective mappings and channel splitting to achieve inherently information-preserving encodings:

• Invertible Activation Transformation (IAT): Injects per-feature, per-pixel scaling and bias, driven by a continuous quality map, into invertible blocks, yielding fine control without breakdown under repeated encode/decode cascades (Cai et al., 2022).
• Multi-Scale Invertible Networks: Use full multi-scale decomposition and invertible units, inserting “gain units” to modulate rate through explicit quality indices. This permits seamless rate sweeping with negligible distortion penalty and, in some cases, outperforms classical codecs even at high bit rates (Tu et al., 27 Mar 2025).

2.5 Transformer-based and ROI-aware Models

Transformer-based architectures equipped with prompt tuning or context tokens allow joint variable-rate and ROI-aware compression:

• Prompt-Based Coding: Small prompt-generation networks condition Swin Transformer layers based on the input image, explicit bitrate parameter, and (optionally) an ROI mask. Prompts injected at multiple transformer blocks enable region-adaptive fidelity and channel-wise or spatial-wise modulation across a continuous spectrum (Qin et al., 2023, Kao et al., 2023).
• Spatial Importance and ROI Scaling: Content-adaptive scaling networks take both desired RD parameter and pixel-level ROI annotation to guide the transformer’s attention, ensuring bit allocation matches semantic or application-specific relevance (Mudgal et al., 28 Sep 2025).

3. Fundamental Rate–Distortion Control Mechanisms

Most variable-rate image compression frameworks are unified by minor variations on the following generalized optimization and operational strategies:

Mechanism	Control Variable(s)	Approach
Latent channel scaling	$\lambda$, vector “a”	BM (Yin et al., 2021), QVRF (Tong et al., 2023)
Modulation or conditional input	$\lambda$, $D+\lambda R$0	MAE (Yang et al., 2019), CondAE (Choi et al., 2019)
Spatial or content masks	Quality or ROI map $D+\lambda R$1	SCR (Lee et al., 2022), SigVIC (Liang et al., 2023)
Per-feature/pixel transformation	Quality map $D+\lambda R$2, $D+\lambda R$3	IAT (Cai et al., 2022), MSINN (Tu et al., 27 Mar 2025)
Interpolation attention	Anchors $D+\lambda R$4, ratio $D+\lambda R$5	IVR (Sun et al., 2021)

The empirical consensus from comparative experiments is that–when coupled with joint training over a sufficiently large set of target RD trade-offs—such mechanisms can yield a continuous operating range with negligible (<0.1 dB) RD loss compared to individualized, per-$D+\lambda R$6 training (Yang et al., 2019, Tu et al., 27 Mar 2025, Lee et al., 2022). Hybrid methods that combine spatial adaptation (importance maps, prompts) and channel scaling further improve ROI fidelity and coding efficiency.

4. Addressing Train–Test Quantization Mismatch

A central issue in practical learned compression is the train–test gap incurred by soft (noisy) quantization surrogates at training and hard rounding at inference:

• Online Meta-Learning (OML): Controls per-image or per-patch meta-parameters (e.g., mapping from $D+\lambda R$7 to decoder gains), which are rapidly adapted online after hard quantization. Small SGD steps at test time allow the model to learn to correct quantization artifacts in the conditional decoder, closing the log-likelihood and rate–distortion gap (Jiang et al., 2021).
• Reparameterization Method: Strategies such as QVRF’s scale–round–scale trick and dead-zone quantization within isometric latent spaces align the differentiable training objective and the true quantized test behavior, enabling a single network to robustly span variable rate operation (Tong et al., 2023, Zhou et al., 2020).

Experiments consistently show that quantization-aware control, especially when jointly optimized or adaptively post-trained, is critical for strong RD performance at low and high bitrates alike.

5. Practical Deployment, Architecture, and Standardization

5.1 Integrability and Overhead

Techniques such as per-channel scaling vectors, tiny modulation networks, prompt generators, and 3D importance masks consistently incur minimal (<1% parameter) and negligible compute overhead—making variable-rate adaptation highly practical for real-world deployment (Yang et al., 2019, Yin et al., 2021, Lee et al., 2022). Many such modules can be attached to existing fixed-rate codecs as plug-ins, sometimes with post-training only (Kamisli et al., 2024).

5.2 Region-of-Interest and Content-Adaptive Coding

Recent frameworks explicitly incorporate region-of-interest (ROI) and content-aware mechanisms, making it possible to allocate bits unevenly across semantic regions or color components:

• ROI Masks and Quality Maps: Inputs such as pixel-wise ROI binary maps or three-dimensional quality maps provide local adaptability (e.g., higher bitrate for faces or text) (Qin et al., 2023, Jia et al., 20 Mar 2025, Mudgal et al., 28 Sep 2025).
• Selective Latent Representation: Content-adaptive selective masking of latent dimensions, or spatially-gated feature scaling, aligns bit allocation with semantic saliency (Lee et al., 2022, Liang et al., 2023).

5.3 Standardization

JPEG AI, the emerging standard for learned still-image codecs, natively includes variable-rate mechanisms based on a three-dimensional quality map, fast bit-rate-matching, and a multi-stage variable-rate training regimen. These adaptations yield continuous RD coverage, flexible cross-channel (Y/UV) and ROI-based bit allocation, and are tied together by robust, hardware-friendly implementations (Jia et al., 20 Mar 2025).

6. Experimental Evidence and Benchmarking

Across standard datasets (Kodak, CLIC, JPEG AI) and metrics (PSNR, MS-SSIM, BD-Rate):

• State-of-the-art variable-rate approaches consistently match or outperform fixed-rate learned codecs, with rate–distortion performance within <0.1 dB or <1% BD-rate loss over the operating band (Yang et al., 2019, Lee et al., 2022, Kamisli et al., 2024).
• Modern transformer-based and invertible approaches—notably those incorporating cross-window or multi-scale attention—expand the attainable rate range and offer significant headroom over classical codecs (e.g., VVC, BPG) (Tu et al., 27 Mar 2025, Mudgal et al., 28 Sep 2025).
• Hybrid frameworks combining variable-rate, spatial, and semantic adaptivity realize both high coding gain and high flexibility, with competitive inference and storage cost suitable for large-scale deployment (Liang et al., 2023, Qin et al., 2023, Cai et al., 2022).

Empirical evidence affirms that single-model, flexible-rate deployment is feasible without performance compromise and with substantial savings in memory, compute, and sample complexity.

7. Open Challenges and Future Directions

Key areas for further research and potential performance enhancement include:

• Joint online adaptation of both conditioning parameters and core decoder/encoder weights for domain shift and out-of-distribution robustness (Jiang et al., 2021).
• More expressive conditional priors (e.g., normalizing flows) in entropy models for further reduction of quantization mismatch (Jiang et al., 2021, Cai et al., 2022).
• Integration of perceptual or adversarial losses to improve subjective quality at low rates, especially in invertible and transformer-based architectures (Cai et al., 2022).
• Extension to video, including temporal consistency in gain units, meta-update of inter-frame latents, and group-of-pictures adaptation.
• Efficient hardware and mobile deployment, further optimizing integer precision operations, memory footprint, and parallelism inherent in variable-rate modules (Jia et al., 20 Mar 2025).

The maturation of variable-rate image compression methodologies, combined with the move to standardization and integration of semantic and spatial awareness, establishes this area as a central pillar of modern neural image coding research and application.