Comp-X: Interactive Learned Image Compression

• The paper introduces Comp-X as the first interactive learned image compression paradigm, replacing manual codec tuning with an LLM-driven, intent-based approach.
• It employs a three-stage iterative workflow—planning, execution, and refinement—using segmentation masks and quality maps to dynamically adjust bitrate and region-of-interest coding.
• Expert feedback via augmented in-context learning (ICL-EF) significantly improves request understanding, achieving over 80% success in complex mode selection and parameter tuning.

Searching arXiv for the exact Comp-X paper and closely related items to ground the article in current arXiv records. I’m checking arXiv records for the exact title and neighboring terminology to ensure the article aligns with the cited preprints. Comp-X denotes an interactive learned image compression paradigm in which a LLM agent mediates between natural-language user intent and a multi-functional learned image codec. In the formulation introduced in “Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent” (Gao et al., 21 Aug 2025), the system is presented as the first intelligently interactive learned image compression paradigm empowered by an LLM agent. Its defining move is to replace manual codec-mode selection, hand-tuned bitrate control, and task-specific operating knowledge with an intent-driven pipeline in which a user describes the desired outcome, the agent plans codec actions, optional vision tools generate region masks when needed, the codec executes compression, and the system iteratively refines parameters under bitrate or performance constraints.

1. Definition, scope, and motivation

Comp-X is defined as an autonomous system for intelligent interaction and multi-functional image compression. The paper frames it against two limitations of conventional practice. First, commonly used image codecs usually suffer from limited coding modes and rely on manual mode selection by engineers, making them unfriendly for nonprofessional users. Second, much prior learned image compression is task-specific: one model for distortion optimization, another for perceptual quality, another for machine vision, another for region-of-interest coding, and so on. Comp-X is proposed to address both issues simultaneously by unifying multiple compression functionalities in one codec and placing an LLM-based interactive layer on top of that codec (Gao et al., 21 Aug 2025).

The supported request space is explicitly broad. The use cases described include ordinary human-vision compression optimized for distortion, perceptual compression for better subjective visual quality, machine-vision-oriented compression for classification, segmentation, detection, and pose estimation, mixed human-and-machine scenarios, variable bitrate control, region-of-interest or spatially adaptive bit allocation, and partial transmission of objects or foreground/background content. In this setting, image compression is recast as an interactive planning-and-execution problem rather than a fixed parameter-selection problem.

A central misconception addressed by the design is that “interactive compression” would merely mean a conversational front end to a standard codec. In Comp-X, interaction is operationally stronger: the user expresses requirements in natural language; an LLM agent interprets those requirements into codec parameters and tool actions; the codec itself supports multiple objectives within one framework; and the system can iteratively evaluate and refine encoding decisions until bitrate or quality constraints are satisfied. This makes the interaction layer part of the compression procedure itself rather than a superficial interface.

2. System architecture and iterative workflow

The Comp-X pipeline has three iterative stages: planning, execution, and evaluation/refinement. In the planning stage, the user provides a natural-language request. Guided by a system prompt and in-context examples, the LLM agent converts that request into a structured plan whose fields include file path, compression mode, whether RoI coding is needed, what object or region is the RoI, what objects need to be transmitted, encoded size level, whether there is a specific bitrate limit, whether there is a specific performance limit, and which performance metric should be used (Gao et al., 21 Aug 2025).

Execution begins from that structured plan. If the request requires region-of-interest coding or partial object transmission, the system invokes pre-analysis tools such as Grounding-SAM and detectron2 to generate segmentation masks. Those masks are then converted into codec control inputs, specifically group masks and quality maps. Compression is run using the original image together with the quality map, task identifier, and group mask. The workflow is therefore not a single forward pass of a frozen codec; it is a tool-augmented execution graph conditioned on the semantics of the request.

The evaluation/refinement stage is activated when the request contains explicit constraints such as a target file size, a target PSNR, or a weighted PSNR requirement. The current encoded result is evaluated, and if the constraints are not met, the LLM agent receives the historical results and proposes updated control parameters, especially quality-factor adjustments. This loop repeats until the result satisfies the request or the iteration limit is reached; in the reported experiments, the maximum update count is set to 10. The paper summarizes the overall process as: User instruction → LLM planning → optional mask-generation tools → codec execution → metric evaluation → LLM refinement if needed → final compressed output (Gao et al., 21 Aug 2025).

This architecture matters because it merges traditionally separate concerns: semantic understanding of user intent, invocation of external perception tools, rate–distortion control, and multi-objective compression. A plausible implication is that Comp-X treats compression not only as signal coding but also as orchestration over a family of conditional operations.

3. Unified coding framework and mathematical formulation

The codec core is built on semantically structured image compression and is designed to unify previously separate functionalities in one framework. Its main control variables are a group mask $m$, a quality map $q$, and a task identifier. These jointly determine what is transmitted, where bits are allocated, and what objective the codec should prioritize (Gao et al., 21 Aug 2025).

The group mask $m$ partitions the latent representation into object-level segments. Because the bitstream is decoupled into non-overlapping object segments, the system can selectively transmit or reconstruct only certain content, supporting object-level transmission, foreground/background selection, semantic partial transmission, and machine-task-specific object selection. The quality map $q$ is a spatial map whose values range from 0 to 1, where 0 denotes the lowest bitrate and 1 the highest bitrate. It enables variable-rate compression, spatial bit allocation, and RoI coding. The task identifier selects among objectives such as distortion, perception, segmentation, classification, detection, and pose estimation.

For the MSE-optimized variable-rate codec, the paper gives the training loss as

$$\mathcal{L} = \sum_{i}^{N} \lambda_i \frac{(x_i - x'_i)^2}{N} + \mathcal{R}(\hat{y}) + \mathcal{R}(\hat{z}),$$

with the distortion weight mapped from the control signal by

$\lambda_i = T(m_i),$

where $T : [0,1] \rightarrow \mathbb{R}^+$ is a predefined monotonically increasing function. Higher local quality-map values therefore induce higher local distortion weights and thus higher local bitrate allocation. For task-adapter fine-tuning, the generic loss is

$$\mathcal{L} = \lambda d(x, \hat{x}) + \mathcal{R}(\hat{y}) + \mathcal{R}(\hat{z}),$$

where $d(x,\hat{x})$ is a task-specific perceptual loss in feature space.

The task-specific feature losses are also made explicit. For classification,

$$d(x, \hat{x}) = \frac{1}{4} \cdot \sum_{l=1}^{4} MSE(F_l(x), F_l(\hat{x})),$$

where $q$0 are feature maps from a pre-trained ResNet50. For object detection, instance segmentation, and pose estimation,

$q$1

where $q$2 are pyramid features from R50-FPN-based detectors. The perception mode is trained in two stages: a first stage combining MSE, LPIPS, and bitrate terms, and a second adversarial fine-tuning stage combining distortion, LPIPS, adversarial generator loss, and bitrate terms. The paper explicitly states $q$3 and $q$4, and that three independent discriminators span the bitrate range.

Architecturally, the framework uses Task-aware Group-Independent Swin Blocks, Spatial Feature Transform layers, task-specific prompts and adapter branches, and a grouped latent representation. The SFT layers generate scaling and shifting parameters from the quality map and apply them via element-wise affine transformation to internal features. Entropy coding adopts unevenly grouped channel-wise context modeling from ELIC, while the latent $q$5 is split into five channel chunks of 16, 16, 32, 64, and 192. This suggests that Comp-X is not merely a control wrapper around a monolithic codec; the codec itself is structurally organized to expose semantic and task-conditional degrees of freedom.

4. LLM agent, expert feedback, and IIC-Bench

The LLM agent has three stated responsibilities: request understanding, mode/tool selection, and iterative parameter refinement. The planning prompt defines legal compression modes, default rules, distinctions between RoI coding and selective transmission, file-size unit handling, performance metric interpretation, and output format. The defaults are important: the agent is instructed to choose distortion by default if mode is unspecified, to choose perception for unsupported downstream tasks, to default to transmitting task-relevant content for machine-vision tasks, and to use weighted_PSNR if RoI coding is requested for distortion mode (Gao et al., 21 Aug 2025).

A central methodological contribution is augmented in-context learning with expert feedback, denoted ICL-EF. Standard in-context learning would provide the LLM with request/label examples, but the paper argues that this alone is insufficient because pre-trained LLMs lack image-coding priors. In ICL-EF, context examples include not only correct answers but also multi-turn corrections from a coding expert. The sequence is: present user-request cases to the LLM, let the LLM reason and answer, have a coding expert inspect the reasoning, explicitly point out mistakes when the reasoning is wrong, let the LLM revise its decision, and then place the entire corrected dialogue into the prompt context. The paper is explicit that this is a prompt-based approach without gradient update; no model fine-tuning of the LLM is reported.

Expert feedback also has a benchmark role. IIC-Bench is introduced as the first benchmark dedicated to interactive and intelligent image compression evaluation. It contains 195 items, of which 93 are simple and 102 are hard. Requests were collected from experts and general users; some were manually annotated; then GPT-4o was used for automated labeling via in-context learning, followed by human checks and corrections. The benchmark label structure includes fields such as file_path, compression_mode, RoI_coding, RoI_object, Object_needed_to_be_transmitted, encoded_size_level, specific_performance_limit, specific_bitrate_limit, performance_metric, bitrate_min, bitrate_max, bitrate_unit, performance_min, and performance_max. The evaluation metric is the success rate of user requirement analysis.

The reported interaction results on IIC-Bench are as follows (Gao et al., 21 Aug 2025):

Configuration	Split	Success rate
GPT-4o, system prompt only	Simple	70.67%
GPT-4o, system prompt only	Hard	60.46%
GPT-4o, system prompt only	All	65.30%
GPT-4o, + ICL	Simple	74.67%
GPT-4o, + ICL	Hard	78.10%
GPT-4o, + ICL	All	77.26%
GPT-4o, + ICL-EF	Simple	83.87%
GPT-4o, + ICL-EF	Hard	81.70%
GPT-4o, + ICL-EF	All	82.74%
DeepSeek-v3, system prompt only	Simple	72.33%
DeepSeek-v3, system prompt only	Hard	62.67%
DeepSeek-v3, system prompt only	All	67.28%
DeepSeek-v3, + ICL	Simple	82.00%
DeepSeek-v3, + ICL	Hard	84.00%
DeepSeek-v3, + ICL	All	83.04%
DeepSeek-v3, + ICL-EF	Simple	85.33%
DeepSeek-v3, + ICL-EF	Hard	84.67%
DeepSeek-v3, + ICL-EF	All	84.99%

These results are used to argue that expert-feedback-augmented prompting substantially improves request understanding, especially on complex requests. A narrower but important point is that the “expert-driven” aspect of Comp-X is not RLHF-style optimization; it is prompt-level teaching through corrected demonstrations.

5. Compression behavior, implementation, and empirical results

The implementation combines the interactive layer with a learned image codec trained on ImageNet v6 trainset and the COCO 2017 training set using batches of 8 random $q$6 crops. Optimization uses Adam, a 10,000-step linear warmup, cosine learning rate decay, a peak learning rate of $q$7 for discriminators, and $q$8 for other experiments, on an NVIDIA RTX 3090 GPU. The base MSE-optimized variable-rate codec is trained for 1.5M iterations; machine-vision tasks are fine-tuned for 600K iterations with a learnable MLP branch channel dimension of 128 and 16 learnable task-specific tokens; the perception task uses an MLP branch channel dimension of 256, 16 learnable task-specific tokens, 1M iterations without GAN, and 500K iterations with adversarial loss. The LLM implementation uses GPT-4o and DeepSeek-v3 with temperature 0.7, and refinement permits a maximum of 10 updates (Gao et al., 21 Aug 2025).

The experiments evaluate both the interaction layer and the codec layer. In single application scenarios, the paper evaluates distortion, perception, classification, pose estimation, segmentation, and detection. For human vision, Comp-X is reported to achieve competitive performance with state-of-the-art codecs in PSNR and FID. For machine vision, it is reported to be strong in classification and pose estimation, and for instance segmentation and object detection it outperforms most baselines. The comparison set includes VVC/VTM, BPG, ELIC, Cheng2020, Minnen2018, the semantically structured codec, HiFiC, DEVIL, TransTIC, and Adapt-ICMH. The paper’s central empirical claim is that a single unified framework maintains competitive compression performance across diverse tasks.

The mixed human-and-machine evaluation is more specific. Relative to VTM 18.2, the paper reports for pose estimation that ELIC achieves BD-PSNR $q$9 and BD-mAP $m$0, TransTIC achieves BD-PSNR $m$1 and BD-mAP $m$2, and Comp-X achieves BD-PSNR $m$3 and BD-mAP $m$4. For segmentation, ELIC achieves BD-PSNR $m$5 and BD-mAP $m$6, TransTIC achieves BD-PSNR $m$7 and BD-mAP $m$8, and Comp-X achieves BD-PSNR $m$9 and BD-mAP $q$0. The stated interpretation is that Comp-X better balances human and machine metrics than ELIC and offers much higher human reconstruction quality than the machine-task-specialized TransTIC.

The refinement experiments examine explicit constraints. With a rate limit, “Ours (w/ rate limit)” significantly improves instance-level rate control over the baseline; with a performance limit such as PSNR, “Ours (w/ performance limit)” uses higher bitrate to meet the specified quality level. For hard constraints involving foreground high quality, weighted PSNR with RoI weight 0.8, and target file size, the system adaptively tunes region quality factors and outperforms a 10-iteration optimization baseline adapted from Song et al. using weighted distortion plus rate. For RoI reconstruction quality, weighted PSNR is measured on 13 Kodak images with foreground masks extracted by IS-Net. These experiments are used to support the claim that the agent can satisfy complex, multi-constraint requests rather than only selecting a static mode.

6. Nomenclature, significance, and open issues

In the arXiv literature represented here, the exact title “Comp-X” refers to the interactive learned image compression paradigm described above (Gao et al., 21 Aug 2025). The surrounding nomenclature is nonetheless ambiguous. “CompEx II: A Pathway in Search of BSM Physics using Compton Scattering” describes a staged Compton-scattering program and states that mapping this sequence onto a “Comp-X” pathway is reasonable, but it also notes that the paper does not use the spelling “Comp-X” explicitly (MohanMurthy et al., 2018). “Compact x-ray source based on burst-mode inverse Compton scattering at 100 kHz” uses “CXLS, Comp-X” parenthetically for a compact inverse-Compton x-ray source (Graves et al., 2014). By contrast, “CoMP Transmission in Downlink NOMA-Based Cellular-Connected UAV Networks” concerns coordinated multipoint transmission, or CoMP, which is terminologically separate despite superficial orthographic similarity (Sun et al., 2023). For encyclopedia purposes, the exact term “Comp-X” is therefore most precisely anchored to the image-compression paradigm of (Gao et al., 21 Aug 2025).

The significance claimed for Comp-X rests on five points: it is presented as the first intelligently interactive learned image compression paradigm using an LLM agent; it introduces a unified multi-functional codec integrating human-machine perception, variable coding, and spatial bit allocation; it proposes augmented in-context learning with expert feedback for request understanding, mode selection, and tool use; it introduces IIC-Bench as the first benchmark for interactive image compression; and it reports over 80% instruction parsing success while maintaining competitive compression performance. The paper explicitly frames this as a shift from parameter-driven coding to intent-driven coding and as a promising avenue for AGI in image compression (Gao et al., 21 Aug 2025).

At the same time, the paper states only limited limitations analysis. Future work is said to incorporate a wider variety of codecs, including traditional codecs, and to extend to more realistic scenarios. It does not deeply analyze latency, robustness of segmentation tools, or LLM cost, although those concerns are implicit in the architecture. Another misconception the paper implicitly rejects is that expert knowledge enters through model fine-tuning or reward modeling; in the reported method, expert knowledge primarily improves request understanding and decision quality through prompt-level demonstrations and benchmark annotation rather than gradient-based supervision.

Taken together, Comp-X represents a redefinition of learned image compression as a compound system with three inseparable layers: a multi-objective codec, an LLM-based planning and control agent, and an expert-informed evaluation interface. Its main technical claim is not merely that it compresses images well, but that it operationalizes compression as an intent-conditioned, tool-augmented, iterative decision process.