Utility-Constrained Compression: Theory & Methods

• Utility-constrained compression is a framework that defines compression by optimizing application-specific utility metrics instead of solely minimizing distortion.
• It leverages techniques such as Lagrangian relaxation, task-aware preprocessing, and adaptive quantization to balance bit-rate and utility in various domains.
• Empirical results in imaging, ML distillation, smart grids, and distributed training demonstrate its effectiveness in maintaining high performance while reducing data size.

Utility-constrained compression refers to a broad class of information-theoretic and algorithmic frameworks in which data or model compression is performed subject to explicit, often application-driven, utility criteria, rather than the traditional distortion-centric measures. Instead of minimizing expected reconstruction error for a fixed bit-rate or minimizing bitrate for a tolerable distortion (standard rate–distortion theory), the compression process is constrained or optimized with respect to end-task utility—such as hypothesis testing accuracy, per-pixel error bounds, decision quality post-decompression, or privacy-preserving efficacy—within a given rate budget, or vice versa. This paradigm underpins a wide range of advances from deep rate-utility model compression in neural networks to task-dependent quantization and adaptive vector quantization in data transmission, offering a principled mechanism for meaningful and tunable trade-offs between compression and downstream utility.

1. Theoretical Foundations and Formal Problem Statements

The formal mathematical setup of utility-constrained compression generalizes rate–distortion by optimizing a compression scheme to satisfy a constraint on an application-driven utility function $U$, typically subject to a bit-rate or storage constraint. The description varies by context:

• General utility-constraint framework: Minimize rate $R$ subject to $U(x,\hat x)\geq U_0$, where $U$ may be a per-pixel error measure, downstream decision performance, statistical divergence, or custom application metric. Equivalently, maximize $U$ given a constraint $R\leq R_0$ (Zhang et al., 2020, Sun et al., 2020).
• Large deviation approach: The set of compressible codewords is restricted by both information (entropy) and utility constraints; each codeword $x$ of length $n$ must satisfy

$$\frac{1}{n}\phi_n(x)\geq \eta,~~-\frac{1}{n-1}\log P^{(\text{so})}_n(x)\leq h+\epsilon,$$

where $\phi_n(x)$ is the utility and $R$0 is the source law (Suhov et al., 2016).

• Task-aware rate optimization: In hypothesis testing, the utility is the test error exponent; compression is directly tuned to maximize asymptotic test power under a rate constraint (Carpi et al., 2021).
• Primal–Lagrangian relaxation: Problems are often recast as unconstrained objectives via weighted sum of rate and utility via Lagrangian multipliers:

$R$1

and dual parameters can be shown to correspond to hard trade-off thresholds (Li et al., 2024, Sun et al., 2020, Bao et al., 23 Jul 2025).

This approach unites diverse domains by treating utility as the constraint or regularization target, subsuming distortion, statistical, and application-specific requirements.

2. Methods and Algorithmic Strategies

2.1 Rate–Utility Lagrangian Approaches

• In dataset distillation, synthetic datasets are parameterized as latent vectors and decoded, where the bits needed to represent latents and decoders define $R$2, and distillation-style downstream loss provides $R$3. The optimization objective is $R$4 (Bao et al., 23 Jul 2025).
• For point cloud compression, the optimal transport map is learned via a GAN, with a global rate term (entropy) directly penalized in the generator loss. The objective becomes $R$5 (Li et al., 2024).
• In smart grid data, both linear and non-linear transforms are learned to minimize the expected loss in a downstream decision utility (e.g., an $R$6-norm optimization) under fixed code length constraints (Sun et al., 2020).

2.2 Task-Aware or Utility-Aware Preprocessing

• Use-oriented transforms (linear or neural autoencoders) optimize for low end-to-end utility loss (e.g., energy scheduling) rather than reconstruction mean-squared error, and are fine-tuned to the specific decision task (Sun et al., 2020).

2.3 Explicit Hard Constraints (ℓ∞, Privacy, Hypothesis Testing)

• For near-lossless image compression, the $R$7 maximal pixel error is enforced exactly by quantization, leading to compressed outputs $R$8 with $R$9 (Zhang et al., 2020, Bai et al., 2021).
• In privacy–utility scalable offsite-tuning, constraints not only minimize emulator size but also ensure the performance gap between emulated and full models (privacy gap) surpasses a threshold, while the "plug-in" accuracy meets or nearly matches the non-compressed model (Yao et al., 2024).
• In binary hypothesis testing, the encoder clusters symbols to maximize the exponent of test power (Chernoff–Stein lemma), given a hard rate constraint (Carpi et al., 2021).

2.4 Dictionary Methods and Adaptive Quantization

• The constrained-dictionary LZ78(LRU), preceded by adaptive block vector quantization, achieves the finite-state distortion-limited compressibility of any sequence (Ziv, 2014). The quantizer assigns blocks to cluster-representatives within distortion, and LZ78(LRU) on these blocks attains optimal rate asymptotically.

2.5 Gradient Compression for Distributed Training

• Gradient compression is assessed directly by its impact on end-to-end time-to-accuracy (TTA), reflecting overall training utility, rather than throughput or bit-savings alone. Compression hyperparameters are tuned to maximize utility $U(x,\hat x)\geq U_0$0 (Han et al., 2024).

3. Key Domains and Empirical Findings

Imaging and Point Clouds

• ℓ∞-constrained approaches achieve strict per-pixel fidelity bounds and often also improve PSNR or perceptual metrics compared to unconstrained codecs, even at high compression ratios ($U(x,\hat x)\geq U_0$1-ED$U(x,\hat x)\geq U_0$2 and joint lossy+residual VAE coders outperform BPG, WebP, and JPEG-LS with exact max-error control) (Zhang et al., 2020, Bai et al., 2021).
• For point cloud datasets, constrained optimal-transport GANs (COT-PCC) achieve state-of-the-art results on Chamfer Distance and PSNR under fixed bit budgets (Li et al., 2024).

Machine Learning and Training Compression

• In dataset distillation, up to 170$U(x,\hat x)\geq U_0$3 compression over vanilla distillation at similar utility is achieved via joint rate–utility optimization, consistent across architectures and distillation loss functions (Bao et al., 23 Jul 2025).
• ScaleOT for LLM tuning offers near-lossless plug-in accuracy with a privacy gap exceeding uniform layer-drop by 1–2% across major models (Yao et al., 2024).

Smart Grids and Energy Optimization

• Use-oriented transforms (linear and non-linear) reduce end-to-end utility loss by more than $U(x,\hat x)\geq U_0$4 compared to Karhunen-Loève or wavelet transforms in energy scheduling, with negligible additional computational cost (Sun et al., 2020).

Communication-Constrained Decision Making

• In distributed hypothesis testing, greedy clustering of symbols by KL divergence preserves error-exponent far better than universal log-loss coding for most rates, rapidly approaching the optimal test exponent as rate increases (Carpi et al., 2021).

4. Trade-Offs, Limitations, and Open Problems

• The trade-off between rate and utility is typically convex, with diminishing returns on utility past a certain rate (e.g., $U(x,\hat x)\geq U_0$5 bits/sample in smart grid data); excessive compression degrades utility in both rate–distortion and decision-theoretic settings (Sun et al., 2020, Bao et al., 23 Jul 2025).
• Non-convexities and local minima present optimization challenges, especially in non-linear transform or high-dimensional settings (Sun et al., 2020).
• Many approaches rely on gradient-based or greedy heuristics; exact solutions are generally infeasible for combinatorial or high-dimensional cases (NP-hardness in task-aware encoders, majorization bounds in adaptive quantization) (Carpi et al., 2021, Ziv, 2014).
• Extensions to streaming/online/heterogeneous utility constraints, richer tasks (e.g., multi-task, time-series), and black-box utility remain active research areas (Bao et al., 23 Jul 2025, Sun et al., 2020, Zhang et al., 2020).

5. Broader Implications and Generalizations

• The utility-constrained framework subsumes and generalizes classical rate–distortion, enabling compression that is principled for any utility (distortion, test accuracy, task performance, privacy gap) (Bao et al., 23 Jul 2025, Sun et al., 2020, Suhov et al., 2016, Yao et al., 2024).
• Lagrangian forms and duality tie hard constraint and unconstrained settings; empirical evidence supports the equivalence between dual-weighted and constrained problems under mild regularity (Li et al., 2024, Sun et al., 2020).
• Explicit utility-constrained evaluation addresses common misconceptions in the field: throughput or bit-rate reduction alone is insufficient, and reported gains must reflect end-to-end utility (e.g., time-to-accuracy, downstream test performance) (Han et al., 2024).

6. Summary Table: Representative Utility-Constrained Compression Instances

Domain	Utility Constraint	Compression Mechanism
Point clouds	Chamfer/PSNR + global Wasserstein + rate	COT-PCC: OT-GAN + entropy term
Imaging	$U(x,\hat x)\geq U_0$6-error bound	Uniform quantization + CNN restoration
ML distillation	Task loss (accuracy, gradient match, etc.)	Latent quantization + entropy model
Smart grid	Downstream $U(x,\hat x)\geq U_0$7-norm-based decision	Linear/non-linear utility-oriented
Hypothesis test	Error exponent at fixed test error	Greedy KL clustering, block encoding
Gradient agg.	TTA at target accuracy	TopKC, THC quantization, PowerSGD

All listed methods achieve tunable, application-relevant trade-offs, enabling system design and optimization that aligns the compression procedure with concrete statistical or task requirements, extending the domain and efficacy of information theoretic principles (Li et al., 2024, Sun et al., 2020, Carpi et al., 2021, Bao et al., 23 Jul 2025, Ziv, 2014, Suhov et al., 2016, Zhang et al., 2020, Bai et al., 2021, Han et al., 2024, Yao et al., 2024).