Machine-centric Image Quality Assessment

• MIQA is a paradigm that assesses image quality based on its performance in machine vision tasks rather than human visual aesthetics.
• It leverages semantic rate–distortion theory and task-specific losses to optimize both compression efficiency and inference accuracy.
• MIQA frameworks drive efficiency in autonomous and edge systems by employing tailored feature selection and resource allocation strategies.

Machine-centric Image Quality Assessment (MIQA), often situated under the broader research of task-oriented or semantic image compression, fundamentally departs from classical human-centric paradigms in evaluating image quality. Instead of optimizing for human visual perception, MIQA quantifies the suitability of an image (or its compressed representation) for downstream machine vision tasks such as classification, detection, segmentation, or inference within autonomous or edge intelligence systems. This shift reflects the reality that, in many contemporary pipelines, images may never be seen by a human; instead, their utility hinges on their informativeness for a particular computational or inference task. Modern frameworks for MIQA are intrinsically linked to the development of task-oriented compression, semantic communications, and task-aware perceptual metrics.

1. Principles of Machine-centric Image Quality Assessment

MIQA is formulated around evaluating how image degradation or compression affects machine task performance metrics, e.g., classification accuracy, mean average precision (mAP), or success probability of an inference outcome. Unlike traditional metrics such as PSNR or SSIM—which are strongly correlated with human visual experience—MIQA primarily considers the impact of perturbations on feature activations or final predictions by machine learning models. Paradigmatic objective functions in MIQA include the rate–distortion–task loss

$$L = D(I',I) + \alpha L_T(I';\text{Task}) + \beta R(\hat{I})$$

where $D$ is a classical distortion (e.g., MSE), $L_T$ is the task loss (e.g., classification cross-entropy), and $R$ the compression rate (Kubiak et al., 2021).

This framework generalizes to

• Feature-space or task-induced distortion: measuring $\| \phi(I) - \phi(\hat I) \|$ where $\phi(\cdot)$ is a feature extractor relevant to the machine task.
• Downstream task performance: using the task metric itself (e.g., prediction accuracy, value function for RL) as the ultimate MIQA score.

2. Mathematical Foundations and Metrics

The theoretical underpinning of MIQA stems from information-theoretic and empirical task-performance curves:

• Semantic rate-distortion theory: The achievable rate-distortion region is characterized with respect to task variables $S$, observations $X$, and, optionally, side information $Y$:

$$R(D_X, D_S) = \min_{p(u|x,y), \hat x(u,y), \hat s(u,y)} I(X; U|Y)$$

subject to bounds on distortion of $D$0 (image) and $D$1 (semantics) (Guo et al., 2022).

• Accuracy-vs.-compression function: For neural classifiers, accuracy under varying compression ratios $D$2 is typically non-linear, empirically fit by a weighted sum of exponentials:

$D$3

providing the core MIQA predictive model in adaptive semantic compression frameworks (Liu et al., 2022).

• Task-specific semantic integrity: Metrics such as the Semantic Transmission Integrity Index (STII)

$D$4

where $D$5 is the channel/task relevance, and $D$6 the error probability, directly link channel and compression artifacts to machine-task performance (Sun et al., 29 Apr 2025).

3. Model Architectures and MIQA Algorithms

Task-oriented compression schemes embed MIQA within their architecture and optimization. Representative model types include:

• End-to-end learned semantic coding chains: Feature extraction, compression, transmission, and task inference are trained or optimized jointly under both bitrate and task constraints (Liu et al., 2022, Liu et al., 2022).
• Gradient-based semantic feature selection: Adaptable Semantic Compression (ASC) evaluates the importance of each latent feature or map by the gradient of the task loss w.r.t. that feature:

$D$7

masking those least relevant to the task (Liu et al., 2022).

• Task-coupled entropy models: Algorithms such as selective entropy coding, hierarchical entropy models, or expert mixtures compress visual tokens or features prioritized by their impact on the downstream task (Yuan et al., 17 Mar 2025, Shao et al., 2024).
• Rule-based or hybrid feature coding: In graph-based pipelines, MIQA is operationalized by compression of only the scene graph relations used in the target inference, yielding extreme reductions in data volume with high semantic fidelity (Ribouh et al., 9 Mar 2026).
• Rate allocation and resource optimization: MIQA-aware resource allocation frameworks (e.g., CRRA, IRCSC) solve for compression ratios, bandwidth, and power to maximize the probability of successful task inference under delay and energy constraints (Liu et al., 2022, Sun et al., 29 Apr 2025).

4. Evaluation Protocols and Empirical Results

Empirical MIQA protocols involve benchmarking downstream task accuracy or performance under controlled compression and transmission settings:

• Benchmark datasets: STL-10, ImageNet, CelebAMask-HQ, DAIR-V2X, and multimodal QA datasets are used to benchmark task accuracy under aggressive compression (Liu et al., 2022, Zhang et al., 2023, Shao et al., 2024, Yuan et al., 17 Mar 2025, Ribouh et al., 9 Mar 2026).
• Baselines: Human-oriented codecs (JPEG, WebP), task-agnostic autoencoders, and fixed-rate semantic communication models.
• Performance highlights:
  • Adaptive semantic compression in image classification or detection yields up to 80% reduction in data volume at <1% accuracy loss (Liu et al., 2022).
  • In device-edge multimodal pipelines, MIQA-driven schemes halve transmission and system latency at fixed accuracy (Yuan et al., 17 Mar 2025).
  • On V2X settings, MIQA-based feature selection and compression deliver up to 10–15 absolute mAP points improvement over uniform compression baselines, sometimes at as little as 1/10th the bandwidth (Shao et al., 2024, Ribouh et al., 9 Mar 2026).
  • Semantic coding for graph-based representations achieves >0.9 semantic fidelity and risk prediction accuracy with 99.9% data size reduction (Ribouh et al., 9 Mar 2026).

5. Information Bottleneck and Theoretical Limits

The Information Bottleneck (IB) principle provides the formal mathematical foundation for MIQA, especially in systems where direct optimization for task relevance is possible: $D$8 This formulation ensures that compression preserves only the features essential for the machine task $D$9, with $L_T$0 controlling the tradeoff (Furutanpey et al., 2024, Shi et al., 2023). Variational implementations of IB are realized in both deep end-to-end networks (DVIB) and shallow bottleneck injection (SVBI), with the IB loss upper-bounding the retained redundant information (Furutanpey et al., 2024). Theoretical results show that, in the presence of side information (auxiliary variables or context), the semantic rate-distortion function can be tightly characterized and indicates when focusing solely on semantic variables yields large rate savings (Guo et al., 2022, Gunduz et al., 2022).

6. Task Diversity, Design Considerations, and Security

MIQA's scope spans a variety of visual and multimodal tasks:

• Classification: Standard for most MIQA studies; measured as top-1 or top-5 accuracy (Kubiak et al., 2021).
• Segmentation: Usually in terms of mIoU and pixel-wise accuracy; bit allocation is task-object or region-driven.
• Detection and Risk Assessment: V2X driving and autonomous robotics require MIQA linked to AP, risk, or control success metrics (Shao et al., 2024, Ribouh et al., 9 Mar 2026).
• Multi-task and Multi-modal Settings: Layered coding and clustering/disentanglement methods allow inclusion of multiple simultaneous MIQA objectives (e.g., joint classification and segmentation) (Zhang et al., 2023, Yuan et al., 17 Mar 2025).

Security and robustness also enter MIQA via adversarial considerations. Bottleneck-based (IB) compressors can be more robust to attacks that perturb salient pixels, although reliance on generative models may introduce new vulnerabilities that must be mitigated with robust optimization or adversarial training (Furutanpey et al., 2024).

7. Open Challenges and Research Directions

Several directions remain at the frontier of MIQA research:

• Generalization to new tasks: Developing universal MIQA methods capable of supporting a wide spectrum of machine vision applications without per-task customization (Wood, 2022).
• Joint human–machine quality tradeoff: Designing metrics and codecs that offer tunable compromise between perceptual (human) and task (machine) quality for cases requiring both (Reddy et al., 2021, Gunduz et al., 2022).
• Edge deployment and efficiency: Lightweight, MIQA-aware compressors for resource-constrained and ultra-low-latency edge deployments (Yuan et al., 17 Mar 2025, Shi et al., 2023).
• Theoretical bounds: Refining single-letter rate–distortion and information bottleneck bounds for complex models and real-world modalities (Guo et al., 2022, Furutanpey et al., 2024).
• Standardization and benchmarks: Establishment of widely accepted MIQA datasets, metrics, and open benchmarks to enable cross-task comparison (Wood, 2022).

The integration of MIQA into machine-driven image and video coding architectures is a hallmark of modern semantic communication and task-oriented compression research, with rigorous theoretical and empirical grounding indicating substantial efficiency gains and informed design principles for future intelligent systems.