JPEG AI Image Compression Visual Artifacts: Detection Methods and Dataset

Abstract: Learning-based image compression methods have improved in recent years and started to outperform traditional codecs. However, neural-network approaches can unexpectedly introduce visual artifacts in some images. We therefore propose methods to separately detect three types of artifacts (texture and boundary degradation, color change, and text corruption), to localize the affected regions, and to quantify the artifact strength. We consider only those regions that exhibit distortion due solely to the neural compression but that a traditional codec recovers successfully at a comparable bitrate. We employed our methods to collect artifacts for the JPEG AI verification model with respect to HM-18.0, the H.265 reference software. We processed about 350,000 unique images from the Open Images dataset using different compression-quality parameters; the result is a dataset of 46,440 artifacts validated through crowd-sourced subjective assessment. Our proposed dataset and methods are valuable for testing neural-network-based image codecs, identifying bugs in these codecs, and enhancing their performance. We make source code of the methods and the dataset publicly available.

• The paper develops sophisticated detection methods, including texture, boundary, and color artifact detection, to address the nuances of neural compression versus traditional codecs.
• It employs metrics like MS-SSIM, CIEDE2000, and edge detection alongside a curated dataset of 46,440 annotated examples to quantify visual distortions.
• Validation through subjective assessments and higher AUC values demonstrates the potential to refine JPEG AI and other neural-based compression standards.

An Evaluation of Neural Network-Based JPEG AI Image Compression Artifacts

This paper presents a meticulous investigation into the detection and characterization of visual artifacts arising from neural-network-based image compression, specifically within the context of JPEG AI. The research endeavor focuses on distinguishing the artifact characteristics inherent to learning-based compression techniques, which are qualitatively and quantitatively different from those produced by traditional codecs.

Objective and Methodology

The authors propose several sophisticated methods to isolate various types of compression-induced visual artifacts. These include texture and boundary degradation, color changes, and text corruption. To facilitate this, they developed metrics capable of pinpointing distortions within images that remain comparatively hidden in classical compression outputs yet become pronounced when using neural codecs.

Noteworthily, the research emphasizes three specific artifact detection methods:

1. Texture-Artifact Detection: Utilizing pixel-wise MS-SSIM and a specialized spatial information masking technique, the method locates and quantifies texture-related aberrations.
2. Boundary-Texture-Distortion Detection: A technique leveraging the Canny edge detector and Sobel operator results to flag discrepancies in edge and boundary reconstruction that favor classical codecs over neural ones.
3. Color Distortion: Distinct methods were designed to identify both large and small scale color distortions by utilizing the CIEDE2000 metric and detailed segmentation techniques to discern variances between traditional and neural codec compressions.

Additionally, the study introduces methods for text-artifact detection, employing OpenMMLab text detectors to assess and differentiate neural codec text restorations from traditional outcomes.

Dataset and Contributions

Central to this research is the compilation of a comprehensive dataset amassed from approximately 350,000 images processed to capture neural compression artifact signatures. The dataset, comprising 46,440 examples, offers detailed annotations regarding artifact types and locations and has been rigorously validated through crowdsourced subjective assessment to ensure reliability.

The following are key contributions of the paper:

• Development of novel detection methods showcasing enhanced performance in identifying neural-network-compression artifacts.
• A meticulously curated dataset with extensive examples to aid in the benchmarking and improvement of neural-based image codecs.
• Validation of proposed techniques via subjective assessments, providing robust, empirically grounded conclusions.

Results and Evaluation

Comparative analyses employing traditional image quality metrics such as PSNR, SSIM, and FSIM demonstrate the superior sensory acuity of the proposed methods in detecting neural compression artifacts, achieving higher AUC values across respective artifact types. This indicates that conventional quality assessments exhibit limitations in adequately addressing the nuanced distortion patterns induced by neural codecs.

Implications and Future Directions

The paper underscores the necessity of bespoke metrics tailored to the characteristics of neural-network-based compression techniques. This work holds significant implications for the refinement of JPEG AI and other neural codecs, aiding in the development of enhanced compression performance that minimizes perceptual distortions.

Future research may venture into extending these methodologies to encompass video compression artifacts and adaptations for real-time artifact detection mechanisms. Additionally, further exploration into synthesizing human perceptual assessments with algorithmic detections could yield improved hybrid appraisal systems, ultimately advancing the field of neural image compression standards.

In sum, this paper provides an expansive evaluation framework for the analysis of visual artifacts resulting from neural image compression, contributing a valuable toolkit toward the cultivation of robust, perceptually-aware compression methodologies.