---
title: 'CleanVision: Automated Image Quality Audit'
url: https://www.emergentmind.com/topics/cleanvision
type: topic
---

# CleanVision: Automated Image Quality Audit

Searching arXiv for the specified paper and closely related context so the article can be grounded in current literature.
CleanVision is an open-source Python package for automatically detecting problematic images in large image datasets. Within a data-centric workflow for image quality assessment, it functions as an automated audit layer that scores images for multiple anomaly and quality dimensions, applies thresholding or binary logic to flag suspect instances, and summarizes the results for inspection. In the study “A Data-Centric Perspective on the Influence of Image Data Quality in Machine Learning Models,” CleanVision is treated as a central component of a broader cleaning pipeline, both as originally implemented and as a basis for several refinements, including adaptive threshold selection and improved duplicate handling [2509.24420].

## 1. Scope and issue taxonomy

CleanVision is described as targeting nine main categories of problematic or anomalous images. These categories define its operational scope in the referenced workflow and distinguish pixel-level defects from structural irregularities and duplication phenomena [2509.24420].

| Issue type | Description in the workflow |
|---|---|
| Light | Overexposure |
| Dark | Underexposure |
| Blurry | Blur-related degradation |
| Low-information | Insufficient detail or content |
| Odd size | Size outlier |
| Odd aspect ratio | Aspect-ratio outlier |
| Grayscale | Images in grayscale form |
| Exact duplicates | Identical images |
| Near duplicates | Highly similar images |

The issue taxonomy matters because the paper explicitly argues that image quality is not a unitary construct. Not all quality issues exert the same level of impact on downstream training, and the empirical results indicate that convolutional neural networks are particularly vulnerable to degradations that obscure critical visual features, such as blurring and severe downscaling, while showing resilience to certain other distortions [2509.24420]. This positions CleanVision less as a generic “good versus bad data” oracle than as a structured detector of concrete image-level failure modes.

## 2. Audit workflow and decision logic

The CleanVision workflow is defined as a sequence of five steps: load images; specify issue types, with all enabled by default; compute a per-image score for each enabled issue type; apply thresholding or binary logic to flag images; and output summarized and visualized results for user inspection [2509.24420].

A central design feature is the per-image, per-issue score in the interval $[0,1]$, where lower scores typically indicate a higher probability that an image is problematic. This score-based formulation supports heterogeneous detectors under a common interface. For threshold-based issue types, an image is flagged when its score crosses a set threshold. For other issue types, notably grayscale and duplicate detection, CleanVision uses set membership or binary logic rather than scalar thresholding [2509.24420].

This architecture supports automation at dataset scale. The paper emphasizes that the resulting process moves beyond subjective, manual inspection by making dataset auditing score-driven and reproducible. A plausible implication is that CleanVision’s modular scoring interface is what enables the paper’s later extensions, especially adaptive thresholding and hybrid deduplication.

## 3. Core scoring mechanisms

The underlying mechanisms described for CleanVision are heuristic and issue-specific. For brightness-related detection, the system converts RGB to grayscale using a human-vision–weighted transformation similar to Photoshop:

$$
\text{Grayscale} = \sqrt{0.241 R^2 + 0.691 G^2 + 0.068 B^2}
$$

After pixel normalization, it computes several percentiles and the mean. The dark score is the 99th percentile value, so a low value indicates a dark image. The light score is $1-$ the 5th percentile value, so a low value indicates a bright image [2509.24420].

Blurry-image detection also begins with grayscale conversion, but it combines two signals: Laplacian variance, which captures edge sharpness, and grayscale histogram standard deviation, which captures texture variation. Lower combined scores indicate blurrier images. Odd size detection uses the interquartile range method, with thresholds

$$
\text{min\_threshold} = Q1 - \alpha \cdot IQR
$$

$$
\text{max\_threshold} = Q3 + \alpha \cdot IQR
$$

with default $\alpha = 3.0$, and the score is the normalized distance to these thresholds. Aspect ratio is represented as $\min(\text{width}/\text{height},\ \text{height}/\text{width})$ [2509.24420].

Grayscale detection flags an image if PIL reports mode “L” or if all three RGB channels are identical. Exact duplicates are identified by MD5 hashing, while near duplicates are identified using perceptual hash, or pHash, with images flagged when they share a hash [2509.24420]. In aggregate, these mechanisms show that CleanVision’s baseline design is dominated by direct pixel-statistical heuristics rather than learned representations.

## 4. Enhancements introduced in the 2025 study

The 2025 study does not merely use CleanVision as-is; it analyzes its mechanisms and introduces several modifications intended to improve detection quality without requiring manual threshold tuning. One refinement concerns light detection. The authors found that the original 5th-percentile formulation was too harsh, and they added the 25th, 30th, 40th, 50th, 60th, and 75th percentiles, reporting that the 60th or 75th percentile was often more discriminative [2509.24420].

A second refinement addresses grayscale detection. The logic checking for equal RGB channels was moved to after mode determination, which catches more “hidden” grayscales. A third refinement targets near-duplicate detection. The paper characterizes CleanVision’s default near-duplicate detector as highly strict because it catches only images with identical pHashes. To relax that strictness, the authors adopt clustering based on Hamming distances between pHashes, using HDBSCAN and hierarchical clustering, with the best results obtained from hierarchical clustering with single linkage [2509.24420].

The most consequential enhancement is adaptive or automatic threshold selection. Instead of using static, hard-coded thresholds, the paper systematically compares histogram-based thresholding methods for each score, including Otsu’s, Kittler-Illingworth MET, Li’s Minimum Cross-Entropy, Max Entropy, Generalized Histogram Thresholding, Modified Valley Emphasis, and Gamma Mixture Model [2509.24420]. The stated purpose is dataset- and problem-adaptive thresholding. This suggests a shift from heuristic cutoffs chosen a priori toward thresholds inferred from score distributions in the dataset under audit.

## 5. Binary-classification formulation and empirical performance

The study formalizes low-quality image detection as a binary classification task. Positive examples are degraded or otherwise problematic images, and negative examples are clean images. CleanVision provides the per-image scores, the dataset construction supplies ground-truth information on which images have been degraded, and a prediction is made by flagging an image as problematic when its score is below a threshold [2509.24420].

Evaluation uses the F1 score, with precision defined as the fraction of flagged images that were truly problematic and recall defined as the fraction of true problem images that were flagged. Under this formulation, adaptive thresholding substantially improves performance relative to the original thresholding strategy. For single perturbations, the best adaptive method, Li’s method, raises F1 from 0.6794 to 0.9468. For dual perturbations, Li’s method raises F1 from 0.7447 to 0.8557 [2509.24420].

Near-duplicate detection is also evaluated in this framework. The abstract reports that the deduplication strategy increases F1 from 0.4576 to 0.7928. The detailed CleanVision-related summary reports a baseline of 0.4579 for CleanVision, 0.6466 for Fastdup, and 0.7928 for the proposed combined pipeline [2509.24420]. Regardless of that minor reporting difference, the stated conclusion is that the refined workflow materially strengthens duplicate and near-duplicate detection.

## 6. Relationship to Fastdup and significance for data-centric image quality assessment

The study presents CleanVision and Fastdup as complementary rather than competing systems. Fastdup is summarized as detecting invalid images, duplicates, outliers, and brightness and blurriness; it uses deep ONNX models to compute semantic, latent-space features; its duplicate search is based on nearest neighbors with cosine similarity; its brightness score is the mean of the RGB channels; and its blur metric is Laplacian variance, the same primitive used by CleanVision for blur detection [2509.24420].

By contrast, CleanVision is described as excelling in direct pixel-level anomaly detection, including overexposed, underexposed, blurry, size-related, aspect-ratio-related, grayscale, and pHash-based duplicate issues. Fastdup excels in semantic or latent-space similarity, especially for near-duplicates that differ at the low-level representation while preserving meaning or content. The proposed pipeline therefore combines CleanVision’s pixel-level duplicate detection with Fastdup’s latent-space duplicate detection for more robust deduplication [2509.24420].

In the paper’s broader framing, this combination is significant for data-centric AI because it turns image dataset cleaning into a more principled and quantitative procedure. CleanVision’s score-based anomaly detection supports scalable audits; its modularity allows its thresholds and logic to be refined; and the binary-classification formulation permits direct performance measurement through F1. The study explicitly argues that these contributions provide a foundation for advancing data quality assessment in image-based machine learning [2509.24420].

Source: https://www.emergentmind.com/topics/cleanvision