Banana100: NR-IQA & Iterative Editing Analysis

• The dataset systematically quantifies NR-IQA failures by demonstrating how iterative multi-step editing induces undetected quality deterioration.
• Banana100 is a controlled repository of 36,400 images from 100-step editing sequences designed to isolate noise accumulation effects in generative pipelines.
• Evaluation findings reveal that classical NR-IQA metrics fail to track degradation while large-VLM methods like RALI and VisualQuality-R1 better capture quality decline.

Banana100 is a large-scale dataset designed to expose and systematically quantify the limitations of no-reference image quality assessment (NR-IQA) metrics in detecting degradation caused by iterative multi-step image editing in multi-modal agentic systems. Comprised of approximately 36,400 images, Banana100 consists of 100-step editing sequences across diverse seed images, capturing both the process and effects of noise accumulation resulting from repeated “replication” operations by state-of-the-art image editing models. The dataset provides a controlled environment to evaluate the failure modes of classical and modern NR-IQA methods, revealing that existing evaluators are largely incapable of identifying quality collapse in such scenarios (Tang et al., 3 Apr 2026).

1. Construction Protocol and Dataset Organization

The central element of Banana100 is its meticulous construction protocol, which aims to isolate the impact of iterative editing independent of semantic or geometric modification. The procedure comprises the following steps:

• Seed Selection: 13 high-resolution AI-generated images (resolution range: 1,664×2,432 to 5,632×3,072) were curated under five criteria: copyright-free, minimal artifacts, stylistic and textural diversity, and absence of real human faces. The set includes 11 photorealistic scenes produced by Nano Banana Pro and two anime-style character grids created with SPICE.
• Iterative Editing: Each sequence entailed the core instruction “Produce an exact replica of the provided image, with no alterations,” repeated for 100 consecutive editing steps. The output of each step is used as input for the next, and each run is conducted in a fresh Nano Banana Pro API session to avoid session-based drift. Five independent runs per seed image accommodate model sampling randomness.
• Prompt & Hyperparameter Variants: Variations in the instruction (“duplicate exactly,” “recreate without change”), inversion maneuvers (e.g., mirror, rotate, decolorize/colorize reference), hyperparameter sweeps (seed strategy, temperature, resolution), and localized object-addition protocols (“add-apples,” “add-100-fruits”) were explored to assess stability and generality.
• Baseline Model Comparisons: Additional 20-step replication experiments were performed with Nano Banana 2 Fast, FLUX.2 [dev], and Qwen Image Edit 2.5B, each with the same replication prompts and seed set, permitting model-agnostic benchmarking.

Table: Dataset Composition Breakdown

Subset	Images (Approx.)	Description
Replication runs	28,000	13 seeds × 100 steps × 5 runs
Prompt/hyperparameter variants	~4,200	Alternative replicates, reference-only edits
Object addition experiments	~4,200	Targeted object insertions, e.g., apples
Total	36,400	Comprehensive dataset size

No cropping, geometric, or photometric augmentations were applied except those inherent in model API settings or explicit inversion experiments.

2. Image and Texture Diversity

Banana100’s seed images span a wide range of content and textures to test generality of observed degradation. Categories include:

• Architecture: Building grids.
• Food: Dongpo potstickers, rice terraces, Holi powder explosion.
• Nature: Foggy forest, sand dunes, mossy bark.
• Man-made Interiors: Library, wooden table.
• Textural Objects: Ekphrasis still life, peacock feather.
• Animation: Two anime-style character grids (Kokoro, Yuiman).

Texture modalities represented cover regular patterns, organic irregularities, high-contrast particle fields, smooth gradients, and multi-scale detail. Resolutions exceed 2K for 80% of the replication runs, with targeted subsets at 1K and 4K.

3. Quantification of Degradation and Metric Definition

Banana100 characterizes both perceptual and task-induced degradation:

• Artifact Accumulation: After 5–10 replication steps, visibly perceptible artifacts such as static noise, greenish tints, blurring, and scatter points emerge. Beyond visual noise, semantic degradation is observed, including counting errors and object misplacement in object-addition runs.
• Metric Exclusion: Full-reference measures—PSNR, SSIM, and LPIPS—are omitted due to lack of access to reference images in multi-agentic pipelines.
• No-Reference IQA Metrics: 21 classical NR-IQA metrics (from pyiqa), including ARNIQA, BRISQUE, NIQE, PIQE, MUSIQ, NIMA, CLIPIQA, MANIQA, and TOPIQ NR, are benchmarked. Two large-VLM approaches, RALI and VisualQuality-R1, are included to test scaling trends.

Normalization is performed to map each metric's native range to $[0, 100]$; quality change is summarized as the normalized gap $\Delta_i = N_i - N_1$ at steps $i \in \{5, 10, 20\}$, with negative $\Delta_i$ indicating correct detection of iterative quality decline.

4. Evaluation Findings: Breakdown of NR-IQA

• Classical NR-IQA Failure: Across all 21 metrics, heatmaps indicate that every method assigns higher quality to degraded images in at least one seed (e.g., BRISQUE raw: 34.1 at step 1 vs. $-9.8$ at step 20 yields positive $\Delta_{20}$ after normalization). No metric demonstrates monotonic loss tracking (all $\Delta_5, \Delta_{10}, \Delta_{20} < 0$) across the complete 12-seed test set.
• Large-VLM Successes: RALI and VisualQuality-R1 exhibit zero failures in core replication—i.e., $\Delta_i < 0$ on all seeds and at all checkpoints—though RALI's scores fluctuate during semantic region edits. VisualQuality-R1 violates its minimum occasionally (score $<1$) but maintains the correct global trend.
• Thresholds of Failure: Classical NR-IQA methods typically lose correlation with visible quality beyond 5–10 editing steps, coinciding with the emergent artifact threshold for both human observers and large-VLMs.
• Extension to Other Models: Experimentation with Nano Banana 2, FLUX.2, and Qwen demonstrates equivalent artifact emergence and NR-IQA metric failure; noise patterns vary (e.g., wrinkles, scatter, texture duplication) but consistently evade detection by classical evaluators.

5. Applications and Recommendations

Banana100 provides a highly controlled and scalable testbed for NR-IQA and generative modeling research:

• For NR-IQA Development: The dataset serves as a critical complement to distortion-centric corpora (e.g., KonIQ-10k), enabling evaluation under real, model-induced degradations. It supports the design of metrics robust to generative model artifacts, with large-VLM-based metrics like RALI and VisualQuality-R1 as strong baselines but caveated by their own semantic editing sensitivities.
• For Generative Pipelines: Results highlight the necessity of internal denoising mechanisms or artifact-aware feedback loops within multi-step editors, as well as inversion pipelines that suppress iterative noise without incurring semantic drift.
• Operational Guidelines: Classical no-reference metrics (BRISQUE, NIQE, PIQE, etc.) should not be relied upon solely for multi-turn data filtering. NR-IQA tools must be validated against Banana100 degradations pre-deployment in production or research pipelines. A hybrid quality control strategy—combining artifact detection with large-language-model-based semantic consistency checks—is recommended for robust filtration.

6. Access and Community Contribution

Banana100 is publicly available at https://huggingface.co/datasets/kenantang/Banana100 and is intended to support community-driven advances in multi-modal agentic content creation, robust image editing, and NR-IQA formulation. The dataset’s inclusion of both core replication, variant, and region-focused editing runs provides a spectrum of challenges suitable for benchmarking methodological improvements and for stress-testing evaluative metrics under high-iteration, real-model noise (Tang et al., 3 Apr 2026).