AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control

Abstract: Affective Image Manipulation (AIM) aims to evoke specific emotions through targeted editing. Current image editing benchmarks primarily focus on object-level modifications in general scenarios, lacking the fine-grained granularity to capture affective dimensions. To bridge this gap, we introduce the first benchmark designed for AIM termed AIM-Bench. This benchmark is built upon a dual-path affective modeling scheme that integrates the Mikels emotion taxonomy with the Valence-Arousal-Dominance framework, enabling high-level semantic and fine-grained continuous manipulation. Through a hierarchical human-in-the-loop workflow, we finally curate 800 high-quality samples covering 8 emotional categories and 5 editing types. To effectively assess performance, we also design a composite evaluation suite combining rule-based and model-based metrics to holistically assess instruction consistency, aesthetics, and emotional expressiveness. Extensive evaluations reveal that current editing models face significant challenges, most notably a prevalent positivity bias, which stemming from inherent imbalances in training data distribution. To tackle this, we propose a scalable data engine utilizing an inverse repainting strategy to construct AIM-40k, a balanced instruction-tuning dataset comprising 40k samples. Concretely, we enhance raw affective images via generative redrawing to establish high-fidelity ground truths, and synthesize input images with divergent emotions and paired precise instructions. Fine-tuning a baseline model on AIM-40k yields a 9.15% relative improvement in overall performance, demonstrating the effectiveness of our AIM-40k. Our data and related code will be made open soon.

• The paper introduces AIM-Bench, a comprehensive benchmark and AIM-40k dataset for fine-grained affective image manipulation using dual-path emotion modeling.
• The methodology employs hierarchical human-in-loop workflows and multi-stage filtering that integrates discrete and continuous affective frameworks.
• Experimental results demonstrate improved control over emotion-specific outputs, effective mitigation of positivity bias, and enhanced editing fidelity.

Benchmarking and Improving Affective Image Manipulation: A Comprehensive Overview of AIM-Bench

Introduction

Affective Image Manipulation (AIM) poses unique challenges distinct from general-purpose image editing, as it requires eliciting precise emotional responses through targeted visual changes. Existing benchmarks predominantly focus on object- or attribute-centric edits that are not designed to evaluate or quantify the fine-grained spectrum of affective modifications. The paper "AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control" (2604.10454) addresses critical gaps in the AIM landscape by proposing a rigorously constructed benchmark (AIM-Bench), a large-scale instruction-tuning dataset (AIM-40k), and a comprehensive evaluation suite tailored to the nuances of affective editing. This essay provides a detailed summary and analysis of the paper's contributions, methodologies, empirical findings, and implications for future research.

Dual-Path Emotional Modeling and Benchmark Design

The core innovation of AIM-Bench is its dual-path affective modeling scheme, which integrates the discrete Mikels emotion taxonomy with the continuous Valence-Arousal-Dominance (VAD) framework. This approach enables hierarchical control by supporting both categorical emotion shifts and graded intensity manipulations. The construction of AIM-Bench leverages hierarchical human-in-the-loop workflows and advanced generative models.

The process encompasses three main stages:

1. Data Collection: Stratified clustering yields a diverse set of 800 source images, reflecting the distributional variety of human-annotated emotions.
2. Emotion-Targeted Editing: Each source image is paired with an alternative emotion goal (distinct in both category and VAD vector), and multi-faceted editing instructions are generated using Gemini-2.5-Pro. Gemini-2.5-flash-image acts as the edit engine for producing diverse candidate outputs.
3. Hierarchical Controlled Filtering: Candidate images are subjected to iterative filtering—automatic evaluation (semantic, aesthetic, and preservation metrics), model-based affect recognition, and expert curation—to maximize both visual fidelity and affective accuracy.

   Figure 1: The AIM-Bench benchmark comprises 800 high-quality samples, spanning 8 emotions and 5 editing paradigms.

   

   Figure 2: The construction pipeline employs stratified sampling, emotion-targeted editing, and multi-stage filtering to curate the dataset.

This architecture rigorously ensures that each curated triplet (original, instruction, target) exhibits semantically coherent and finely graded affective transitions, mitigating the noise, ambiguity, and annotation bias that challenge prior datasets.

Evaluation Framework and Model Analysis

AIM-Bench introduces a composite evaluation suite, harmonizing rule- and model-based metrics to assess editing performance across three axes:

• Semantic Fidelity: Quantified via CLIP-T, VIEScore-SC, and GPT-4o-based ground-truth consistency (GTC).
• Aesthetic Quality: Measured by CLIP-based aesthetic predictors and VIEScore-PQ.
• Emotional Expressiveness: Assessed by discrete classification metrics (ACC/F1 using Gemini-2.5-Pro) and VAD-space alignment (Euclidean distance).

  Figure 3: Radar plot demonstrating task-level trade-offs for baseline and advanced models on AIM-Bench.

Empirical results covering 13 editing architectures reveal several insights:

• Large unified multimodal models (ULMs) and instruction-finetuned diffusion systems outperform conventional expert editors in emotion-driven scenarios.
• Positivity Bias is pervasive; most models demonstrate significantly higher accuracy and affective alignment when synthesizing positive emotional content, often due to pre-training data imbalance.
• There exists a clear hierarchy of editing competencies: shallow attribute manipulations are more robust than deep compositional or affective reasoning tasks.

  Figure 4: Qualitative comparison across editing systems illustrates limits in negative-emotion synthesis and strengths in ULM models.

Scalability and Bias Mitigation: The AIM-40k Dataset

To redress the positivity bias and limited scale of existing datasets, the authors propose a scalable inverse repainting strategy for constructing AIM-40k:

• High-Fidelity Repainting: Source images from EmoSet are captioned and conditionally regenerated to boost visual quality without altering emotional labels—sampling is filtered by CLIP-based semantic coherence.
• Inverse Synthesis: For each high-fidelity target (emotion-grounded), a divergent input image is synthesized with a contrasting affect, followed by natural language instruction generation that encodes the semantic and affect transition.
• The dataset yields 40,000 triplets balanced across eight emotion categories, ensuring comprehensive coverage of the affective-editing action space.

  Figure 5: AIM-40k employs a repaint-to-inverse-edit workflow enabling both fidelity and affective diversity.

Fine-tuning a state-of-the-art model (Qwen-Image-Edit-2509) on AIM-40k yields a 9.15% boost in overall benchmark score and a +22.1% increase in negative emotion classification accuracy, demonstrating effective bias correction and improved fine-grained control.

Figure 6: Confusion matrices visualizing how the fine-tuned model reduces positive emotion dominance and restores class balance.

Practical and Theoretical Implications

AIM-Bench and AIM-40k recalibrate the methodological foundations and evaluation standards for affective image manipulation. The dual-path emotion modeling, hierarchical data curation, and model-agnostic evaluation pipelines provide a robust experimental platform for benchmarking future AIM systems. The contrastive design of AIM-40k directly addresses class imbalance in affective contexts, a critical limitation in the practical deployment of generative models in areas such as digital creative tools, emotion-aware AI companions, therapeutic art systems, and media forensics.

From a theoretical perspective, the findings underscore that existing large-scale pretraining pipelines (on natural images) induce strong affective priors, limiting expressive range in emotionally negative or complex regions of the semantic space. Data-driven, balanced, and multi-level annotation strategies such as those in AIM-Bench represent a definitive solution framework for these limitations.

Future Directions

The research identifies two avenues for further exploration:

• Evaluation Robustness: Current ground-truth and metric pipelines rely on advanced, proprietary MLLMs, which may present reproducibility and transparency challenges. Subsequent work should consider open-source alternatives and/or consensus-based human-machine evaluation ensembles.
• Semantic-Affective Controllability: There remain open questions regarding the disentanglement and compositional logic of semantics and affect in image generation. Future architectures will benefit from explicit conditioning mechanisms that separate and recombine these dimensions for even finer control.

Conclusion

AIM-Bench establishes a rigorous standard for affective image manipulation, providing both a fine-grained benchmark and a large-scale, class-balanced dataset. Through hierarchical curation, human-machine evaluation, and explicit addressing of training-set biases, the proposed framework and experiments demonstrate substantial gains in emotional fidelity and editing controllability, setting a new baseline for subsequent research in emotion-aware generative models. The analytic breakdown and dataset resources detailed in this work are foundational contributions to the methodology and practice of affect-driven AI systems.