Aes-R1: Multimodal Aesthetic Assessment

• Aes-R1 is a multimodal aesthetic reasoning framework that unifies textual critique generation with scalar score prediction for image aesthetic assessment.
• It employs a two-stage learning pipeline, starting with synthetic data construction via AesCoT and cold-start supervised fine-tuning, followed by Relative-Absolute Policy Optimization (RAPO).
• Evaluations on multiple datasets demonstrate improved ranking (SRCC) and score calibration (PLCC), though the method is computationally intensive due to multi-sample rollouts and staged training.

Searching arXiv for Aes-R1 and closely related aesthetic assessment work. Aes-R1 is a multimodal aesthetic reasoning framework for image aesthetic assessment that couples structured textual critique with scalar score prediction through a two-stage learning pipeline. It is introduced as a system for predicting aesthetic quality from images while also generating interpretable rationales, with outputs consisting of a critique and a final score in $[0,1]$ (Liu et al., 26 Sep 2025). The framework combines a synthetic reasoning-data construction process called AesCoT, cold-start supervised fine-tuning, and a reinforcement-learning method called Relative-Absolute Policy Optimization (RAPO), whose purpose is to jointly optimize absolute score regression and relative ranking order (Liu et al., 26 Sep 2025). In the paper’s formulation, Aes-R1 is not an image quality assessment method in the narrow distortion-centric sense, but a model for higher-level aesthetic judgment involving composition, lighting, color harmony, mood, and related visual-semantic factors (Liu et al., 26 Sep 2025).

1. Concept and scope

Aes-R1 is designed for Image Aesthetic Assessment (IAA), a task in which a model receives an image and predicts its aesthetic quality in a way that aligns with human judgments (Liu et al., 26 Sep 2025). The framework explicitly requires two outputs: a textual aesthetic critique or reasoning trace, and a final scalar aesthetic score in $[0,1]$ (Liu et al., 26 Sep 2025). This design distinguishes it from score-only regressors and from conventional image quality assessment systems focused on blur, compression, or other low-level distortions (Liu et al., 26 Sep 2025).

The central claim of Aes-R1 is that multimodal LLMs are well suited to IAA because they combine visual understanding, language generation, and cross-modal reasoning (Liu et al., 26 Sep 2025). In the authors’ account, these properties are relevant because aesthetic judgment depends not only on low-level signal properties but also on semantic and affective factors such as subject emphasis, atmosphere, mood, and visual balance (Liu et al., 26 Sep 2025). Aes-R1 therefore treats explanation as part of the prediction problem rather than as a post hoc wrapper around a regressed score (Liu et al., 26 Sep 2025).

The framework is motivated by three deficits identified in prior practice: the scarcity of multimodal aesthetic reasoning data, the subjective nature of aesthetic judgments, and the mismatch between explanation quality and score fidelity in score-only or weakly prompted systems (Liu et al., 26 Sep 2025). The paper further argues that reinforcement learning alone is insufficient to induce meaningful aesthetic reasoning if the backbone has not first learned a structured critique format, a point illustrated by the reported behavior of the Aes-R1-Zero variant (Liu et al., 26 Sep 2025).

2. Architecture and output formulation

Aes-R1 is organized as a two-stage framework consisting of AesCoT-based cold-start training followed by RAPO-based reinforcement learning (Liu et al., 26 Sep 2025). Its generative policy is written as

$$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$

where $c$ is the generated critique and $s$ is the final score conditioned on image $\mathcal{I}$ and prompt $\mathcal{P}$ (Liu et al., 26 Sep 2025). In the paper’s inference setup, prompts require reasoning to appear inside > ... tags and the scalar score inside <answer>...</answer>, with the score rounded to two decimals and constrained to $[0,1]$ (Liu et al., 26 Sep 2025).

This “reasoning-before-scoring” structure is treated as functionally important. The critique is intended to force inspection of multiple aesthetic dimensions before numerical prediction, thereby constraining the score to be supported by visible evidence and improving interpretability (Liu et al., 26 Sep 2025). The paper presents this as a mechanism for aligning free-form explanation with scalar judgment rather than merely producing chain-of-thought-style text for its own sake (Liu et al., 26 Sep 2025).

In implementation, the principal backbone is Qwen2.5-VL-7B-Instruct (Liu et al., 26 Sep 2025). The framework samples multiple outputs per image during reinforcement learning, but its deployed output remains per-image critique plus scalar score rather than pairwise preference only (Liu et al., 26 Sep 2025). This is important because the method optimizes both ranking and regression while preserving a single-image assessment interface.

3. AesCoT: synthetic aesthetic reasoning data

AesCoT is the data-construction component of Aes-R1. It is described as an aesthetic reasoning data construction pipeline that produces multimodal supervision tuples of the form

$(\mathcal{P}, \mathcal{I}_i, c_i, s_i),$

starting from image-score pairs

$$\mathcal{D} = \left\{ (\mathcal{I}_i, s_i), i \in [1,M] \right\}$$

and constructing

$[0,1]$0

(Liu et al., 26 Sep 2025). The critique $[0,1]$1 is synthetically generated by an external expert model and then filtered for quality (Liu et al., 26 Sep 2025).

The data-generation pipeline deliberately hides the exact numeric score during rationale construction. Instead, the original mean opinion score is grouped into three intervals—bad ($[0,1]$2), fair ($[0,1]$3), and good ($[0,1]$4)—and the expert model is asked to produce a critique aligned with the image and its coarse quality level without disclosing the underlying scalar target (Liu et al., 26 Sep 2025). This setup is intended to reduce direct score leakage while preserving score-conditioned explanation quality (Liu et al., 26 Sep 2025).

The resulting data are then aggressively filtered. The paper defines

$[0,1]$5

where $[0,1]$6 collects error categories including score leakage, reasoning inconsistency or alignment issues, and factual errors (Liu et al., 26 Sep 2025). Filtering combines automatic checks, model judgments, and human audits (Liu et al., 26 Sep 2025). The appendix states that GPT-4.1 is used as the external expert model for generating AesCoT trajectories (Liu et al., 26 Sep 2025).

AesCoT rationales are organized around five aesthetic dimensions: Light and Shadow, Mood and Narrative, Composition, Color, and Exposure (Liu et al., 26 Sep 2025). The paper describes these dimensions as covering tonal contrast and atmosphere, affect and implied story, balance and framing, palette control and hue relationships, and tonal distribution or brightness control, respectively (Liu et al., 26 Sep 2025). In this sense, AesCoT functions as a schema for decomposing aesthetic judgment into recurrent critical categories.

Two releases are reported: AesCoT-3K and AesCoT-10K (Liu et al., 26 Sep 2025). AesCoT-3K contains 1000 bad, 1000 fair, and 1000 good examples, while AesCoT-10K contains 2928 bad, 3312 fair, and 3660 good examples, sourced from AVA, TAD66K, and FLICKR-AES (Liu et al., 26 Sep 2025).

4. Learning procedure and RAPO

Aes-R1 uses a two-stage optimization procedure. The first stage is supervised fine-tuning on AesCoT. Its objective is written as

$[0,1]$7

This stage teaches the model to emit both critique and score in the required structure (Liu et al., 26 Sep 2025). The paper emphasizes that this cold-start stage is necessary because direct RL can improve score accuracy without producing substantive explanations (Liu et al., 26 Sep 2025).

The second stage is Relative-Absolute Policy Optimization (RAPO), the framework’s RL algorithm (Liu et al., 26 Sep 2025). RAPO is built around two reward components. The first is a relative rank reward based on pairwise preference consistency. Under the paper’s Gaussian assumption for score distributions, the pairwise comparison probability is defined as

$[0,1]$8

with ground-truth binary preference

$[0,1]$9

and rank reward

$$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$0

(Liu et al., 26 Sep 2025)

The second component is an absolute error reward,

$$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$1

which encourages score calibration with respect to the ground-truth MOS $$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$2 (Liu et al., 26 Sep 2025). The total reward is the sum

$$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$3

This directly encodes the paper’s thesis that aesthetic assessment should preserve both ordinal preference and numeric score fidelity (Liu et al., 26 Sep 2025).

RAPO further applies group-relative advantage normalization,

$$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$4

over multiple generations per input, and uses a clipped objective in the style of PPO/GRPO with KL regularization to a reference policy (Liu et al., 26 Sep 2025). The paper frames the novelty of RAPO not as the clipping mechanism itself, but as the reward design tailored to the joint ranking-and-regression character of IAA (Liu et al., 26 Sep 2025).

5. Evaluation, datasets, and empirical results

Aes-R1 is evaluated on five open-source image aesthetic assessment datasets: TAD66K, AVA, FLICKR-AES, PARA, and AADB (Liu et al., 26 Sep 2025). Training uses TAD66K, AVA, and FLICKR-AES, while PARA and AADB are used for out-of-distribution evaluation (Liu et al., 26 Sep 2025). All MOS values are normalized to $$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$5, and the main training set contains 15K samples collected in a 2:2:1 ratio from AVA, TAD66K, and FLICKR-AES (Liu et al., 26 Sep 2025).

The principal evaluation metrics are PLCC and SRCC, reflecting absolute score agreement and rank-order consistency, respectively (Liu et al., 26 Sep 2025). On the average across five datasets, Aes-R1 reports PLCC 0.6337 and SRCC 0.6186, compared with 0.4285 / 0.4589 for Qwen2.5-VL-7B, 0.5771 / 0.5663 for DeQA-Score, 0.5954 / 0.5813 for Q-Insight, and 0.4429 / 0.5930 for VisualQuality-R1 (Liu et al., 26 Sep 2025). The abstract summarizes these gains as improvements of the backbone’s average PLCC by 47.9% and SRCC by 34.8% (Liu et al., 26 Sep 2025).

Per-dataset Aes-R1 scores are reported as 0.4513 / 0.4248 on TAD66K, 0.6702 / 0.6619 on AVA, 0.7243 / 0.6973 on FLICKR, 0.7842 / 0.7666 on PARA, and 0.5386 / 0.5423 on AADB (Liu et al., 26 Sep 2025). The paper interprets the PARA and AADB results as evidence of robust OOD behavior under limited supervision (Liu et al., 26 Sep 2025).

Ablation results are central to the paper’s argument for RAPO. The reported average scores for reward variants are 0.4255 / 0.4433 for binary only, 0.5655 / 0.5600 for error only, 0.4542 / 0.5908 for rank only, 0.5964 / 0.5825 for binary + rank, and 0.6297 / 0.6102 for error + rank (Liu et al., 26 Sep 2025). This supports the claim that rank-only optimization improves SRCC while weakening PLCC, whereas the combined reward yields the best balance (Liu et al., 26 Sep 2025).

The study also examines the effect of cold-start SFT duration. Reported averages are 0.6297 / 0.6102 for 0 SFT epoch + RAPO, 0.6337 / 0.6186 for 1 epoch + RAPO, 0.6027 / 0.5903 for 2 epochs + RAPO, and 0.4624 / 0.4705 for 10 epochs + RAPO (Liu et al., 26 Sep 2025). The paper interprets this as an entropy-related tradeoff: a small amount of SFT is beneficial, but too much SFT reduces token entropy and impairs RL exploration (Liu et al., 26 Sep 2025).

6. Interpretation, significance, and limitations

Aes-R1 is significant chiefly as an attempt to unify explanation generation and aesthetic score prediction within a single multimodal policy (Liu et al., 26 Sep 2025). Its substantive novelty lies in the combination of synthetic reasoning-data construction, structured critique-first generation, and reward shaping that explicitly couples regression with ranking (Liu et al., 26 Sep 2025). A plausible implication is that Aes-R1 belongs to a broader class of post-supervised multimodal evaluators in which rationales are not auxiliary artifacts but part of the target behavior.

The framework also clarifies an important distinction within aesthetic prediction. A system can be good at ranking images yet poor at calibrating scores, or vice versa. The comparison with VisualQuality-R1—stronger SRCC than PLCC—illustrates this separation in the paper’s own benchmark table (Liu et al., 26 Sep 2025). RAPO is explicitly constructed to address that split objective, and the reward ablation supports the claim that the combined formulation is empirically better than either component alone (Liu et al., 26 Sep 2025).

At the same time, the paper acknowledges several limitations. Aesthetic judgment remains fundamentally subjective, and its ground truth is based on aggregated human opinion rather than an objective standard (Liu et al., 26 Sep 2025). The AesCoT data are synthetic and inherit the priors of GPT-4.1, which suggests the possibility of stylistic or cultural bias in the rationales (Liu et al., 26 Sep 2025). The method is also computationally expensive because it requires rationale generation, multi-sample rollouts with $$\tau = (c, s) \sim \pi_{\theta}(\cdot \mid \mathcal{I}, \mathcal{P}),$$6, long completion lengths up to 1024 tokens, and staged SFT plus RL training (Liu et al., 26 Sep 2025). In addition, explanation quality is supported mainly through case studies and filtering design rather than through a separate large-scale human-rated rationale benchmark (Liu et al., 26 Sep 2025).

In technical summary, Aes-R1 is best understood as a multimodal aesthetic assessment framework that makes the model explain before it scores, initializes that behavior with curated synthetic chain-of-thought data, and then refines it with an RL objective that joins relative ranking and absolute calibration (Liu et al., 26 Sep 2025). Within the literature represented by the paper, its contribution is therefore not a new aesthetic dataset alone, nor merely an RL fine-tuning recipe, but a unified formulation of aesthetic reasoning and scoring as a single multimodal policy-learning problem (Liu et al., 26 Sep 2025).