ArtiMuse-10K: Expert Image Aesthetics Dataset

• The paper introduces ArtiMuse-10K as an expert-annotated dataset that advances image aesthetics assessment via joint scoring and structured 8-dimensional analysis.
• It covers diverse visual domains such as photography, painting, AIGC, 3D, and graphic design, ensuring comprehensive evaluation across mixed media.
• The dataset serves as a benchmark for multimodal aesthetic reasoning, supporting both precise numeric score prediction and expert-level textual appraisal.

ArtiMuse-10K is the dataset component of the ArtiMuse project, introduced together with the multimodal LLM ArtiMuse for fine-grained Image Aesthetics Assessment (IAA) with “Joint Scoring and Expert-Level Understanding.” It comprises 10,000 images spanning 5 main categories and 15 subcategories, and each image is annotated by professional experts with 8-dimensional attributes analysis and a holistic score. The dataset is positioned as the first expert-curated image-aesthetic dataset of this scale with both 8-dimensional attribute analysis and a holistic score, and it is intended to support professional, fine-grained, and interpretable aesthetic assessment across mixed visual domains including design, photography, painting/calligraphy, 3D, and AI-generated imagery (Cao et al., 19 Jul 2025).

1. Conceptual role in image aesthetics assessment

ArtiMuse-10K was created to address three limitations that the ArtiMuse paper identifies in prior IAA resources. First, many datasets provide only overall aesthetic scores and no textual interpretation. Second, some datasets include text, but the text is coarse, emotional, or impressionistic rather than structured expert critique. Third, most are domain-narrow, especially centered on photography, with limited coverage of AIGC, design, or other mixed-domain material. In response, ArtiMuse-10K formalizes aesthetics as a multi-dimensional expert task rather than an opaque scalar regression problem (Cao et al., 19 Jul 2025).

The dataset therefore serves several roles simultaneously. It is the expert-annotated core dataset used to train and evaluate ArtiMuse’s joint scoring and expert-level understanding capability; it defines the eight-attribute schema; it provides expert references for textual evaluation; it functions as a dedicated scoring benchmark; and it supports human and MLLM-based assessment of explanation quality. The broader ArtiMuse pipeline also uses a much larger pool of public datasets converted into structured supervision, but ArtiMuse-10K is the high-quality anchor set rather than just one more source of weak labels (Cao et al., 19 Jul 2025).

A central design principle is that the aesthetic schema is intended to be content-agnostic and applicable “from natural to AIGC.” This suggests that the dataset’s significance lies not only in annotation density, but also in its attempt to normalize heterogeneous visual media under a single evaluative framework. A plausible implication is that ArtiMuse-10K is better understood as a benchmark for multimodal aesthetic reasoning than as a conventional narrow-domain MOS-style dataset (Cao et al., 19 Jul 2025).

2. Corpus composition, categories, and data sources

ArtiMuse-10K contains exactly 10,000 images. The appendix specifies a split of 9,000 training images and 1,000 test images; no validation split is specified. The train and test score distributions are described as approximately Gaussian, like other aesthetic scoring datasets, but with relatively strong score diversity (Cao et al., 19 Jul 2025).

Main category	Images
Photography	4,111
Painting / Calligraphy	3,095
AIGC	1,453
3D Design	823
Graphic Design	518

The 15 subcategories make the cross-domain structure explicit. Photography comprises Daily Photo (3,071), Photographic Art (758), Architecture (119), Portrait (82), and Movie still (81). Painting / Calligraphy comprises Digital Art (1,314), Children’s Painting (699), Chinese Painting (511), General Painting (485), Sketch (43), and Calligraphy (43). AIGC is represented by a single AIGC subcategory with 1,453 images. Three-dimensional design comprises Product Design (516) and Sculpture (307). Graphic Design contributes 518 images under a single Graphic Design subcategory (Cao et al., 19 Jul 2025).

The image sources are mixed. For non-AIGC images, the authors state that they collaborated with domain experts to curate professionally created works from academic settings, including student assignments and competition entries, as well as works collected from “reputable online art and photography platforms.” For AIGC images, they used state-of-the-art generation systems including the Stable Diffusion series, Dreamlike Photoreal 2.0, and FLUX, and they additionally included open-source community contributions created with comparable architectures (Cao et al., 19 Jul 2025).

The paper describes the images as “carefully curated” and “professionally selected,” but does not provide an explicit deduplication algorithm, a per-image provenance manifest, or a formal license table for ArtiMuse-10K. This omission is consequential for reproducibility and redistribution analysis, even though it does not diminish the dataset’s stated benchmark function (Cao et al., 19 Jul 2025).

3. Annotation schema, expert protocol, and score semantics

Each image in ArtiMuse-10K is annotated with textual analysis for 8 aesthetic attributes and an overall aesthetics score. The paper describes this as “structural analysis,” meaning that the textual supervision is organized around predefined dimensions rather than free-form holistic commentary (Cao et al., 19 Jul 2025).

The annotators are described as professional experts in aesthetics and art-related domains. Their experience ranges from at least 3 years to over 30 years, and the expert pool includes “distinguished authorities in the field.” The paper, however, does not specify the number of annotators per image, whether annotations were single-expert or multi-expert, or whether the final labels were consensus judgments, averages, or adjudicated outcomes. It also does not report inter-annotator agreement statistics such as Cohen’s $\kappa$, Krippendorff’s $\alpha$, or correlation-based measures (Cao et al., 19 Jul 2025).

The eight attributes are defined in the appendix as follows:

• Composition / Design: “Evaluate the balance, contrast, layout aesthetics, and rhythm of the composition. Focus on the use of dynamic focal points, unity, and harmony in the design.”
• Visual Elements / Structure: “Analyze the interplay of color, geometry, spatial organization, and illumination to optimize visual contrast and structural clarity.”
• Technical Execution: “Examine the mastery of medium and materials, including brushstrokes, focus, exposure, light handling, as well as clarity and resolution of the image.”
• Originality / Creativity: “Analyze the uniqueness of the concept and execution, focusing on how the work exceeds common styles with imagination, and creative breakthroughs.”
• Theme / Communication: “Evaluate the clarity of the subject and its communication. Consider how effectively the narrative, cultural significance, and societal context are conveyed.”
• Emotion / Viewer Response: “Assess how well the work evokes an emotional response, engages the viewer, and creates lasting impressions with personal significance.”
• Overall Gestalt: “Evaluate the overall visual appeal and artistic impact of the image, considering how well the elements combine to create an engaging, meaningful impression.”
• Comprehensive Evaluation: “Provide a comprehensive aesthetics assessment of the image, evaluating its effectiveness in visual impact, theme communication, and artistic depth.” (Cao et al., 19 Jul 2025)

The holistic score is described as an “Overall aesthetics score derived from multi-dimensional evaluation.” In model training and inference, scores are normalized to the $[0,100]$ range and mapped to one of 101 score tokens corresponding to integers $0$ through $100$. The paper does not provide a formal aggregation rule from the eight attributes to the overall score, so the score is defined operationally rather than by an explicit weighted formula (Cao et al., 19 Jul 2025).

A recurring motivation for expert annotation is the paper’s claim that MLLMs exhibit a positivity bias, tending to generate overly positive assessments regardless of image quality. ArtiMuse-10K is therefore framed not merely as a labeled image set, but as a corrective to pseudo-annotation regimes that the authors regard as insufficiently balanced and reliable (Cao et al., 19 Jul 2025).

4. Benchmark tasks and evaluation protocols

ArtiMuse-10K supports at least three benchmark formulations. The most direct is aesthetic score prediction: given an image, predict its overall aesthetic score in $[0,100]$. The paper evaluates this task with Spearman rank correlation coefficient (SRCC) and Pearson linear correlation coefficient (PLCC), which are the principal quantitative metrics reported on ArtiMuse-10K and related datasets (Cao et al., 19 Jul 2025).

A second task is fine-grained textual aesthetic analysis. Here the input is an image and an attribute prompt, and the output is a short textual evaluation for that attribute. Rather than using BLEU, ROUGE, or CIDEr, the paper evaluates this task with MLLM-as-judge comparison, where Gemini-2.0-flash selects the best model output using the human expert text as reference, and with a human preference study in which volunteers compare outputs and vote (Cao et al., 19 Jul 2025).

A third task is the joint score-and-understanding setting emphasized by the ArtiMuse project. In this formulation, a system is expected to provide both precise scalar scoring and expert-level explanatory understanding. ArtiMuse-10K is what enables this joint benchmark because it pairs structured expert rationales with numeric supervision. The dataset format is therefore effectively: image, 8 textual fields, and 1 overall score (Cao et al., 19 Jul 2025).

The paper also indicates an instruction-style usage pattern. Prompt templates include “Please evaluate the aesthetic quality of this image from the attribute of <attribute>,” and, for generic score-only inference, “Please rate the aesthetic quality of this image and provide a score between 0 and 100, where 0 represents the lowest quality and 100 represents the highest. Your response should contain only an integer value.” This makes ArtiMuse-10K suitable ոչ only for regression-style benchmarking but also for multimodal instruction tuning and explanation generation (Cao et al., 19 Jul 2025).

5. Relation to the ArtiMuse model and reported results

ArtiMuse is built on InternVL-3-8B. The paper states that most of InternVL’s components are retained, but the default dynamic-resolution strategy is replaced with a fixed-resolution input strategy, motivated by the claim that aesthetics depends heavily on holistic properties such as composition, color harmony, and spatial relationships. The architecture includes a vision encoder, an MLP, and an LLM fine-tuned using LoRA; the appendix names InternViT-300M-448px-V2.5 as the vision encoder and Qwen2.5-7B as the LLM. Input resolution is $448 \times 448$, and maximum sequence length is 8192 (Cao et al., 19 Jul 2025).

The training pipeline has two stages. In text pretraining, the model is trained on the full collected image dataset with aesthetic-analysis captions so that it learns structural aesthetic analysis while using LoRA to preserve prior knowledge. In score fine-tuning, the model is adapted for score prediction by converting each score into a special score token. The paper calls this mechanism “Token As Score” and states that 101 existing tokens corresponding to scores 0 to 100 are designated, with examples such as aa $\rightarrow 0$, ab $\rightarrow 1$, and ey $\rightarrow 100$ (Cao et al., 19 Jul 2025).

On ArtiMuse-10K, the main score-prediction result reported for ArtiMuse is SRCC $\alpha$0 and PLCC $\alpha$1. The comparison table reports Q-Align at $\alpha$2, PEAS at $\alpha$3, Qwen-2.5-VL-7B at $\alpha$4, InternVL3-8B at $\alpha$5, and MUSIQ at $\alpha$6. Supplementary comparisons against larger general-purpose MLLMs report GPT-4o at $\alpha$7, Gemini-2.0-flash at $\alpha$8, Qwen-2.5-VL-72B-instruct at $\alpha$9, and InternVL3-78B at $[0,100]$0 (Cao et al., 19 Jul 2025).

Method	SRCC	PLCC
ArtiMuse	0.614	0.627
Q-Align	0.551	0.573
PEAS	0.306	0.293
Qwen-2.5-VL-7B	0.256	0.179
InternVL3-8B	0.187	0.157

The textual-understanding results are similarly emphasized. Across the 8 attributes, MLLM-judge selection rates for ArtiMuse are 76.9% for Composition / Design, 64.2% for Visual Elements / Structure, 79.7% for Technical Execution, 77.8% for Originality / Creativity, 58.5% for Theme / Communication, 58.5% for Emotion / Viewer Response, 75.9% for Overall Gestalt, and 71.7% for Comprehensive Evaluation, with an average of 71.1%. The corresponding human preference rate is 67.8%, compared with 19.2% for InternVL3-8B, 11.5% for Qwen-2.5-VL-7B, and 1.5% for AesExpert (Cao et al., 19 Jul 2025).

ArtiMuse-10K is also used in ablation analysis. On this dataset, the paper reports that “Existing 100 Tokens (ordered)” achieves $[0,100]$1, outperforming “5 Levels” at $[0,100]$2 and several token-expansion strategies that perform much worse. The paper interprets this as evidence that score-token design is not a superficial implementation detail but materially affects convergence and evaluation on ArtiMuse-10K (Cao et al., 19 Jul 2025).

6. Limitations, availability, and disambiguation

The paper states that both the model and dataset will be made public and provides the project page https://thunderbolt215.github.io/ArtiMuse-project/. At the same time, the paper excerpt does not provide a final download URL, a dataset license for ArtiMuse-10K itself, or release packaging details. This leaves open practical questions about redistribution, downstream commercial use, and source-image licensing, especially given the mixture of academic works, platform-collected images, and AIGC material (Cao et al., 19 Jul 2025).

Methodological omissions are also notable. The paper does not specify the number of annotators per image, does not define a consensus or adjudication procedure, and does not report inter-annotator agreement. It also does not provide a formal equation linking the eight attributes to the holistic score. These are not minor editorial gaps: they constrain statistical interpretation of label reliability and make it harder to separate expert consistency from model fit to a single canonical annotation set (Cao et al., 19 Jul 2025).

A further limitation concerns scope. In the conclusion, the authors state that the current model is limited to “understanding and analyzing” and cannot provide professional aesthetic enhancement recommendations. This suggests that ArtiMuse-10K presently supports evaluative and explanatory IAA more directly than prescriptive creative assistance (Cao et al., 19 Jul 2025).

ArtiMuse-10K is also easy to confuse with unrelated but similarly named resources. It is not the same as ArtMUSE, a dataset introduced for multimodal creative writing with approximately 3,000 calibrated Chinese text-image pairs for MMCW; that resource concerns illustrated article generation rather than image aesthetics assessment (Chen et al., 22 Aug 2025). Nor is it the same as the artwork–music resources discussed in adjacent cross-modal generation work: ArtSound contains 105,884 artwork–music pairs, and that paper refers to a prior Art2Mus dataset with 10,000 pairs, but it does not mention ArtiMuse-10K (Rinaldi et al., 19 Feb 2026). The shared “arti/art” prefix and the presence of a “10K” scale marker in adjacent literature make nomenclatural confusion plausible, but the task definitions, modalities, and annotation schemas are distinct.

Taken together, ArtiMuse-10K occupies a specific place in the 2025 multimodal evaluation landscape: it is a 10,000-image, expert-annotated, mixed-domain benchmark for structured aesthetic analysis and scalar aesthetic scoring, designed to move IAA beyond photography-centric score prediction toward professional, interpretable, and attribute-grounded multimodal assessment (Cao et al., 19 Jul 2025).