CreativePair: Explainable Ad Selection

• CreativePair is a behavioral dataset of paired advertising images annotated with binary CTR labels and a 10-step structured evaluation protocol.
• It transforms creative ranking into an explainable chain-of-thought task by integrating human judgments with real-world performance metrics.
• Comprising 8,817 pairs from e-commerce serving logs, it enables fine-grained analysis of creative selection and multimodal reasoning.

Searching arXiv for the primary and related papers to ground the article in current literature. arXiv search query: Creative4U CreativePair advertising creative selection comparative reasoning CreativePair is a comparative reasoning-induced dataset for explainable advertising creative image selection introduced alongside Creative4U, a multimodal LLM system for creative assessment and selection in e-commerce advertising (Lin et al., 18 Aug 2025). In its strict technical sense, CreativePair denotes a collection of real advertising image pairs grounded in serving logs, each pair carrying both a binary label indicating which image achieved higher click-through rate and a structured reasoning trace describing why one creative is preferable. The dataset was designed to address a dual problem that earlier creative ranking pipelines did not resolve: selecting the highest-performing creative before deployment and explaining that selection in human-readable terms. In the supplied literature, the label “CreativePair” is also applied more broadly to paired-data and pairing-centric creative AI paradigms, but the dataset in Creative4U remains its primary referent (Lin et al., 18 Aug 2025).

1. Origin and Problem Formulation

CreativePair emerged from a specific industrial setting: modern AIGC pipelines can produce dozens or hundreds of candidate creative images for a single product, yet advertisers still require a principled way to determine which image should be served and why (Lin et al., 18 Aug 2025). The central claim of the Creative4U work is that existing methods primarily focus on creative ranking, whereas the operational need is explainable creative selection. CreativePair was constructed to support that shift.

The dataset is organized around pairwise comparison rather than absolute scoring. Each sample consists of two creative images for the same product, a query signal derived from real user behavior, a winner label tied to observed CTR, and a multi-step reasoning trace. This pairwise design is consequential. It turns creative evaluation from an opaque scalar prediction problem into a structured comparative reasoning problem in which a model must discriminate between two alternatives and justify the choice (Lin et al., 18 Aug 2025).

This framing also clarifies what CreativePair is not. It is not merely an aesthetic preference corpus, nor a generic vision-language benchmark. Its labels are anchored in real-world advertising outcomes, specifically CTR differences under exposure constraints, and its rationales are organized by an explicit evaluation protocol rather than unconstrained free-form commentary (Lin et al., 18 Aug 2025).

2. Dataset Construction and Statistical Basis

CreativePair contains 8,817 annotated pairs, partitioned into 7,088 training pairs and 1,729 test pairs (Lin et al., 18 Aug 2025). The pairs are mined from large-scale e-commerce serving logs. For each product, the collection pipeline records all associated creative images together with PV and CTR, then forms all candidate pairs for that product and filters them using two constraints: each image must have PV $\ge 1{,}000$, and the CTR difference between the two images must be at least $60\%$ (Lin et al., 18 Aug 2025).

Each retained pair is also tagged with the single most frequent user query associated with the product, filtered to adjectives only. This query attachment is important because Creative4U is described as taking users’ interests into account, so the dataset does not only encode visual comparison; it also preserves a coarse user-intent signal (Lin et al., 18 Aug 2025).

The final label is a binary decision $a \in \{A,B\}$ indicating which image achieved the higher real-world CTR. The underlying preference relation is described as

$$\ell_{ij} = \begin{cases} 1 & \text{if } \mathrm{CTR}(I_i) > \mathrm{CTR}(I_j) + \delta,\ 0 & \text{otherwise,} \end{cases} \qquad \delta = 0.6.$$

This construction gives CreativePair a distinctly behavioral grounding. The supervision signal is not an annotator’s subjective preference alone, but an observed performance differential filtered to emphasize substantial separation (Lin et al., 18 Aug 2025). At the same time, this filtering choice shapes the benchmark’s difficulty profile, because pairs are intentionally skewed toward cases with large performance gaps rather than near ties.

3. Annotation Protocol and Comparative Reasoning Structure

Beyond the winner label, every CreativePair sample includes a 10-step structured annotation called the Creative Evaluation Protocol (Lin et al., 18 Aug 2025). The protocol decomposes pairwise judgment into three major components: similarity, image-text consistence, and image quality, followed by a final decision. Specifically, Q1–Q2 address similarity, Q3–Q4 address image-text consistence, Q5–Q9 cover five image quality facets—Text in Image, Models/Props, Layout, Product Subject, and Background—and Q10 asks which image users are more likely to click (Lin et al., 18 Aug 2025).

For Q1–Q9, annotators choose among $A > B$, $A = B$, and $A < B$. Q10 returns $A$ or $B$. The protocol is sequential. If either Q1 (“Are the two images the same?”) or Q2 (“Too similar to judge?”) is answered YES, the pair is discarded (Lin et al., 18 Aug 2025). This decision criterion ensures that the retained dataset emphasizes discriminable comparisons rather than redundant duplicates.

The annotation workflow includes a pre-annotation stage in which 10% of collected pairs are first labeled by senior creative designers, and their free-form reasoning is distilled into the 10-question protocol (Lin et al., 18 Aug 2025). During formal annotation, any disagreement on Q1–Q2 also leads to pair removal. Although the paper does not report a full inter-annotator agreement table for every question, it states that the multi-step design was adopted only after observing Cohen’s $\kappa > 0.75$ on the pilot (Lin et al., 18 Aug 2025).

After the human annotation steps are completed, Qwen2.5-VL-72B-Instruct is prompted to translate the discrete protocol answers into a fluid natural-language chain of thought that is strictly consistent with the annotated choices (Lin et al., 18 Aug 2025). This makes CreativePair unusual among advertising datasets: the rationale is not a free-form post hoc explanation disconnected from the labels, but a synthesized narrative constrained by stepwise human judgments.

4. Function Within Creative4U

CreativePair is the supervision substrate for Creative4U’s explainable selection pipeline (Lin et al., 18 Aug 2025). Creative4U is described as an MLLMs-based creative selector that integrates assessment and selection into a natural language generation task. Its training recipe, termed Reason-to-Select RFT, combines supervised fine-tuning with Chain-of-Thought and reinforcement learning via Group Relative Policy Optimization. In the paper’s terminology, this includes CoT-SFT followed by GRPO-based reinforcement learning (Lin et al., 18 Aug 2025).

Within that training regime, CreativePair provides both the terminal preference label and the intermediate reasoning structure. This dual supervision matters because it supports two objectives simultaneously: selecting the higher-CTR image and generating an explanation aligned with the evaluation protocol. In effect, the dataset turns comparative advertising judgment into a reasoned sequence prediction task rather than a pure ranking loss.

The dataset also supports fine-grained evaluation. The per-question annotations make it possible to assess not only whether a model selected the correct image, but whether its intermediate judgments align with ground truth. The supplied material identifies a custom Qwen-Score for measuring alignment between generated and ground-truth rationales (Lin et al., 18 Aug 2025). This suggests an evaluation regime in which reasoning quality is itself observable rather than latent.

A further downstream use is explainable ranking of more than two candidates. Given $60\%$0 creative images, one can form the full $60\%$1 set of pairwise comparisons, collect model preferences, and compute each image’s win frequency as

$60\%$2

The result is a rank list whose ordering can be accompanied by pairwise reasons rather than only scores (Lin et al., 18 Aug 2025). The same rationale traces can also be aggregated into attribute-level feedback, such as average performance on “Text in Image,” to support human designers or AIGC systems.

5. Broader Uses of “CreativePair” in Paired Creative AI

The supplied literature also uses “CreativePair” as a broader label for pairing-based creative AI paradigms, though these usages are conceptually adjacent to, rather than identical with, the CreativePair dataset.

In fashion recommendation, the c$60\%$3GAN work is presented in the supplied material as a “CreativePair” system for complementary pairing of clothing items. Its objective is to learn the conditional distribution $60\%$4 where $60\%$5 is a top garment and $60\%$6 is a complementary bottom garment, using a conditional GAN with a compound generator loss and randomized label flipping in the discriminator (Kumar et al., 2019). Here the pairing is between heterogeneous but complementary visual items, not between two alternatives competing for selection.

In paired image editing, “Customize Your Own Paired Data via Few-shot Way” formalizes few-shot learning from paired examples through directional transformations between intra-domain samples (Chen et al., 2024). Given $60\%$7 paired examples, the framework expands supervision from $60\%$8 examples to roughly $60\%$9 ordered directions by sampling source-target pair relations. The pairing structure is therefore generative and transformational rather than comparative.

A related but more minimal formulation appears in “Customizing Text-to-Image Models with a Single Image Pair,” where a single content/stylized pair is used to learn stylistic difference by separating style and content into distinct LoRA subspaces (Jones et al., 2024). In that setting, the pair is not a preference comparison but a demonstration of transformation: the model is optimized to reconstruct the content image with a content adapter and the stylized image with combined content and style adapters.

The supplied overview of PortfolioMentor explicitly describes the system as a “CreativePair” vision for multimodal co-creation (Long et al., 2023). PortfolioMentor structures interactive digital art portfolio creation into six stages—task and vision understanding, visual illustration generation, audio or music suggestion, interaction design, vision narrative and polishing, and an always-on IDE chatbot—thereby operationalizing pairing as a collaboration between creator and AI across modalities rather than as an image-image comparison (Long et al., 2023).

A still different pairing regime appears in “Personality Pairing Improves Human-AI Collaboration,” which studies how human Big Five traits interact with AI personalities in the production of display ads (Ju et al., 17 Nov 2025). The supplied material uses this paper as a blueprint for a “CreativePair” human-AI collaboration system. Here the pair is neither two images nor a source-target example, but a human-AI team whose productivity, teamwork quality, and ad performance depend on personality alignment. The reported findings include jagged task-specific effects, productivity-performance trade-offs, and field outcomes measured through CTR and other campaign metrics (Ju et al., 17 Nov 2025).

Taken together, these works show that “CreativePair” in the supplied corpus spans at least four pairing logics: comparative preference pairs, transformational example pairs, complementary recommendation pairs, and human-AI partner pairs. The dataset introduced in Creative4U is the most formalized of these, because it binds pairwise reasoning to a fixed evaluation protocol and real advertising outcomes.

6. Limitations, Scope Conditions, and Prospective Extensions

CreativePair’s most visible limitation is its extreme-pair bias (Lin et al., 18 Aug 2025). Because the construction retains only pairs whose CTRs differ by at least $a \in \{A,B\}$0, the dataset over-represents relatively easy decisions and under-samples near-tie cases. This improves label reliability from the standpoint of behavioral separation, but it narrows the decision boundary that a model encounters during training and evaluation.

Its domain scope is also narrow by design. The dataset is built solely on e-commerce product ads and adjective queries, so it does not cover other creative regimes such as brand campaigns or lifestyle photography (Lin et al., 18 Aug 2025). This matters because many of the protocol’s criteria—such as product subject salience or text-in-image quality—may behave differently outside product-centric advertising.

A further limitation is the lack of detailed reported multi-annotator statistics per question (Lin et al., 18 Aug 2025). The pilot agreement threshold, Cohen’s $a \in \{A,B\}$1, establishes that the structured protocol improved reliability, but it does not calibrate uncertainty across individual protocol steps or attribute dimensions.

The proposed extensions in the supplied material point toward richer supervision. One is to replace binary labels with a graded preference score $a \in \{A,B\}$2, potentially derived from fine-grained CTR deltas or multi-annotator consensus (Lin et al., 18 Aug 2025). Another is to expand the annotation unit from two images to three or more images, enabling explainable top-$a \in \{A,B\}$3 selection directly rather than through pairwise aggregation. Additional extensions include releasing detailed agreement statistics and applying the same protocol to social-media creatives or video thumbnail selection (Lin et al., 18 Aug 2025).

A plausible implication is that CreativePair’s protocolized comparative supervision could serve as a transferable template across other pairing-based creative systems. The paired-data literature surveyed above already shows that creative AI repeatedly returns to structured relations between examples, alternatives, or collaborators (Kumar et al., 2019, Chen et al., 2024, Jones et al., 2024, Long et al., 2023, Ju et al., 17 Nov 2025). CreativePair’s distinct contribution within that landscape is to make those relations explainable, behaviorally grounded, and directly usable for multimodal reasoning models.

7. Significance in Research on Explainable Creative Selection

CreativePair represents a shift in how advertising creative quality is operationalized (Lin et al., 18 Aug 2025). Instead of treating selection as a latent scoring task solved by ranking alone, it specifies a protocol in which model outputs can be inspected at intermediate steps, tied to human criteria, and validated against real CTR outcomes. This design aligns the dataset with contemporary interest in reasoned multimodal decision-making rather than purely predictive accuracy.

Its importance is also methodological. The dataset links three supervision layers that are often separated in creative AI research: behavioral labels from live systems, structured human judgments, and natural-language rationales (Lin et al., 18 Aug 2025). That combination is rare. Paired-data works in editing and generation often focus on transformation fidelity, while human-AI collaboration systems often focus on workflow or behavioral effects. CreativePair instead makes explanation itself part of the formal prediction target.

For advertising research, the dataset provides a concrete benchmark for explainable pre-deployment selection. For multimodal reasoning research, it offers a testbed in which chain-of-thought is grounded in explicit question-level judgments rather than unconstrained narration. For industrial creative tooling, it offers a route from pairwise preference prediction to actionable feedback on particular visual facets. These properties explain why CreativePair occupies a distinctive position within the broader family of pairing-based creative AI systems introduced across the recent literature (Lin et al., 18 Aug 2025).