One Size, Many Fits: Aligning Diverse Group-Wise Click Preferences in Large-Scale Advertising Image Generation

Abstract: Advertising image generation has increasingly focused on online metrics like Click-Through Rate (CTR), yet existing approaches adopt a ``one-size-fits-all" strategy that optimizes for overall CTR while neglecting preference diversity among user groups. This leads to suboptimal performance for specific groups, limiting targeted marketing effectiveness. To bridge this gap, we present \textit{One Size, Many Fits} (OSMF), a unified framework that aligns diverse group-wise click preferences in large-scale advertising image generation. OSMF begins with product-aware adaptive grouping, which dynamically organizes users based on their attributes and product characteristics, representing each group with rich collective preference features. Building on these groups, preference-conditioned image generation employs a Group-aware Multimodal LLM (G-MLLM) to generate tailored images for each group. The G-MLLM is pre-trained to simultaneously comprehend group features and generate advertising images. Subsequently, we fine-tune the G-MLLM using our proposed Group-DPO for group-wise preference alignment, which effectively enhances each group's CTR on the generated images. To further advance this field, we introduce the Grouped Advertising Image Preference Dataset (GAIP), the first large-scale public dataset of group-wise image preferences, including around 600K groups built from 40M users. Extensive experiments demonstrate that our framework achieves the state-of-the-art performance in both offline and online settings. Our code and datasets will be released at https://github.com/JD-GenX/OSMF.

• The paper presents OSMF, a unified framework that aligns advertising image generation with diverse, group-specific click preferences.
• It integrates Product-Aware Adaptive Grouping and Preference-Conditioned Image Generation, using Group-DPO fine-tuning to capture nuanced user behaviors.
• Experimental results on the GAIP dataset show improved NDCG, AUROC, and a 5.5% CTR boost, demonstrating practical impact in large-scale advertising.

One Size, Many Fits: Aligning Diverse Group-Wise Click Preferences in Large-Scale Advertising Image Generation

Introduction

Advertising image generation has rapidly evolved towards data-driven approaches, with models increasingly optimized via online engagement metrics such as Click-Through Rate (CTR). Despite advances in integrating reinforcement learning from human feedback (RLHF), most existing works optimize for a unified, aggregate CTR, implicitly assuming user homogeneity. This paradigm disregards the heterogeneity of user preferences, resulting in generated assets that often perform suboptimally for substantial user segments and hindering precise targeting in large-scale advertising.

The paper "One Size, Many Fits: Aligning Diverse Group-Wise Click Preferences in Large-Scale Advertising Image Generation" (2602.02033) directly addresses these limitations. It introduces OSMF, a unified framework designed to align generated advertising images with diverse, group-specific click preferences, and demonstrates state-of-the-art group-wise personalization capabilities at industrial scale.

Figure 1: Distinct group-specific preference conflicts in CTR for smartphone (gender-based) and shoe (age-based) advertising.

Framework Overview: OSMF Pipeline

The OSMF framework comprises two principal modules:

1. Product-Aware Adaptive Grouping (PAAG): A dynamic clustering strategy that generates user groups based on both user attributes and product characteristics. Unlike static segmentation, PAAG learns multi-faceted group representations reflecting actual, fine-grained user-product interactions, and adaptively determines cluster granularity for each product.
2. Preference-Conditioned Image Generation (PCIG): A generative process that employs a Group-aware Multimodal LLM (G-MLLM), augmented with group structure, to generate images conditioned on group-level preferences. Fine-tuning is performed using the Group Direct Preference Optimization (Group-DPO) objective, aligning synthesized images with the distinct click behaviors of each group.

   Figure 2: Schematic of OSMF: (a) PAAG adaptively clusters users into coherent preference groups; (b) PCIG generates images per group using the Group-DPO optimization protocol.

Methodology

Product-Aware Adaptive Grouping (PAAG)

PAAG overcomes the rigidity of static segmentation by combining user demographic/behavioral attributes and product semantic features in a multi-modal embedding space. It employs cross-attention to construct product-conditioned user representations, which are then subject to clustering (K-Means with silhouette heuristics for $K$ selection) to yield groupings that reflect actual, per-product preference structure.

To faithfully capture intra-group diversity beyond centroids, group features are further aggregated by sampling representational percentiles within clusters, ensuring the model encodes both core and peripheral group traits.

Preference-Conditioned Image Generation (PCIG)

PCIG incorporates group-level signals through three main stages:

• Group-Aware MLLM Input Augmentation: Group representations are injected as explicit input tokens to the backbone MLLM, facilitating contextually-aware prompt and image synthesis.
• Specialized Pre-Training Tasks: The G-MLLM is primed on group- and product-centric tasks, such as group characterization, behavioral prediction, and prompt generation, thereby enabling both user understanding and content creation aligned with nuanced group features.
• Group-DPO Fine-Tuning: A reward model, GRM, estimates group-conditional CTR from image pairs, and the G-MLLM is fine-tuned with an extension of Direct Preference Optimization (DPO), maximizing the likelihood of prompts/images preferred by each target group.

Grouped Advertising Image Preference Dataset (GAIP)

The paper introduces GAIP, a novel large-scale dataset for group-wise image preference modeling derived from anonymized real-world advertising logs encompassing 40M users, 600K user groups, and 9.6M images. GAIP provides interaction data at the group level (CTR by group per image), enabling research on intra- and inter-group preference divergence previously impossible with existing datasets.

Experimental Results

Metrics and Baselines

Group-wise aggregation is evaluated with NDCG@5 (lower is better, more preference divergence between groups) and AUROC for click prediction accuracy. For image generation, PCIG is compared with current CTR-driven pipelines, and the quality of the reward modeling (GRM) is assessed via Pair Accuracy in predicting group-wise CTR orderings.

Numerical Results:

• PAAG achieves NDCG@5 = 0.3066 (vs. previous best 0.3124) and AUROC = 0.6372, indicating both increased divergence in group preferences and superior click modeling fidelity.
• In live online A/B testing on 10K high-exposure products, OSMF achieves CTR = 0.0154, corresponding to a +5.5% improvement over a pretrained G-MLLM and a significant margin over both general and CTR-optimized baselines.
• The reward model GRM yields a 4.7% absolute improvement in Pair Accuracy versus CTR-focused SOTA models.

Qualitative analysis further shows that OSMF/PCIG generates visually diverse images tailored to distinct group tastes, covering style, color, and content preferences for categories as disparate as cosmetics, electronics, fashion, and daily necessities.

Figure 4: Illustrative examples generated by OSMF, demonstrating group-tailored diversity across multiple product verticals.

Theoretical and Practical Implications

The study empirically and formally substantiates that optimizing only for aggregate CTR not only suppresses divergent preferences but can degrade overall commercial effectiveness when user heterogeneity is high (see Figure 1). The adaptive, group-level RLHF approach demonstrated here provides a scalable alternative to fine-grained one-to-one personalization, enjoying a practical trade-off between performance and resource requirements. The use of representation sampling within-group features sets a precedent for mitigating centroid-bias in segmentation methods, and the extension of DPO to a group-conditioned objective enables robust multi-group alignment even when inter-group preferences are in direct conflict.

Practically, deployment of OSMF can drive significantly higher ad engagement and, by consequence, revenue in e-commerce and advertising. The release of GAIP further supports reproducibility and benchmarking in this domain.

Future Directions

Scalability and latency constraints of the adaptive grouping and reward modeling process remain, particularly for real-time applications. Extensions could explore more efficient clustering methods, streaming group updates, and integration of textual overlays or more complex creative constraints. The methodology is also transferable to other RLHF alignment domains with heterogeneous or conflicting user objectives, such as personalized recommendation, dialogue systems, or content moderation.

Conclusion

OSMF presents a rigorous, scalable framework for aligning advertising image generation with heterogeneous group-wise preferences, demonstrating marked improvements in both offline metrics and live CTR. The formalization of adaptive grouping, preference-conditioned generation, group-DPO optimization, and the introduction of the GAIP dataset collectively establish a new paradigm in group-aware, RLHF-driven generative modeling for large-scale online advertising (2602.02033).