GenCO: Optimizing E-commerce Ad Creatives

• GenCO is a framework for optimizing e-commerce ad creatives by integrating generative modeling with multi-instance reinforcement learning to address combinatorial challenges.
• The methodology employs a two-stage process where the first stage generates creative combinations and the second stage uses multi-instance learning to attribute sparse rewards.
• Industrial deployment demonstrated significant revenue gains, scalability in large design spaces, and robust adaptation to evolving user feedback.

GenCO

GenCO (Generative Modeling with Multi-Instance Reward Learning for E-commerce Creative Optimization) defines a two-stage framework for optimizing e-commerce advertising creatives, explicitly addressing the combinatorial explosion inherent in selecting and evaluating multi-component creative objects such as titles, images, and highlights. The architecture integrates generative modeling optimized via reinforcement learning (RL) with a multi-instance reward attribution mechanism, enabling effective exploration of the action space and robust learning from sparse interaction feedback (Gu et al., 13 Aug 2025).

1. Problem Setting and Motivation

In e-commerce display advertising, the effectiveness of an advertisement depends on the joint selection of creative elements—product images, textual highlights, and titles. The conventional paradigm evaluates creative components individually, which does not scale due to the exponential combinatorial space: if there are $n_1$ titles, $n_2$ images, and $n_3$ highlights, the search space is $n_1 \times n_2 \times n_3$. User feedback (e.g., click-through rates) is only observed at the level of full creative combinations, and typical data sparsity further impedes direct estimation of componentwise effects.

GenCO addresses this by:

• Jointly generating full creative combinations using a generative model.
• Optimizing the generator using RL to efficiently search and adapt.
• Employing multi-instance reward learning to accurately attribute observed combination-level outcomes to individual creative elements and their interactions.

This unified framework enables practical and scalable creative optimization, which is demonstrated to generate significant advertising revenue gains in real-world deployment (Gu et al., 13 Aug 2025).

2. Two-Stage GenCO Architecture

GenCO separates the optimization process into two interdependent stages:

Stage 1: Generative Modeling with RL Optimization

• A generative model (parametric, such as a transformer or VAE) learns a distribution over creative combinations.
• The generator samples combinations and receives reward signals to adapt its policy.
• Reinforcement learning is used to optimize the generative process due to the combinatorial and nondifferentiable nature of online engagement metrics (e.g., click-through rate as the reward function).
• Exploitation-exploration tradeoffs are handled via RL, enabling the generator to refine high-performing combinations while continuing to explore underexplored regions.

Stage 2: Multi-Instance Reward Attribution

• Observed feedback is sparse and only available at the combination level (i.e., when a user clicks an ad, there is no direct signal for each element).
• Multi-instance learning (MIL) is used to decompose the combination-level reward, attributing it probabilistically to individual creative elements or their interactions.
• The derived attribution allows the reward estimator to provide denser, less noisy feedback to the generator, improving reward signal quality and training dynamics.

This closed-loop system enables end-to-end optimization, where the generator and reward model are iteratively improved based on more accurate attributions from sparse data.

3. Generative Modeling and RL Formulation

The generative model $G_\theta$ produces candidate creative combinations $c = (t, i, h)$ given input noise or context. The policy is updated by maximizing expected reward:

$$\max_\theta \ \mathbb{E}_{c \sim G_\theta} [ R(c) ]$$

where $R(c)$ is an (estimated) reward signal attributed to the combination, typically derived from user engagement logs. The reward landscape is high-variance and typically sparse; RL methods (e.g., policy gradients or actor-critic variants) are employed to optimize the non-differentiable reward. The generative process is tuned to encourage both diversity (avoiding mode collapse to a few high-rewarded templates) and high average reward.

Exploration is crucial: standard RL techniques, such as entropy regularization or ε-greedy sampling, are incorporated to ensure sufficient coverage of the combinatorial space.

4. Multi-Instance Learning for Reward Attribution

Observed clicks or conversions in practice are weak labels for the underlying components:

• Each combination is a bag of instances (elements).
• The reward model must infer which components and interactions contributed to the observed outcome.

GenCO applies a multi-instance learning framework, modeling the observed reward as a function $f$ over the set of included elements:

$R(c) = f(t, i, h) + \epsilon$

MIL methods (such as attention-based pooling or instance-wise parameterization) are used to decompose $n_2$0, learning to allocate credit to the constituent elements by statistically disambiguating component contributions across the dataset. This is critical for overcoming the credit assignment problem endemic to multi-component stochastic optimization in sparse feedback regimes.

The improved reward signal is then propagated back to the generative stage, closing the loop and enabling efficient end-to-end credit assignment.

5. Industrial Deployment and Empirical Outcomes

GenCO was deployed at scale on a major e-commerce platform. Key outcomes included:

• Significant increase in advertising revenue compared to conventional heuristic or single-element optimization methods.
• The approach demonstrated scalability to extremely large design spaces, where naive enumeration or componentwise optimization would be computationally infeasible.
• The authors released a large-scale industrial dataset, supporting further research.

Empirical analysis in deployment revealed improved click-through and conversion rates, attributable directly to the multi-instance reward learning feedback and RL-optimized diversity of the generative model. The approach adapts over time to shifts in user preferences and trends, driven by online reward feedback (Gu et al., 13 Aug 2025).

6. Relation to Prior Art and Research Directions

GenCO bridges gaps between generative modeling, reinforcement learning, and multi-instance/multi-label learning for combinatorial recommendation and advertisement design. Prior attempts at e-commerce creative optimization typically used either heuristic combinatorial bandits, which do not scale with interaction terms, or deep learning with insufficient attribution granularity. GenCO's integration of deep generative models with explicit multi-instance reward learning is a distinguishing factor.

Plausible implications are:

• Extensions beyond e-commerce creatives, to general combinatorial recommendation tasks (e.g., menu optimization, configuration of bundle offers, multi-modal content generation).
• Potential adoption for other tasks with sparse, delayed, and combinatorial-structured reward signals, where MIL credit assignment can drive generative model adaptation.

Current limitations include sensitivity to model mis-specification in the multi-instance learning layer and potential scalability issues for extremely high-dimensional element attribute spaces.

7. Dataset Release and Reproducibility

A large-scale industrial dataset comprising creative combinations and corresponding user engagement logs has been released. This enables benchmarking and extension of GenCO and related methods in combinatorial generative modeling under sparse/partial feedback settings. Details of experimental setup, evaluation metrics, and ablation studies are provided to ensure reproducibility and facilitate comparative research (Gu et al., 13 Aug 2025).