Develop multi-dimensional reward models for RL in native image generation

Develop reward models for reinforcement learning in native image generation that effectively capture and balance multiple dimensions, including image quality, instruction following, and alignment with human preferences, to enable reliable optimization of generation policies.

Background

Within the paper’s reinforcement learning section, the authors compare applying RL to the autoregressive versus diffusion components and emphasize that algorithmic choices (e.g., GRPO, Flow-GRPO) are less limiting than the design of reward signals. They categorize rewards into verifiable metrics (e.g., GenEval for object composition and OCR-based text rendering) and model-based preference metrics (e.g., PickScore, HPSv2.1, ImageReward), noting each captures only parts of what matters.

They conclude that the core unresolved issue is constructing reward models that jointly and faithfully assess image quality, adherence to textual instructions, and human preference. Solving this would better guide RL optimization and improve the reliability and alignment of native image generation models.