Extending RL-ZVP to Tasks with Graded or Ambiguous Feedback

Extend Reinforcement Learning with Zero-Variance Prompts (RL-ZVP) beyond verifiable tasks with binary rewards to settings with graded or ambiguous feedback, including open-ended question answering, text summarization, and safety alignment.

Background

RL-ZVP is proposed as an extension of GRPO that extracts learning signals from zero-variance prompts, and the paper evaluates it on verifiable math reasoning tasks with binary correctness rewards.

The authors note that many real-world applications involve graded or ambiguous feedback (e.g., open-ended QA, summarization, safety alignment), and explicitly flag the extension of RL-ZVP to such settings as an open challenge.