Alternative reward computation methods for rubric-based RL in instruction following

Develop and evaluate alternative reward computation methods for rubric-based reinforcement learning in instruction-following tasks, such as weighted sums of per-criterion binary labels v_i produced by the rubric verifier, beyond the current all-or-nothing reward R(prompt, response, rubric) that grants success only when all criteria are satisfied.

Background

RIFL computes a sequence-level reward using rubric verification, with a simple all-or-nothing design that grants reward only when the response satisfies all rubric criteria. The authors conduct preliminary ablations on reward design and observe differences in performance across all-or-nothing, fractional, and hybrid rewards.

They explicitly state that exploring other reward computation methods, including weighted combinations of per-criterion labels, is deferred to future work, indicating an unresolved design space for how best to construct rubric-based rewards for RL training.

References

We leave other reward computation methods (for instance, weighted sum of v_i) for future work.

— Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following  (2511.10507 - He et al., 13 Nov 2025) in Section “Reward Design and Shaping”

This work primarily studies dimension-specific rewards. Combining multiple dimensions into a unified rubric reward or reward model remains an open challenge, including the choice of aggregation weights, the composition of preference data, and the training strategy. Developing and systematically evaluating such integration methods is a central direction for future work.

— LexReward: A Taxonomy-Driven Reward Framework for Legal Language Models  (2609.39071 - Cai et al., 30 Sep 2026) in Section C, Limitations

Therefore, deriving a training-free, per-criterion uncertainty weight that preserves the deterministic verifier decoding remains an open and critical challenge.

— Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation  (2610.21214 - Pan et al., 15 Oct 2026) in Section 3.1, “Motivation”