Integrating Multiple Confidence Signals for Self-Reward

Develop a unified self-reward mechanism for reinforcement learning on unlabeled data that integrates multiple complementary confidence signals to construct more reliable and fine-grained rewards for large language models.

Background

Existing confidence-based reward methods predominantly exploit only a single dimension of confidence, such as sequence likelihood, entropy minimization, or self-consistency consensus, which limits robustness and granularity of intrinsic supervision.

The paper highlights the need to integrate diverse confidence indicators—potentially including log-likelihood, entropy, decisiveness margins, and consensus—to form a more principled and reliable self-reward framework that can guide autonomous improvement without ground-truth labels.