Residual-Difference Convergence for Posterior Concentration

Establish sufficient conditions under which the cross-trace residual differences required by the Posterior Concentration Phenomenon converge, thereby providing a theoretical foundation for characterizing when posterior rewards become less discriminative across reasoning traces.

Background

The paper's theoretical characterization of the Posterior Concentration Phenomenon assumes that the difference between concentration residuals associated with reasoning traces of nearby lengths converges. This assumption supports the result that posterior probabilities for independently sampled traces of similar lengths become increasingly similar as the considered reasoning-length range grows.

The paper explicitly identifies the conditions guaranteeing this residual-difference convergence as unresolved and outside its scope. Establishing such conditions would strengthen the theoretical basis of posterior concentration analysis and could support finite-length guarantees for the loss of discriminative power in posterior rewards.

References

Connecting these findings to posterior rewards offers a useful perspective on verifier-free RL, although establishing when the required residual differences converge remains an open theoretical question beyond our scope.

— Rethinking Probability-Based Reinforcement Learning From Posterior Concentration  (2610.01458 - Kao et al., 1 Oct 2026) in Appendix, Section 6, “Discussion about PCP's Residual Condition”