Develop practical soft-grouping or hierarchical alternatives for summary-reward normalization

Develop soft-grouping or hierarchical alternatives to explicit subgroup partitioning for normalizing summary rewards within trajectories that share the same tool outcome, while avoiding the exponential reduction in effective rollout-group size caused by multi-tool-call settings.

Background

The paper explains that SLCA normalizes summary rewards across an entire rollout group, even when trajectories diverge after the tool segment and therefore correspond to different post-tool states. This creates grouping bias because summary quality may be conflated with variation in the state produced by tool execution.

The paper notes that explicit subgrouping by tool outcome would reduce the effective group size from G to G/C, and for K tool calls potentially to G/CK. Because this scaling makes explicit partitioning impractical without increasing the rollout-group size, the authors identify soft-grouping or hierarchical alternatives as unresolved directions.

References

This is a design trade-off; soft-grouping or hierarchical alternatives remain open.

— SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL  (2609.29050 - Zhan et al., 24 Sep 2026) in Appendix A, Section “Theoretical Analysis: Proofs of Main Properties,” paragraph “Bias–Variance Trade-off of the Summary Advantage Estimator”