Resolve uncertainty about late-stage critic gains for long responses

Establish whether the apparent late-position value-prediction improvement from solution-conditioned critics is statistically supported for long student responses, rather than being an artifact of the pronounced early prediction cost.

Background

The paper analyzes how conditioning the critic on reference solutions affects prediction error at early, middle, and late positions in generated responses. The aggregate pattern suggests an early cost and a later benefit, especially for incorrect source responses.

However, after stratifying by response length, the authors report that the long-response late-gain interval includes zero. They explicitly state that the positive positional contrast may instead reflect the early cost, leaving the existence of a genuine late gain in long responses unresolved.