Disentangle target heterogeneity from weak reflection supervision

Determine whether the observed length collapse and below-baseline reflection fix rate in reflection-heavy, canonical-target single-adapter co-evolution arise primarily from heterogeneous-target drift, weak reflection-target construction, or both.

Background

The paper studies a single-adapter co-evolution design in which GRPO code-generation updates are combined with auxiliary pairwise-comparison and reflection supervision. The reflection objective dominates the auxiliary mixture and uses canonical solutions or correct rollouts rather than verified self-fix trajectories.

The experiments show substantial policy drift, output-length collapse, and a reflection fix rate below the pretrained baseline. However, because the auxiliary targets are heterogeneous and the reflection data are constructed from canonical or correct-rollout targets, the paper does not establish which factor is responsible for the observed failure. Resolving this issue would clarify whether the limitation is caused by the shared-adapter architecture, inadequate reflection supervision, or their interaction.

References

The length collapse and below-baseline reflection fix rate are therefore partially entangled: we cannot fully separate heterogeneous-target drift from weak reflection construction.

— Co-Evolving Policy and Reward for Code Generation: A Mechanistic Study of Single-Adapter Drift  (2610.26416 - Batjargal et al., 19 Oct 2026) in Discussion and Conclusion, paragraph beginning “A caveat applies to how strongly this speaks to co-evolution in general.”