Causes and mitigation of train–evaluation gap under high reset probabilities

Ascertain the underlying causes of the observed train–evaluation performance gap at high frontier reset probabilities in Frontier Checkpointing and investigate whether randomizing checkpoint states across procedurally generated layouts recovers the benefits of high reset without incurring overfitting.

Background

Frontier Checkpointing accelerates training by resetting to saved frontier states; however, very high reset probabilities produce a notable train–evaluation gap, suggesting overfitting to specific checkpoint configurations.

The authors explicitly state they cannot definitively rule out alternative explanations for this gap and propose investigating diversity interventions—such as randomizing checkpoints across different layouts—to determine if the benefits of high reset can be retained without overfitting.

References

While the train-eval gap at high reset probabilities shown in Figure~\ref{fig:eval-train-gap} suggests overfitting to checkpoint configurations, we cannot definitively rule out alternative explanations. Future work should investigate whether diversity interventions such as randomizing checkpoints across procedurally generated layouts can recover the benefits of high reset without the overfitting cost.

— SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Grounding  (2603.09036 - Zabounidis et al., 10 Mar 2026) in Appendix, Section "Limitations and Future Work"

We did not investigate why the two outcomes differ for the sham specifically. The probe resets its random stream and evaluates 16 agents per island for 8 lifetimes in a fixed sequence of worlds, while the in-run number is the population's fitness in the worlds it was selected in; a condition whose advantage is partly specific to the worlds it evolved in would show the two diverging, but we have not tested that here.

— Null-model treatment of the sensory-motor boundary changes an evolutionary connectome comparison  (2609.39248 - Park, 30 Sep 2026) in Appendix, Section “Intervention and dose-response: further details,” subsection “The sham equivalence test on an unregistered outcome”