Determine whether archived runs used the documented scoring source

Determine whether the archived Instruction-Conditioned Motion Control runs executed the documented source code for the placement checker and success-label fallback, thereby establishing the provenance and interpretation of their recorded rewards.

Background

The paper reports that the current and single-epoch runners accept output containing “SUCCESS” and no “FAIL,” but the retained archive does not identify the scoring-source version used for each run. Because the reward checker shared the generated program’s output stream, this provenance uncertainty affects the interpretation of logged success labels.

Resolving this issue is necessary for determining whether the archived labels reflect the intended geometric success criterion or merely text emitted by the generated program.

References

The single-epoch runner used in the learning-rate and epoch comparison, like the current source of the main runner, falls back to accepting any output that contains SUCCESS and noFAIL; whether the archived runs executed that source is not established.

— Instruction-Conditioned Motion Control for Robotic Arms: Language-to-Code Decomposition and Reinforcement Learning  (2610.19617 - Huang et al., 14 Oct 2026) in Section IV-B, Evaluation protocol and interpretation of logged rewards, p. 4

The public page does not provide a full record of certificates or language-model calls from the organizers' run. We therefore do not know whether that run used the optional language-model fallback.

— Rózsa: Certified Equational Reasoning  (2610.26467 - Kissas, 19 Oct 2026) in Section 5.1, paragraph “Official result”