Compare reward estimators at equal computational cost

Compare the M=5 and M=1 downstream reward estimators under equal computational budgets to determine the effect of reward averaging independently of differences in downstream sampling cost.

Background

The study compares averaging five downstream program rewards per instruction with using one program reward per instruction. However, the two runs use different downstream sampling costs, so the observed differences cannot be attributed solely to the reward estimator.

A controlled equal-compute comparison is required to isolate the effect of reward averaging from the amount of downstream program sampling.

References

The M=5 and M=1 runs compare reward estimators at different downstream sampling costs; equal-compute comparisons remain future work.

— Instruction-Conditioned Motion Control for Robotic Arms: Language-to-Code Decomposition and Reinforcement Learning  (2610.19617 - Huang et al., 14 Oct 2026) in Section VI, Training through a frozen code generator, p. 6