Determine the source of the remaining logic and mathematics performance gap

Identify the causes of the remaining performance gap between the Athanor team’s post-trained Qwen3-4B-Base submission and Qwen3-4B-Instruct-2507 on logic and mathematics, beyond answer truncation at the output limit.

Background

The team’s submission remains substantially below Qwen3-4B-Instruct-2507 on logic and mathematics even after the output limit is increased from 16,384 to 32,768 tokens and sampled decoding is used. The paper establishes that truncation explains part, but not all, of the difference.

The authors explicitly state that the remaining causes have not been identified, leaving the source of the residual performance gap unresolved.

References

Raising the output limit does not recover these answers. In the final measurement, with an output limit of 32,768 tokens and sampled decoding, the team's submission's logic score changes by less than one point (63.4 to 63.0), and the four final submissions remain 15--19 points below Instruct-2507 on logic (\cref{tab:v4-tasks}). On mathematics, the answers that finish are at least as accurate as Instruct-2507's, so under the campaign protocol much of the gap shows up as truncation; that the gap persists with the longer limit (38.4 against 44.2) shows that truncation is not all of it, and we have not identified the rest.

— Athanor: An Agent-Native System for Autonomous Post-Training Research  (2610.21055 - Zhou et al., 15 Oct 2026) in Appendix, Section “Why the Submission Trails on Logic and Mathematics”