Determine whether the residual protocol difference is absent after successful parsing

Determine whether a residual performance difference remains between ReAct and function calling for Qwen2.5-Coder-7B after conditioning on items that both protocols parse successfully.

Background

For Qwen2.5-Coder-7B, conditioning on successful parsing produced a residual ReAct–function-calling difference of 8.4 percentage points, but the estimate was statistically inconclusive and had a wide confidence interval. Because successful parsing is determined by the interface under study, the conditioned comparison is also post-treatment and cannot by itself establish equivalence.

References

We do not detect a residual difference, which is not the same as establishing there is none.

Interface-Induced Trajectory Censoring  (2609.03966 - Wang, 3 Sep 2026) in Section 3.7, subsection “Repair loop: how much comes back”; Section 4, Limitations, item 9