Identify the cause of missing reporting-intervention outputs

Identify whether generator errors, aborted collection, or lost output caused the unobserved targeted and matched reporting-intervention outputs in the held-out reporting experiment.

Background

The held-out reporting experiment compares natural, structured, targeted, and budget-matched summaries using report-based recovery of previously omitted task state. Eight outputs are missing: four targeted outputs and four matched outputs, corresponding to checkpoints from the Django task 15098 and SymPy task 22714.

The paper conditions its reported estimand on observed outputs and explicitly states that the mechanism producing the missing cells was not established. Resolving this uncertainty would help assess whether missingness could bias comparisons among reporting strategies.

References

The available held-out result tables establish the missing cells but do not record whether the cause was a generator error, aborted collection, or lost output.

— What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports  (2610.20003 - Xu et al., 14 Oct 2026) in Appendix, Section Reporting Interventions, subsection “Recovery and availability”