Dropped-grounding fault localization

Determine a method that achieves a substantial and stable gain over the appropriate candidate-pool floor when localizing dropped-grounding faults in the CatchBench-Gold dependency-graph benchmark.

Background

CatchBench-Gold evaluates localization of two injected dependency-state faults: stale-state reads and dropped grounding. Dropped grounding removes one required dependency from a step, leaving that step ungrounded. On the eligibility-matched candidate pool, the reported methods do not produce a substantial and stable improvement over the random floor; the span-based and degree-based detectors remain at or below the relevant baseline, and the generic graph detector’s apparent lift is too small to establish reliable localization.

The authors explicitly classify dropped-grounding localization as unresolved rather than treating the diagnostic results as evidence of a solved problem. Because the Gold substrate is itself artifact-limited and remains a mechanism diagnostic, progress must also be evaluated on a defensible substrate before the result can carry evidential benchmark weight.

References

Dropped grounding remains open because no reported method shows a substantial and stable gain over the appropriate candidate-pool floor.

CatchBench: When Can an Agent Failure Be Caught?  (2608.22808 - Zhao, 24 Aug 2026) in Section 6, Discussion and Limitations, paragraph “Measured Headroom Locates Three Open Problems”; see also Section 10, subsection “Gold Board Mechanism Diagnostics,” paragraph “Stale State Localizes; Dropped Grounding Does Not.”