Online stale-state detection

Develop an online detector that reliably identifies stale-state reads in causal trace prefixes at a fixed false-alarm budget, outperforming the signal-free dependency-count control.

Background

The online stale-state task asks a detector to flag a run during execution using only causal prefix evidence. The benchmark’s causal dependency-span z-score performs only slightly above or at the signal-free control: at a realized 6.1% false-positive rate it catches 6.1% of stale reads, equal to the dependency-count control, while raw span has a higher point estimate. Across five injection seeds, the causal detector remains weak.

The authors therefore identify online stale-state detection as an open problem. The corresponding Gold data are artifact-limited mechanism diagnostics rather than evidential benchmark data, so a successful solution would need both stronger online signal extraction and evaluation on a validated substrate.

References

Online stale-state detection is the second open problem.

CatchBench: When Can an Agent Failure Be Caught?  (2608.22808 - Zhao, 24 Aug 2026) in Section 6, Discussion and Limitations, paragraph “Measured Headroom Locates Three Open Problems”; see also Section 10, subsection “Online Stale-State Detection.”