Evaluate recovery from naturally occurring failures

Evaluate whether Outcome Monitors enable agents to recover from naturally occurring silent tool failures in live or organic tool-use settings, rather than only from experimentally injected failures.

Background

The paper evaluates recovery primarily through fault-enriched experiments in which silent failures are injected into ToolMaze, AppWorld, and τ-bench. Although the supplement measures detector firing on recorded StableToolBench responses, that organic-traffic analysis contains no agent, receipt, or counterfactual and therefore does not assess recovery. The authors explicitly identify recovery from organic failures as unresolved.