Quantify the benefit of closed-loop XRL use

Determine how much of each explainable reinforcement learning method's diagnostic utility derives from allowing repeated, hypothesis-driven invocations with adjustable parameters rather than restricting the coder to one-shot explanation consumption.

Background

EvalXRL allows a coding agent to invoke an explainable reinforcement learning method repeatedly, adjust its parameters in response to intermediate results, and use the outputs to test successive hypotheses about a malfunctioning agent. The proposal adopts this closed-loop design because explanation and diagnosis are treated as dialogical and iterative processes.

The paper notes that prior empirical findings are mixed: iterative querying can improve downstream verification, while users may initially struggle to select useful parameters and non-interactive explanations can sometimes yield better initial hypotheses. The authors therefore identify the relative contribution of closed-loop interaction versus one-shot explanation use as an unresolved empirical issue and suggest varying the per-session invocation budget, down to a single invocation, to measure this effect.

References

We adopt closed-loop use as a design choice on conceptual grounds and treat its actual benefit as an empirical question. A natural extension is to vary the per-session invocation budget down to a single invocation in the limit, which would isolate how much of each method's utility comes from closed-loop use versus one-shot consumption.

— Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents  (2608.17524 - Rachum et al., 18 Aug 2026) in Section 5, “Open Design Questions,” paragraph “Closed-loop use is not unconditionally helpful”

We see no clean answer in advance: the measurement-cleanness vs. scaffold-capability tradeoff is exactly the kind of choice we expect community input to shape.

— Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents  (2608.17524 - Rachum et al., 18 Aug 2026) in Section 5, “Open Design Questions,” paragraph “Scaffold choice”