PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

Abstract: Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small held-out set. Almost all effort has gone into the proposer that generates candidates; we argue the weak point is the acceptor, the rule that decides whether to commit a change. Applied hundreds of times against the same noisy dev estimate, the ubiquitous "keep it if the score went up" rule is uncontrolled adaptive multiple testing: the agent effectively p-hacks itself, accumulating false commits that make it churn and drift rather than improve. We recast committing as a sequential hypothesis test and propose PACE (Paired Anytime-valid Commit Evaluation), a training-free, anytime-valid commit gate. Each candidate is compared to the incumbent on identical instances and committed only when a testing-by-betting e-process accumulates decisive evidence, stopping early to save evaluations and controlling each candidate's false-commit probability at a user-set level even under optional stopping (a per-decision guarantee). On Qwen2.5 agents (0.5B-3B) self-evolving at the prompt level on GSM8K, SVAMP, and ARC-Challenge, greedy acceptance commits 30-42% false and 10-33% harmful edits when a genuine improvement is hidden among noisy proposals, while PACE commits the real one and essentially nothing else, matching greedy's held-out accuracy at sharply lower variance and about 18% lower evaluation cost. With no real gain available, greedy commits 13-21 spurious self-modifications per run (72-100% false) and degrades the most fragile agent by 4.9 points, while PACE holds at baseline. Reliability of self-evolution depends on the acceptor, not only on the proposer.

• The paper presents a sequential hypothesis testing method using a betting-based e-process to validate candidate modifications in evolving agents.
• It demonstrates that PACE achieves 0% false commits in controlled tests, with significantly lower variance and an 18% reduction in evaluation cost compared to greedy methods.
• The approach provides anytime-valid statistical guarantees, ensuring stable self-improvement and reducing performance drift in autonomous agents.

PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents

Background and Motivation

The paper "PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents" (2606.08106) addresses the failure mode in self-evolving agent architectures where the modification acceptance mechanism is governed by an unexamined heuristic: typically, retaining a change if a small held-out set yields an improved score. This approach constitutes uncontrolled adaptive multiple testing on reused noisy estimates, resulting in chronic false acceptance of modifications. The agent is prone to "p-hacking," accumulating changes that by chance appear beneficial but ultimately degrade true performance. Despite advancements in proposer methodologies—such as prompt rewriting, skill induction, and workflow restructuring—the acceptor remains neglected, limiting reliable agent self-improvement.

PACE: Methodology

PACE (Paired Anytime-valid Commit Evaluation) reframes the acceptance decision as a sequential statistical hypothesis test rather than a naive empirical comparison. Candidate modifications are evaluated against the incumbent configuration on identical instances via a paired test, akin to McNemar, isolating discordant pairs (cases where candidate and incumbent diverge in correctness). The core innovation is using a testing-by-betting e-process that incrementally accumulates evidence of superiority, governed by Ville's inequality and martingale properties. Acceptance occurs when the betting process's wealth exceeds $1/\alpha$, enforcing a user-defined false-commit probability $\alpha$ per candidate, valid at any stopping point (anytime-validity). This method requires no retraining, is proposer-agnostic, and interfaces simply through paired correctness outcomes.

Empirical Evaluation

Experiments are conducted on Qwen2.5-Instruct agents (0.5B–3B parameters) evolving prompts for three tasks: GSM8K, SVAMP, and ARC-Challenge. Evaluation paradigms include:

• Controlled Regime: A known harmful instruction is embedded, with the proposer tasked to discover its removal amid noise. PACE consistently commits only the genuine gain (0/5 false commits), matching greedy's accuracy but with significantly reduced variance and ~18% lower evaluation cost.
• Stochastic Regime: The agent is sampled at temperature 0.7, with no real improvement available. Greedy commits 13-21 self-modifications per run (up to 100% false), causing up to 4.9 points degradation in fragile agents, while PACE rejects all noise-driven modifications and preserves baseline accuracy.

Sensitivity sweeps across $\alpha$ and dev set size $n$ reveal PACE's robustness. Across a 10x range in $\alpha$ and a 4x range in $n$, the gate retains the real improvement at 0% false commits, whereas greedy consistently accumulates 33-53% false accepts. Additional tests on SVAMP and ARC-Challenge reaffirm the generality: PACE maintains its precision and screening power across domains and model scales.

Implications and Theoretical Consequences

PACE delivers a formal per-candidate guarantee on false-commit probability, correcting the critical statistical pathology of adaptive multiple testing in self-evolution loops. This stands in contrast to the run-level familywise error, with implications for loop stability—false-acceptance suppression directly translates to lower churn and performance drift. The method is proposer- and loop-agnostic, only requiring outcome pairs, and is therefore applicable to broader self-modification regimes (prompt, code, skills, workflows), though full cross-system validation remains prospective.

Practically, PACE offers heightened reliability for autonomous agents under resource-constrained, noisy evaluation, and its incremental testing decreases evaluation cost without sacrificing precision. The trade-off is reduced recall for marginal improvements—a consequence dictated by anytime-validity and evidence strength, which is tunable via $\alpha$ and evaluation budget. The method's limitations relate to evaluation coverage: improvements unseen by the dev set cannot be certified.

Future Directions

PACE opens avenues for integrating anytime-valid statistical gates into broader self-modifying agents, especially in settings where incrementality and automation necessitate rigorous control of hypothesis-testing error. Extensions may explore richer candidate evaluation metrics, broader proposers, and alternative e-process calibrations for balanced recall/precision. System-level generalization to complex agent topologies and external validation in third-party frameworks is a key area for subsequent work.

Conclusion

PACE offers a principled solution to the statistical weakness in the self-evolution loop of autonomous agents, enforcing an anytime-valid per-candidate false-commit bound and empirically suppressing spurious and harmful acceptance events. Its simplicity, reliability, and generalizability make it an essential tool for stable agent self-improvement, highlighting the critical importance of rigorous acceptance decision-making alongside proposer innovation.