AgentDropoutV2: Test-Time Pruning for MAS

• The paper demonstrates that applying test-time interception and iterative rectification in MAS improves accuracy by 6.3 percentage points on math benchmarks.
• The framework uses a retrieval-augmented rectifier to dynamically evaluate and correct agent outputs based on failure-driven indicators mined from historical errors.
• The method balances local message pruning with a global fallback mechanism to maintain system integrity and prevent cascading errors in complex reasoning tasks.

Searching arXiv for the target paper and closely related work to ground citations. AgentDropoutV2 is a test-time rectify-or-reject pruning framework for Multi-Agent Systems (MAS) that is designed to optimize information flow without retraining. It acts as an active firewall: each agent’s outgoing message is intercepted before broadcast, evaluated against retrieved indicators distilled from historical failures, iteratively rectified when feasible, and pruned when errors persist. The framework is motivated by the cascading impact of erroneous intermediate outputs in MAS, particularly in math and code tasks where downstream reasoning depends on the correctness and consistency of prior steps. In the reported experiments, AgentDropoutV2 achieves an average accuracy gain of 6.3 percentage points on math benchmarks, while also exhibiting robust generalization and adaptivity (Wang et al., 26 Feb 2026).

1. Problem setting and design objective

In standard MAS, agents produce intermediate outputs that are broadcast to successors according to the system’s topology. A single erroneous output can contaminate the shared knowledge state and mislead downstream agents, causing cascading degradation. The paper emphasizes that this failure mode is especially acute in math and code tasks, where intermediate reasoning steps must be correct and consistent (Wang et al., 26 Feb 2026).

The method is positioned against two principal prior paradigms. The first is structural optimization, including optimized DAGs, sparse communication, and learned routing. These approaches can reduce the chance that errors spread widely, but the topology remains static and cannot salvage correctable errors at test time. The second is parameter internalization, including fine-tuning on failure trajectories or process supervision such as verifiers and step-wise supervision. These methods improve intrinsic reasoning, but weights are frozen at deployment and cannot adaptively correct context-specific mistakes at test time.

AgentDropoutV2 is introduced specifically to address these limitations. Its stated goals are to intercept agent outputs before dissemination, test them against context-relevant adversarial indicators mined from failure trajectories, iteratively rectify outputs when indicators flag correctable issues, prune outputs only when errors persist, and preserve system integrity through a global fallback that maintains sufficient message “critical mass.” This operational framing distinguishes it from Classic AgentDropout (V1), which permanently removes low-performing agents based on training-time statistics and does not attempt output correction.

A plausible implication is that AgentDropoutV2 treats reliability as a test-time control problem over message flow rather than solely as an offline optimization problem over topology or weights. In the terminology of the paper, the mechanism functions as an “active firewall” for MAS.

2. Core mechanism: active firewall and tri-state gating

The framework intercepts each agent’s output before it is broadcast. A rectifier model evaluates the output against a small set of retrieved indicators targeting likely error types given the current context. This produces a tri-state decision process: pass, retry, or reject (Wang et al., 26 Feb 2026).

The pass condition applies when all checks are satisfied. The output is accepted immediately and broadcast to successors. The retry condition applies when at least one indicator signals a violation and the iteration budget has not been exhausted; in that case, the output is regenerated with targeted feedback. The reject condition applies when the output still violates one or more indicators at the maximum iteration budget, in which case the message is pruned to prevent error propagation.

The paper formalizes MAS as

$$\mathcal{S} = (A_1, A_2, \dots, A_N), \quad A_i = (P_i, R_i, K_i),$$

with agent output

$o_i = P_i(x_i, R_i, K_i).$

Information flow is represented by

$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$

and control flow by

$$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$

The rectify-or-reject stage introduces an iterative sequence of candidate outputs $o_i^{(t)}$. For each iteration, the rectifier evaluates the output under retrieved indicators and defines a global error state:

$$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$

The zero-tolerance rule is explicit: if any retrieved indicator flags a violation, then $E^{(t)} = 1$. Iterative rectification proceeds as

$$o_i^{(t+1)} = P_i \big( x_i, R_i, K_i, F^{(t)} \big), \quad t=0,1,\dots,T_{\max}-1.$$

The final broadcast message is

$$o_i = \begin{cases} o_i^{(t)} & \exists t < T_{\max} \text{ s.t. } E^{(t)} = 0, \ \varnothing & \text{otherwise}. \end{cases}$$

The paper explicitly notes that no additional score $s(x)$ or threshold $o_i = P_i(x_i, R_i, K_i).$0 is defined; the decision rule is the zero-tolerance global error state combined with a bounded iteration budget. This yields a discrete gating mechanism rather than a continuous confidence-thresholding scheme.

3. Retrieval-augmented rectifier and failure-driven indicators

The central supervision signal in AgentDropoutV2 is the indicator pool $o_i = P_i(x_i, R_i, K_i).$1, constructed offline from MAS failure trajectories and used online through semantic retrieval. Each indicator $o_i = P_i(x_i, R_i, K_i).$2 contains a name $o_i = P_i(x_i, R_i, K_i).$3, an error definition $o_i = P_i(x_i, R_i, K_i).$4, and a trigger condition $o_i = P_i(x_i, R_i, K_i).$5. The trigger condition specifies the context in which the indicator is relevant, while the definition specifies what to check and how to decide (Wang et al., 26 Feb 2026).

At test time, the rectifier model $o_i = P_i(x_i, R_i, K_i).$6 extracts keywords from the current context: $o_i = P_i(x_i, R_i, K_i).$7 for task scenarios and $o_i = P_i(x_i, R_i, K_i).$8 for proposed actions. These are embedded using $o_i = P_i(x_i, R_i, K_i).$9 to form the retrieval query

$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$0

The top-$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$1 indicators are then selected by semantic similarity:

$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$2

For each retrieved indicator $$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$3, the rectifier returns a binary violation flag and a diagnostic rationale:

$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$4

The indicator pool is mined from historical failures using a teacher model $$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$5. On a source dataset with ground truth, full MAS trajectories are rolled out and failures with $$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$6 are collected. The teacher inspects agents’ outputs in these failures and synthesizes candidate indicators:

$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$7

The paper states that indicators encode diverse error patterns across math and code, including logical consistency and contradiction checks, numeric and calculation errors, domain-specific semantics such as algebra, geometry, probability, and algorithmic logic, and role-specific concerns such as functional correctness for a “Python Coder.”

Pool compactness is maintained by dual-stage deduplication. Each candidate indicator is embedded as

$$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$8

nearest neighbors are retrieved from the current pool, and a deduplication LLM $$N : \mathcal{A} \to 2^{\mathcal{A}}, \quad K_j \leftarrow K_j \cup \{(R_i, o_i)\}, \ \forall A_j \in N(A_i),$$9 determines whether the candidate reflects a novel error pattern:

$$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$0

This design makes the rectifier retrieval-augmented in a specific sense: the retrieval object is not external factual knowledge, but prior knowledge about failure modes. This suggests that the method operationalizes test-time supervision through context-aware retrieval of historically distilled error patterns.

4. Algorithmic pipeline, fallback, and implementation

The runtime pipeline is defined agent by agent. For each active agent, the system generates an initial output $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$1, extracts scenario and action keywords, embeds them into the query vector $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$2, retrieves $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$3 as the top-$$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$4 indicators by semantic similarity, evaluates each indicator to obtain $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$5, aggregates $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$6 and $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$7, and then chooses pass, retry, or reject. After propagation, if the number of valid messages falls below the safety threshold $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$8, the framework triggers a global fallback by resetting and re-initializing execution (Wang et al., 26 Feb 2026).

The pruning policy is therefore local at the message level but guarded by a global structural integrity mechanism. If an output remains flawed after $$A_{i+1} = \mathcal{T}(Q, A_{1:i}, o_{1:i}, \mathcal{A}), \quad y = o_N.$$9 iterations, it is pruned. If pruning reduces the set of valid messages below the critical threshold, a system-wide reset prevents structural collapse due to over-pruning. The paper identifies the default safety threshold as $o_i^{(t)}$0.

A zero-shot scenario is also specified. When no domain-specific pool is available, $o_i^{(t)}$1 can be initialized with a general indicator $o_i^{(t)}$2, for example “General Logic Check,” so that the framework remains functional without a mined domain pool. The reported results indicate that this generic setting provides a safety net but is less effective than domain-specific indicator pools.

The implementation used in the paper adopts AutoGen SelectorGroupChat with global broadcast, automatic routing, and 6 max chat turns; a decision agent produces the final conclusion. The selector is GPT-4.1-mini-2025-0414. Reasoners and rectifiers are Qwen3-8B and Qwen3-4B with thinking mode disabled. The teacher for pool construction is GPT-4o-2024-08-06, the deduplicator is GPT-4.1-mini-2025-0414, and the embedding model $o_i^{(t)}$3 is Qwen3-Embedding-8B. The principal hyperparameters are $o_i^{(t)}$4, $o_i^{(t)}$5, $o_i^{(t)}$6, safety threshold $o_i^{(t)}$7, rectifier temperature 0, and temperature 0.7 for other agents.

The paper characterizes the framework as plug-and-play: it requires intercepting the MAS output broadcast and inserting the rectifier and retrieval step, with no retraining of agents and compatibility with existing MAS stacks. Complexity is described in operational terms. Retrieval cost scales with indicator pool size, with embedding indexes such as ANN recommended for sublinear retrieval. Rectification cost scales linearly with $o_i^{(t)}$8 and $o_i^{(t)}$9, since each indicator invokes the rectifier model a bounded number of times.

5. Empirical performance and ablation findings

Evaluation covers math benchmarks—GSM8K, MATH-500, AQuA, AMC23, OlympiadBench, OlymMATH Easy, OlymMATH Hard, AIME24, and AIME25—and code benchmarks—MBPP, HumanEval, CodeContests, and LiveCodeBenchV1. The main MAS configuration uses AutoGen SelectorGroupChat with broadcasting to all participants, rectification $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$0, and retrieval $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$1 (Wang et al., 26 Feb 2026).

The principal quantitative findings on math benchmarks with a Qwen3-8B backbone are summarized below.

Configuration	Average accuracy
Single	47.34%
AutoGen	48.95%
With generic indicators (no domain pool)	52.16%
With retrieved indicators (full AgentDropoutV2)	55.25%

These values correspond to a gain of +1.62 over Single for AutoGen, +4.82 over AutoGen for generic indicators, and +6.30 over AutoGen for the full retrieved-indicator setting. Representative benchmark-level improvements include AIME25 from 23.33% to 30.00%, AIME24 from 26.67% to 40.00%, and OlympiadBench from 48.89% to 51.11%.

The paper also reports cross-model portability. An indicator pool mined with Qwen3-8B and transferred to Qwen3-4B raises average accuracy from 49.38% in AutoGen to 50.54%, which is +2.97 over Single and +1.16 over AutoGen. In the code domain, with generic indicators, average accuracy rises from 46.44% to 48.65%, with notable gains on CodeContests from 6.06% to 9.26% and on LiveCodeBench from 29.25% to 32.75%.

The ablation studies isolate the sensitivity of the framework to its main control parameters.

Ablation factor	Setting	Result
Iteration rounds $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$2	0	50.61%
Iteration rounds $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$3	2	52.27%
Iteration rounds $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$4	3	55.25%
Iteration rounds $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$5	4	52.61%
Retrieved indicator count $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$6	3	53.24%
Retrieved indicator count $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$7	5	55.25%
Retrieved indicator count $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$8	8	52.63%

Further ablations show that random selection of indicators yields 50.21%, lower than no rectification, which the paper interprets as confirmation of the necessity of semantically relevant indicators. Without pool deduplication, performance drops to 53.03%, supporting the claim that a diverse, high-entropy pool improves downstream behavior.

The iteration-outcome analysis illustrates the adaptive behavior of the mechanism. “Pass @ 1st” is 60.1% for GSM8K, whereas AIME24 and AIME25 exhibit multi-round refinements with rejection rates exceeding 60%. The paper therefore argues that the system conserves compute on easier tasks while allocating more rectification effort to harder ones.

6. Robustness, limitations, and relation to prior approaches

The reported robustness claims have three components. First, cross-model transfer indicates that indicators mined by a larger model can effectively supervise smaller models. Second, cross-domain gains on code tasks suggest that part of the indicator pool captures general logical auditing mechanisms rather than only narrow benchmark-specific heuristics. Third, the retrieval process is context-aware: Jaccard similarity analysis of the top-10 most used indicators across benchmarks shows clusters, with GSM8K and AQuA sharing 0.43 overlap and overlap near zero between foundational and Olympiad-level tasks such as GSM8K versus AIME25 (Wang et al., 26 Feb 2026).

These observations support the paper’s claim of dynamic adaptivity. Iteration depth and rejection rate increase with task difficulty, and aggregate rectification depth and pruning rate are described as correlating with task complexity. A plausible implication is that the framework’s own control signals may function as a difficulty proxy for MAS execution traces.

The limitations are also explicitly stated. Over-pruning can degrade connectivity, though the global fallback $$E^{(t)} = \max_{I_k \in \mathcal{I}^{(t)}} v_k^{(t)}, \quad F^{(t)} = \{ r_k^{(t)} \mid I_k \in \mathcal{I}^{(t)} \land v_k^{(t)} = 1 \}.$$9 is intended to mitigate this. Indicator quality is a critical dependency: poorly constructed or redundant indicators can produce misdiagnosis, making deduplication and careful mining central to system quality. Excessive iteration can induce over-correction or noise, as seen when $E^{(t)} = 1$0 degrades performance relative to the default. Large $E^{(t)} = 1$1 can cause information overload and distract the model, with $E^{(t)} = 1$2 identified as a balance between diversity and relevance. The zero-shot generic-indicator setting remains functional but is less effective than domain-specific pools. Finally, because the indicators in the reported setup are mined from math failures, transfer to very different domains may require rebuilding domain-specific pools for best performance.

In relation to prior work, the paper characterizes structural methods as reducing noise through optimized DAGs, sparse topologies, or orchestration and routing, but remaining static and unable to recover correctable errors. Error monitoring methods such as graphs, anomaly detection, and guards are described as often focusing on passive detection without targeted rectification. Trajectory-utilization methods based on preference or process supervision improve models through retraining or frozen-weight adaptation. AgentDropoutV2 is presented as distinct in combining test-time interception, retrieval of prior-guided indicators, iterative rectification, pruning of irreparable messages, and a global fallback mechanism.

The contrast with AgentDropoutV1 is especially explicit. V1 uses static, training-derived structural priors to permanently drop agents and includes no rectification attempt. V2 instead uses dynamic rectify-or-reject control, salvages correctable outputs, prunes only when irreparable, and improves average accuracy by 6.3 points on math benchmarks. This difference marks a shift from agent-level exclusion based on offline statistics to message-level intervention based on test-time diagnosis.