AgentArk: Distilling Multi-Agent Debate

• AgentArk is a framework that distills multi-agent reasoning dynamics into a single efficient model by internalizing debate, critique, and self-correction during training.
• It employs offline hierarchical distillation strategies—RSFT, DA, and PAD—to replicate multi-turn, multi-role interactions without runtime orchestration.
• Empirical results show significant in-domain performance improvements and enhanced robustness, reducing computational overhead compared to full multi-agent systems.

AgentArk is a framework for distilling the reasoning dynamics of LLM multi-agent systems into a single model, so that explicit test-time debate, critique, self-consistency, and self-correction are transformed into implicit capabilities encoded in model parameters (Luo et al., 3 Feb 2026). Its central premise is to shift computation from inference to training: rather than orchestrating multiple agents during deployment, AgentArk performs offline hierarchical distillation and then executes one-pass inference with a single agent. In the 2026 arXiv literature, the nearby acronym ARK also denotes “Adaptive Retriever of Knowledge,” a distinct agentic system for knowledge-graph retrieval rather than debate distillation (Polonuer et al., 20 Jan 2026).

1. Problem setting and conceptual basis

AgentArk is motivated by the tension between the strong reasoning performance of multi-agent systems (MAS) and their practical deployment costs. In the reported formulation, MAS improve reasoning by structuring multi-turn, multi-role dialogue in which agents debate, critique, and converge to a solution; this diversifies hypotheses, exposes inconsistencies, and refines solutions. However, the same interaction structure introduces two difficulties: computational overhead grows rapidly with multi-turn, multi-role orchestration, and errors or hallucinations can propagate through the dialogue and become amplified at the collective level (Luo et al., 3 Feb 2026).

The framework’s answer is to internalize these dynamics during training. A distilled single agent is trained to generate, verify, and revise within a single pass by imitating and being reinforced by multi-agent traces and learned process rewards. Relative to methods that preserve explicit interaction structures at inference, AgentArk focuses on distilling induced reasoning behaviors rather than maintaining runtime orchestration. The paper positions this against MAS debate, Tree-of-Thoughts, Self-Consistency, and Reflection, arguing that the critical resource is the induced critique-and-revision dynamic rather than the explicit topology of the interaction graph (Luo et al., 3 Feb 2026).

Architecturally, the framework spans several model families. Reported teacher-student settings include Qwen3-32B to Qwen3-8B, Qwen3-1.7B, and Qwen3-0.6B; Gemma3-27B-it to Gemma-7B; and Llama3-8B-Instruct. A multimodal extension distills Qwen2.5-VL-32B-Instruct into Qwen2.5-VL-3B-Instruct. In all cases, the distilled model differs from the teacher MAS by performing one-pass inference without agent orchestration (Luo et al., 3 Feb 2026).

2. Multi-agent debate generation and knowledge extraction

AgentArk begins from explicit multi-agent interaction logs. For each input $x$, it initializes $n$ agents $A=\{a_1,\dots,a_n\}$ sharing the same LLM and runs debate for $K$ rounds. In round $k$, agent $a_i$ generates a trace $\tau_{i,k}$ conditioned on $x$ and on peers’ previous-round traces $\{\tau_{j,k-1}\}_{j\neq i}$. The debate log is

$$L_x = \{\tau_{i,k} \mid i \in [1,n], k \in [1,K]\}.$$

The default reported configuration uses $n$0 agents and up to $n$1 rounds, with scaling studies at $n$2 (Luo et al., 3 Feb 2026).

Knowledge extraction is correctness-first. A verifier, Qwen2.5-72B-Instruct, marks answer-correct agents and forms

$n$3

If $n$4, the sample is dropped from augmentation because the framework requires at least two correct agents to preserve diversity. Among the retained traces, AgentArk prioritizes corrective trajectories, meaning traces that pivot from earlier errors to a correct final answer after critique. It then uses a high-capacity LLM judge, also Qwen2.5-72B-Instruct, to select up to $n$5 correct yet structurally diverse traces that reach the same answer through different decompositions, identities, or heuristics (Luo et al., 3 Feb 2026).

This filtering stage is central to the framework’s interpretation of multi-agent intelligence. Rather than treating every debate transcript as equally informative, it preserves answer consistency and reasoning diversity while explicitly preferring traces that exhibit self-correction. The resulting processed dataset is reported as approximately 342k questions and approximately 2M trajectories after filtering and augmentation (Luo et al., 3 Feb 2026).

3. Hierarchical distillation strategies

AgentArk organizes distillation into three strategies: Reasoning-Enhanced Supervised Fine-Tuning (RSFT), Distillation with Data Augmentation (DA), and Process-Aware Distillation (PAD). These can be used separately or combined.

Strategy	Supervision source	Reported role
RSFT	Raw multi-agent reasoning traces and correct final answer	Jointly imitates reasoning process and outcome
DA	Multiple correct, diverse trajectories per problem	Teaches multiple valid paths to the same answer
PAD	Process Reward Model (PRM) with GRPO optimization	Encodes debate-consistent step preferences and self-correction

RSFT pairs debate rationales $n$6 with the correct final answer $n$7. Its objective is written as

$n$8

with

$n$9

The intended effect is to internalize coherent multi-step thinking rather than learning a direct mapping from $A=\{a_1,\dots,a_n\}$0 to $A=\{a_1,\dots,a_n\}$1 (Luo et al., 3 Feb 2026).

DA uses multiple correct and diverse trajectories for the same problem. Its objective is

$A=\{a_1,\dots,a_n\}$2

The paper interprets this as a robustness mechanism: the student is exposed to several valid solution routes, which can improve generalization and resistance to spurious shortcuts. At the same time, the reported analyses indicate that quality matters more than quantity; PAD’s process supervision is described as more stable than merely scaling the number of raw trajectories (Luo et al., 3 Feb 2026).

PAD adds a Process Reward Model and policy optimization. The PRM is trained in two stages: a feature-alignment stage that freezes the backbone except the final layer and reward head, followed by full end-to-end specialization. Its contrastive objective is written as

$A=\{a_1,\dots,a_n\}$3

where positive steps are debate-consistent steps and negatives come from other agents. Policy optimization then uses GRPO, without a value function:

$A=\{a_1,\dots,a_n\}$4

The paper also tests PPO and reports that PPO is slightly more sample-stable but heavier because of value learning, whereas GRPO achieves comparable performance with lower overhead (Luo et al., 3 Feb 2026).

4. Evaluation regime and empirical performance

The reported evaluation spans mathematical reasoning, domain knowledge, out-of-domain transfer, robustness, and multimodal transfer. In-domain tasks include GSM8K, MATH, MetaMathQA, and MedMCQA. Zero-shot transfer is evaluated on HotpotQA, QASPER, and QMSum. Robustness is evaluated on TruthfulQA. Metrics include accuracy for closed-form tasks; F1, ROUGE-1/2/L, and BERTScore for open-ended tasks; BLEU and ROUGE variants on TruthfulQA; and reasoning-quality measures such as NLL, perplexity on reasoning tokens, and InternLM-2.5-20B-chat judge scores for step decomposition, intermediate verification, error localization, and coherence (Luo et al., 3 Feb 2026).

Across these settings, AgentArk is reported to improve single-agent performance by 4.8% on average while remaining only slightly worse than full MAS and retaining single-agent efficiency. The gains are larger in-domain than out-of-domain: maximum improvement is approximately 30% in-domain versus approximately 7% out-of-domain, and average improvement is approximately 4–6% in-domain versus approximately 1–3% out-of-domain. PAD is described as consistently robust across datasets, whereas RSFT and DA can improve performance but fluctuate more by dataset (Luo et al., 3 Feb 2026).

Dataset-specific dynamics indicate that the largest gains appear on MetaMathQA and GSM8K, moderate gains on MATH, and the smallest gains on MedMCQA. This suggests that the framework’s advantages are more strongly tied to reasoning intensity than to memorization-heavy settings. Cross-family transfer is also reported: same-family distillation is stable, smaller students benefit most, and cross-family students such as Gemma-7B and Llama3-8B show larger and more consistent improvements (Luo et al., 3 Feb 2026).

The paper also reports zero-shot transfer from GSM8K training to QA and summarization. On QMSum, for example, Qwen3-8B F1 improves from 14.94 to 17.82, and Llama3-8B F1 improves from 13.05 to 14.92. A multimodal extension shows that distillation from Qwen2.5-VL-32B to Qwen2.5-VL-3B using text-only reasoning data improves GSM8K and MedMCQA performance, with PAD strongest or near-strongest (Luo et al., 3 Feb 2026).

5. Reasoning quality, robustness, and ablation results

The framework evaluates not only task accuracy but also the internal quality of reasoning traces. Reported averages show that, relative to the single-agent baseline, RSFT, DA, and PAD all reduce NLL and perplexity on reasoning tokens and improve judge-based reasoning attributes. For example, average NLL decreases from 0.6529 for the single-agent baseline to 0.4092 for RSFT, 0.4449 for DA, and 0.5876 for PAD. Perplexity decreases from 1.9211 to 1.6388, 1.5603, and 1.7996 respectively. On reasoning-quality scores, PAD produces the strongest gains in intermediate verification and coherence, while DA is strongest in verification and error localization. Intermediate verification rises from 2.41 for the single-agent baseline to 4.07 under PAD, and reasoning coherence rises from 1.88 to 3.96 (Luo et al., 3 Feb 2026).

Robustness results on TruthfulQA point in the same direction. BLEU improves from 0.6034 for the single-agent baseline to 0.6634 under PAD; ROUGE-1 improves from 0.6144 to 0.6659; ROUGE-2 from 0.5704 to 0.6414; and ROUGE-L from 0.6132 to 0.6573. The paper interprets this as evidence that process-aware supervision helps reduce compounding reasoning errors and improves resilience to noisy or misleading conditions (Luo et al., 3 Feb 2026).

Ablations refine this picture. Teacher scaling helps only when matched to student capacity: a Qwen3-0.6B student saturates or degrades beyond 5 agents, while a Qwen3-8B student benefits modestly with diminishing returns at higher $A=\{a_1,\dots,a_n\}$5. Data scaling is non-monotonic for RSFT and DA, and more data can degrade performance; PAD remains comparatively stable, reinforcing the claim that quality matters more than quantity. PRM size is also significant: larger PRMs yield better student outcomes even for small policy models, so PRM capacity matters more than student size, although student capacity still bounds ultimate gains. Combining methods, such as PAD+DA or RSFT+DA, yields consistent but modest improvements (Luo et al., 3 Feb 2026).

6. Efficiency, limitations, and related usage of the name

AgentArk’s deployment argument rests on inference-time efficiency. The distilled model performs one generation pass, whereas MAS requires multiple agent invocations and rounds. On the reported 8B-student setup with NVIDIA H100 80GB GPUs, RSFT takes approximately 6 hours on 1 H100, DA approximately 8 hours on 1 H100, and PAD approximately 20 hours on 8 H100s, with PRM training taking approximately 8 hours and GRPO approximately 12 hours. All methods share the same multi-agent data-generation cost; their differences arise in supervision and optimization. The reported global batch size is 4, with identical sequence length across methods (Luo et al., 3 Feb 2026).

Several limitations are stated explicitly. Task coverage remains limited to a particular set of reasoning and multimodal benchmarks. The exploration of distillation variants is selective, and alternative or hybrid process-oriented strategies may improve transfer further. Although AgentArk is described as MAS-agnostic, the experiments focus on debate, so the extent to which the same results carry over to other MAS protocols remains an open empirical question (Luo et al., 3 Feb 2026).

The name also sits near a distinct 2026 line of work, ARK, or “Adaptive Retriever of Knowledge,” which addresses autonomous knowledge-graph exploration rather than debate distillation. ARK formalizes a KG retriever with two tools—global lexical search over node descriptors and one-hop neighborhood exploration—and uses an LLM to balance breadth-oriented discovery with depth-oriented expansion. On STaRK, the reported training-free GPT-4.1 system reaches average Hit@1 59.14, R@20 71.51, and MRR 67.44, and its label-free imitation setup improves a Qwen3-8B student by +7.04 on AMAZON, +26.57 on MAG, and +13.50 on PRIME Hit@1 (Polonuer et al., 20 Jan 2026). A plausible implication is that “AgentArk” may be encountered in two adjacent but methodologically distinct contexts: one centered on single-agent distillation of multi-agent debate, and one centered on adaptive tool use for knowledge-graph retrieval.