Multi-Turn Multi-Agent Orchestration

• Multi-turn multi-agent orchestration is the structured organization of agents that interact over multiple steps with a central controller enabling role delegation and context management.
• It encompasses diverse approaches, including runtime-interactive systems, holistic upfront planning, and consensus protocols, each tailored to specific task structures and performance objectives.
• Empirical studies show notable gains in accuracy, latency, and cost-efficiency, while also highlighting challenges in state management, credit assignment, and inter-agent communication.

Multi-turn multi-agent orchestration denotes the organization of multiple agents, role-conditioned executors, or callable components across repeated interaction steps, usually under a controller that manages decomposition, context propagation, tool use, verification, and stopping. Recent work spans centralized runtime orchestration for long-horizon environments, planner/worker decompositions for tool-integrated reasoning, deterministic consensus protocols, and holistic workflow synthesis that emits an executable multi-agent structure in one step rather than replanning after every observation (Ruan et al., 3 Feb 2026, Mo et al., 6 Oct 2025, Song et al., 9 Jul 2025, Tian et al., 28 Sep 2025, Ke et al., 21 Jan 2026, Chen et al., 14 May 2026).

1. Architectural scope and major forms

A first distinction is between runtime-interactive orchestration and holistic upfront orchestration. In runtime-interactive systems, the orchestrator repeatedly observes intermediate outputs and decides what should happen next. AOrchestra is explicit about this structure: it treats long-horizon solving as a repeated decision process over trajectories, constrains the orchestrator to delegation or termination, and emphasizes that the orchestrator never directly takes environment actions (Ruan et al., 3 Feb 2026). MATPO likewise uses a planner-agent that emits subtasks over multiple turns and a worker-agent that runs a local tool-using sub-trajectory before returning a compact result to the planner (Mo et al., 6 Oct 2025). Gradientsys centers a ReAct-style LLM Scheduler that plans, dispatches, observes, retries, and replans over multiple internal turns (Song et al., 9 Jul 2025). MARCO routes each conversational turn through an Intent Classifier and, for action intents, into a hierarchical task-agent framework with shared context, deterministic tasks, and guardrails (Shrimal et al., 2024).

A second family plans more globally. MAS-Orchestra formulates orchestration as a single latent action that specifies a complete multi-agent system, with the orchestrator generating the entire orchestration in one decision step and not observing intermediate states or partial results during execution (Ke et al., 21 Jan 2026). LEMON similarly generates a single executable YAML orchestration specification containing roles, duties, capacities, and dependencies, which is then compiled into a layered directed acyclic graph (Chen et al., 14 May 2026). HERA is dynamic per query and evolves across episodes through experience accumulation, but the paper repeatedly states that the orchestrator produces query-specific execution plans in a single step rather than replanning after every micro-step (Li et al., 1 Apr 2026).

A third family is consensus-oriented rather than delegation-oriented. “Beyond the Strongest LLM” studies a protocol in which several heterogeneous LLM agents iteratively propose answers or cast votes until consensus is reached, with a dynamic restart when a new answer appears (Tian et al., 28 Sep 2025). ORCH uses a deterministic “many analyses, one decision” pipeline: a dispatcher proposes complementary sub-questions, heterogeneous base agents analyze them in parallel, and a dedicated merge agent outputs the final choice (Zhou et al., 2 Feb 2026). These systems are multi-agent and multi-step at the protocol level, but they are not long-horizon controllers in the same sense as AOrchestra or MARCO.

This suggests a practical taxonomy: some systems orchestrate who should act next in an evolving environment, while others orchestrate what complete structure should exist before execution starts. The literature treats both as orchestration, but their state, credit assignment, and failure-recovery mechanisms differ substantially.

2. State, context, and communication across turns

The multi-turn substrate is often formalized explicitly. AOrchestra models interaction as

$\tau = (s_0, a_0, o_0, s_1, a_1, o_1, \dots, s_T),$

with state updates

$s_{t+1} = \delta(s_t, a_t, o_t),$

making orchestration a repeated state-update process rather than a one-shot decomposition (Ruan et al., 3 Feb 2026). MATPO writes a full multi-agent trajectory as

$\tau \triangleq [a_1,\tau^1,s_1,\dots,a_{T-1},\tau^{T-1},s_{T-1},a_T],$

where each $\tau^t$ is a worker rollout and each $s_t$ is the parsed worker response consumed by the planner on the next turn (Mo et al., 6 Oct 2025). HERA also uses trajectory notation,

$\tau = (s_0, a_0, s_1, a_1, \dots, s_T),$

but applies it to topology-conditioned multi-step RAG execution rather than live conversational turn-taking (Li et al., 1 Apr 2026).

A central issue is what context crosses delegation boundaries. AOrchestra treats context as a first-class component of its agent tuple and reports a GAIA ablation where curated context outperforms both no context and full context while keeping sub-agent model, tools, and system prompt identical: No-Context average 86.00, Full-Context average 84.00, and Ours 96.00 (Ruan et al., 3 Feb 2026). MATPO operationalizes a similar idea through planner/worker context partitioning: the planner keeps the user query, planner history, and compact worker outputs, whereas the worker absorbs the subtask, local tool calls, and raw tool responses (Mo et al., 6 Oct 2025). MARCO encodes shared conversational state with typed roles such as [SYSTEM], [USER], [AGENT], [FUNCTION_RESPONSE], and [GUARDRAILS], and explicitly states that the common chat history thread is shared among all Task-Agents for a chat session (Shrimal et al., 2024). Gradientsys uses an append-only scratchpad with summarization and truncation to keep multi-turn planning within context limits (Song et al., 9 Jul 2025).

Communication topology is usually centralized. AOrchestra states that communication is mediated through the orchestrator rather than direct agent-to-agent exchange; sub-agents return structured observations and the orchestrator decides the next move (Ruan et al., 3 Feb 2026). MARCO relies on a shared history thread and explicit agent-switch markers rather than peer-to-peer agent dialogue (Shrimal et al., 2024). Gradientsys is also centralized: tools and agents do not usually orchestrate one another; the current design keeps a single central planner (Song et al., 9 Jul 2025). A notable exception is the cross-domain workflow in “LLM-Driven Cross-Domain Orchestration Using Multi-Agent Workflow,” where two chat groups—one for the OTN domain and one for the robotic domain—contain manager, planner, writer, and executor roles, and the groups are linked through the executors (Xu et al., 2024).

3. Core orchestration primitives and abstractions

Different systems choose different units of orchestration.

Abstraction	Representative systems	Key property
Agent tuple	AOrchestra	Delegation unit is $\Phi=(I,C,T,M)$
Planner/worker rollout	MATPO	Planner emits subtask; worker runs local tool trajectory
Executable graph/specification	MAS-Orchestra, LEMON, OrchDAG	Orchestrator emits callable graph or DAG-like structure
Consensus protocol	“Beyond the Strongest LLM”, ORCH	Multiple analyses or votes are merged into one answer
Hierarchical task agent	MARCO	Agent chooses tool call or child agent under shared context

AOrchestra’s central abstraction is the four-tuple

$\Phi = (I, C, T, M),$

where instruction and context define working memory, while tools and model define capabilities (Ruan et al., 3 Feb 2026). This permits “on-demand specialization”: the orchestrator dynamically synthesizes a sub-agent for the current subtask instead of relying on a static worker role. LEMON moves the unit of orchestration from a tuple to an executable specification:

$y \sim \pi_\theta(\cdot \mid x), \qquad G=\mathcal{C}(y)=(V,E),$

with each node carrying role specification and capacity and each edge transmitting outputs as context (Chen et al., 14 May 2026). OrchDAG formalizes multi-turn tool execution as an attributed DAG,

$\mathcal{G} = (\mathcal{V}, \mathcal{E}),$

where edges carry explicit output-to-input dependencies and later turns may reuse previous outputs or reschedule unfinished branches (Lu et al., 28 Oct 2025). MAS-Orchestra uses a related but more holistic formulation,

$s_{t+1} = \delta(s_t, a_t, o_t),$0

where $s_{t+1} = \delta(s_t, a_t, o_t),$1 encodes a complete MAS orchestration and $s_{t+1} = \delta(s_t, a_t, o_t),$2 sets the degree of MAS (Ke et al., 21 Jan 2026).

Another axis concerns how answers are merged. The propose-or-vote protocol in “Beyond the Strongest LLM” constrains each agent at each step to exactly one of two actions—propose a new candidate answer or vote for an existing answer—and uses dynamic restart to invalidate prior votes whenever a new answer enters the set (Tian et al., 28 Sep 2025). ORCH is more deterministic: it normalizes the task, dispatches three complementary sub-questions, gathers structured analyses from three heterogeneous LLMs, and uses a dedicated merger agent to output exactly one final answer (Zhou et al., 2 Feb 2026). The difference is substantive. Majority-vote systems aggregate labels; ORCH aggregates textual analyses; AOrchestra and MARCO aggregate structured observations and tool responses.

MARCO contributes a different primitive: deterministic tasks embedded inside a multi-agent controller (Shrimal et al., 2024). Rather than using the LLM for every business-logic transition, MARCO packages many multi-step API chains as deterministic tasks callable by the active task agent. Gradientsys adds a typed MCP-style registry abstraction with tool metadata such as name, description, endpoint, schema hints, max_parallel, and optional cost information, making orchestration partly a scheduling problem over typed resources (Song et al., 9 Jul 2025).

4. Learning, optimization, and routing

A large part of recent work treats orchestration itself as a learnable policy. AOrchestra formalizes cost-aware orchestration as

$s_{t+1} = \delta(s_t, a_t, o_t),$3

and then separates learning into task orchestration and model routing (Ruan et al., 3 Feb 2026). Task orchestration can be improved with supervised fine-tuning from expert orchestration trajectories,

$s_{t+1} = \delta(s_t, a_t, o_t),$4

while prompt-level optimization of the main orchestrator instruction is used to search for better cost-performance trade-offs (Ruan et al., 3 Feb 2026).

MATPO addresses a different problem: how to assign credit when only the planner’s final answer is directly verifiable. Its contribution is a policy-gradient factorization over the whole composite planner/worker trajectory so that all planner and worker tokens are trained under the same final reward signal, with the planner-level normalized advantage broadcast to associated worker rollouts (Mo et al., 6 Oct 2025). This makes planner/worker orchestration trainable without requiring separate verifiable subtask rewards.

LEMON localizes orchestration credit even further. It augments orchestration-level GRPO with counterfactual edits that remove one dependency reference, roll back one customized duty to its inherited base-role description, or downgrade one capacity level, then applies the resulting reward contrast only to the edited spans (Chen et al., 14 May 2026). The practical effect is to train role design, capacity assignment, and dependency construction jointly rather than optimizing them sequentially. MAS-Orchestra also uses GRPO, but at the level of a one-shot orchestration output rather than a turn-by-turn controller (Ke et al., 21 Jan 2026).

HERA is notable for non-parametric orchestration memory. It samples a group of query-specific topologies, compares them by task performance and token cost, distills comparative insights into an Experience Library, and then conditions future topology sampling on that library (Li et al., 1 Apr 2026). In parallel, Role-Aware Prompt Evolution updates role-specific prompts through operational rules and behavioral principles extracted from failure cases (Li et al., 1 Apr 2026). This is not standard policy-gradient optimization, but it is still an orchestration learning loop.

At a more abstract level, “When Should We Orchestrate Multiple Agents?” argues that orchestration is only effective if there are performance or cost differentials between agents, and formalizes an online cost-aware routing rule over heterogeneous experts under constraints (Bhatt et al., 17 Mar 2025). This is a useful corrective to papers that assume multi-agent coordination is intrinsically beneficial. MAS-Orchestra arrives at a closely related empirical conclusion: MAS gains depend critically on task structure, verification protocols, and the capabilities of both orchestrator and sub-agents, rather than holding universally (Ke et al., 21 Jan 2026).

5. Systems infrastructure, observability, and operational control

Several papers treat orchestration as a systems problem rather than only a prompting or RL problem. Gradientsys couples a centralized LLM Scheduler with a typed Model-Context Protocol layer, hybrid synchronous/asynchronous execution via concurrent.futures and asyncio, per-tool concurrency constraints through max_parallel, and an SSE observability layer that streams thought, action, result, and final_answer events (Song et al., 9 Jul 2025). This gives orchestration a concrete runtime with queueing, timeouts, retry-and-replan, and visibility into intermediate control flow.

MARCO treats real-time chat orchestration similarly. Its top-level modules—Intent Classifier, RAG, and MARS—are independent and can be executed in parallel to reduce latency, with the final output selected according to the intent classification (Shrimal et al., 2024). Within MARS, shared typed memory and reflection guardrails allow the controller to maintain stable turn-level execution under strict output schemas and bounded retries. This places output validation, tool grounding, and user-facing messaging inside the orchestration loop rather than outside it.

ProRL Agent and AgentRL are more infrastructural than algorithmic from a multi-agent standpoint. ProRL Agent presents “rollout-as-a-service”: a standalone HTTP server with a three-stage asynchronous pipeline INIT → RUN → EVAL, pluggable AgentHandler(init, run, eval) methods, and a min-heap LLM backend pool with dynamic registration and checkpoint swapping (Zhang et al., 19 Mar 2026). The paper is explicit that it targets multi-turn agent rollouts, not explicit multi-agent coordination, but the lifecycle management, cancellation, backend routing, and sandbox isolation are directly relevant orchestration primitives (Zhang et al., 19 Mar 2026). AgentRL likewise focuses on scalable multi-turn, multi-task RL with a fully asynchronous generation-training pipeline, a unified function-call API, containerized environment workers, and a centralized controller; its relevance to multi-agent orchestration is therefore architectural rather than a demonstrated cooperative-agent result (Zhang et al., 5 Oct 2025).

A plausible implication is that orchestration research is bifurcating into policy design and control-plane engineering. The former studies roles, prompts, and rewards; the latter studies queues, backends, containerization, fault containment, and observability. Current systems often need both.

6. Empirical evidence, misconceptions, and open problems

The empirical record shows that orchestration can help substantially, but not uniformly.

System	Setting	Reported result
AOrchestra	GAIA, Terminal-Bench 2.0, SWE-Bench-Verified	16.28% relative improvement against the strongest baseline with Gemini-3-Flash; 80.00, 52.86, and 82.00 pass@1 (Ruan et al., 3 Feb 2026)
MATPO	GAIA-text, WebWalkerQA, FRAMES	18.38% average relative improvement over single-agent GRPO (Mo et al., 6 Oct 2025)
HERA	Six knowledge-intensive benchmarks	38.69% average improvement over recent baselines (Li et al., 1 Apr 2026)
MARCO	DRSP-Conv, Retail-Conv	94.48% and 92.74% task-execution accuracy; 44.91% latency improvement and 33.71% cost reduction versus single-agent baseline (Shrimal et al., 2024)
Gradientsys	GAIA	24.1% accuracy versus 15.0% baseline, 35 s versus 52 s latency, and 0.22 versus 1.00 normalized cost (Song et al., 9 Jul 2025)
ORCH	MMLU-Pro, GSM8K	improves accuracy by over 10 points on MMLU-Pro and yields gains exceeding 50 points on GSM8K (Zhou et al., 2 Feb 2026)

These gains, however, coexist with strong counterexamples to a simplistic “more agents are better” narrative. MAS-Orchestra states directly that MAS gains are not universal and are most effective at the edge of sub-agent competence; strongly sequential tasks characterized by Depth often do not favor MAS, whereas Robustness is the clearest positive regime (Ke et al., 21 Jan 2026). “When Should We Orchestrate Multiple Agents?” makes the same point theoretically: orchestration is only effective if there are performance or cost differentials between agents, and unnecessary orchestration can overestimate gains while underestimating cost (Bhatt et al., 17 Mar 2025).

Consensus-style systems expose additional social pathologies. “Beyond the Strongest LLM” shows that revealing authorship increases self-voting and raises Consensus Tie Rate from 14.1% to 23.2%, while visible vote tallies increase the First-voted Selected Rate from 54.1% to 67.8%, amplifying herding and sometimes yielding premature consensus (Tian et al., 28 Sep 2025). ORCH’s response is deterministic routing and merge rules, but ORCH also explicitly limits its claim of determinism to fixed routing and aggregation rules under a fixed evaluation protocol, not strict bit-level reproducibility across deployments (Zhou et al., 2 Feb 2026).

Another recurring limitation concerns what counts as multi-turn orchestration. MAS-Orchestra, HERA, and LEMON all study orchestration over complex agents or multi-step executions, but their controllers mostly plan upfront rather than reactively after every intermediate observation (Ke et al., 21 Jan 2026, Li et al., 1 Apr 2026, Chen et al., 14 May 2026). AgentRL and ProRL Agent, despite their importance for multi-turn systems, do not themselves implement explicit multi-agent coordination (Zhang et al., 5 Oct 2025, Zhang et al., 19 Mar 2026). ORCH, despite being a strong multi-agent orchestrator for discrete-choice reasoning, is mostly single-round. This suggests that the field still distinguishes sharply between interactive runtime orchestration and workflow synthesis for multi-step execution.

A final misconception is that orchestration quality is reducible to stronger base models. Several papers argue otherwise. AOrchestra’s learnable-orchestrator ablation shows that weaker orchestration degrades results even when sub-agents remain strong (Ruan et al., 3 Feb 2026). MATPO reports that planner/worker coordination improves robustness to noisy tool outputs beyond what single-agent RL provides (Mo et al., 6 Oct 2025). MARCO shows that narrower, hierarchical task agents can outperform a single broader agent while lowering latency and cost (Shrimal et al., 2024). The cumulative implication is that orchestration is not merely packaging; it is itself a substantive modeling problem.

Open problems follow directly from these limits: richer stateful replanning over long dialogues, principled confidence or verification signals for live routing without gold feedback, finer-grained credit assignment for collaborative teams, direct inter-agent communication protocols where needed, and better integration of cost, latency, and reliability into orchestration policies. Current work has established that multi-turn multi-agent orchestration can be effective, but it has also made clear that its benefits depend on task structure, context management, communication topology, and the degree to which orchestration itself is treated as a first-class object of design and learning.