OrchVis: Multi-Agent Orchestration Framework

• OrchVis is a multi-agent orchestration framework that visualizes, verifies, and coordinates goal-driven LLM collaborations through hierarchical goal alignment and task decomposition.
• It employs a grammar-constrained goal parser, deterministic agent matching using skill matrices, and layered visualizations to enhance transparency and minimize misalignment.
• The system continuously verifies execution with machine-checkable predicates and facilitates human oversight via conflict-aware replanning and interactive visual panels.

OrchVis is a multi-agent orchestration framework that visualizes, verifies, and coordinates goal-driven collaboration among LLM-based agents through hierarchical goal alignment, task assignment, and conflict resolution. It is designed to make complex LLM-driven collaborations legible, verifiable, and controllable to end users through hierarchical goal alignment, layered visualization, and conflict-aware replanning. Its central premise is that humans should supervise at a strategic level—inspecting high-level goals and drilling into task-level details or inter-agent workflows only when needed—rather than micromanaging each procedural step (Zhou, 28 Oct 2025).

1. Conceptual basis and design goals

OrchVis is motivated by the difficulty of supervising multi-agent LLM workflows at scale. Prior work cited in the system description states that the user effort needed to directly coordinate $k$ sub-agents grows at least $O(k)$, making direct manipulation increasingly impractical as agent collectives grow. The framework addresses two recurring failure modes: misalignment between user intent and agent-inferred goals, and the opacity of text-heavy interfaces, which make dependencies and conflicts difficult to perceive and correct (Zhou, 28 Oct 2025).

The framework organizes its design around three explicit goals. Design Goal 1 introduces an orchestration agent, described as an “orchestrator/supervisor,” that consolidates low-level multi-agent activity into high-level summaries. Design Goal 2 separates “goals” from “tasks,” where goals denote desired target states and tasks denote procedures, so that alignment occurs on the former before execution proceeds on the latter. Design Goal 3 visualizes inter-agent dependencies and conflicts through a GUI, particularly through an interactive Planning Panel that reveals how the orchestrator routes subgoals to sub-agents, where conflicts occur, and what repairs are proposed (Zhou, 28 Oct 2025).

Several technical contributions follow from these design goals. OrchVis introduces a grammar-constrained goal parser that produces a structured, machine-checkable goal hierarchy; deterministic agent matching using agent skill matrices; continuous verification via machine-checkable predicates with per-goal satisfaction scores; and a two-layer Planning Panel that exposes the relationship between goals and the underlying task, workflow, and tool-call layers. The system therefore frames orchestration not as message passing alone, but as an explicitly modeled alignment problem over goals, plans, dependencies, and verification artifacts (Zhou, 28 Oct 2025).

2. System architecture and core representations

The high-level control flow proceeds through four tightly coupled stages: goal alignment, task assignment and plan generation, progress tracking and verification, and conflict detection with Planning Panel correction. The stated data and control flow is: user intent to goal parser to goal hierarchy; orchestrator constructs plan to deterministic agent matching to composable task graph; execution to verifier to UI; and conflict resolver to Planning Panel to user-selected repair to partial replan and resume (Zhou, 28 Oct 2025).

The architecture distinguishes four principal roles. The orchestrator or supervisor agent interprets intent, proposes and repairs the goal hierarchy, assigns subgoals to specialized sub-agents, coordinates execution, and triggers partial re-planning on conflicts. Sub-agents or workers are LLM- or tool-augmented specialists for search, retrieval, programmatic tools, and verification subroutines. The verifier evaluates execution state against machine-checkable success predicates on each goal or subgoal, computes satisfaction scores, and generates “difference reports.” The conflict resolver or re-planner translates verifier differences into human-readable summaries, proposes repairs with predicted outcomes, and updates the plan upon user acceptance (Zhou, 28 Oct 2025).

The core data structures are equally explicit. The goal hierarchy tree or graph $H$ contains goals as nodes with constraints, ontology bindings, and predicates, while edges encode hierarchical and temporal relations such as sequential, parallel, and conditional relations. The inter-agent dependency graph $D$ includes goals, tasks, and agent roles as nodes, with edges for task dependencies and agent-tool interactions. Plans are represented as a composable task graph with assignments to sub-agents and temporal or resource constraints. Verification artifacts include predicate bindings, satisfaction scores, “difference reports,” and predicted outcomes for alternative repairs (Zhou, 28 Oct 2025).

A formalization consistent with the design treats goals as $G = \{g_1, \dots, g_n\}$, each with hard predicates $P_i(x) \in \{0,1\}$ and soft preferences $Q_i(x) \in [0,1]$, organized in a directed acyclic graph $H = (G, E_H)$. Task decomposition assigns each goal a set of tasks $T_i = \{\tau_{i1}, \dots, \tau_{im}\}$, and executable workflows are encoded as composable task graphs with precedence constraints, parallel branches, and conditional edges rather than as an explicit MDP formalism (Zhou, 28 Oct 2025).

3. Goal parsing, agent matching, and plan generation

The goal parser converts free-form text into a structured goal graph using few-shot exemplars and grammar-constrained decoding. In the described implementation, it identifies hierarchical relations such as sequential, parallel, and conditional structure; fills domain-specific fields; normalizes attributes such as time and cost; grounds each goal to an ontology; and attaches machine-checkable success predicates. A “lightweight repair loop” maintains consistency between user-edited goals and the internal hierarchy representation (Zhou, 28 Oct 2025).

This parsing stage is central because OrchVis distinguishes the “what” from the “how.” The system aligns on goals before task execution, then emphasizes per-goal progress during execution while keeping lower-level task details collapsible. The user interface therefore presents top-level goals by default and defers workflow detail until it is needed for inspection or intervention (Zhou, 28 Oct 2025).

Task assignment is handled by deterministic agent matching. The system compares each task’s declared requirements against every sub-agent’s skill matrix, where the matrix records available tools, I/O schema, and historical performance. A formalization consistent with the system expresses this as

$A(\tau) = \arg\max_{a \in A} f(r(\tau), s(a)),$

where $O(k)$0 is the task requirement vector and $O(k)$1 is the agent skill vector. The paper emphasizes that the matching is deterministic rather than probabilistic, and that the result is an executable plan represented as a composable task graph (Zhou, 28 Oct 2025).

The planning objective is described implicitly rather than as a fully specified optimizer. A formalization consistent with the design seeks a plan $O(k)$2 that maximizes aggregate utility over aligned goals and their task realizations, subject to hard predicates, dependency feasibility, and resource constraints. This suggests that OrchVis treats planning as a constrained hierarchical decomposition problem in which alignment on success conditions precedes execution and constrains subsequent task routing (Zhou, 28 Oct 2025).

4. Verification, satisfaction scoring, and conflict-aware replanning

Automated verification is continuous and per-goal. The verifier checks hard and soft constraints using machine-checkable predicates already bound during goal alignment. It returns both a binary achievement flag and an overall satisfaction score, while unstructured evidence such as text or images is normalized by a “lightweight extractor LLM” for rule-based comparison (Zhou, 28 Oct 2025).

The explicit satisfaction score given in the design is

$O(k)$3

where $O(k)$4 weights soft satisfaction relative to hard constraints. The verifier reports both $O(k)$5 and $O(k)$6, together with a “difference report” $O(k)$7 describing unsatisfied constraints and evidence gaps (Zhou, 28 Oct 2025).

Conflict handling is organized around these difference reports. The system description states that verifier differences are transformed into natural-language summaries and proposed repairs, and that the orchestrator “pauses only affected branches” while “allowing other workflows to continue executing in parallel.” A formalization consistent with the design treats conflicts as arising from logical incompatibility, resource contention, or dependency violations, and limits re-optimization to the impacted subgraph of the dependency graph (Zhou, 28 Oct 2025).

Selective replanning is then driven by a multi-criteria objective over candidate repairs:

$O(k)$8

subject to feasibility and consistency with the user-edited goal hierarchy. The Planning Panel surfaces these alternatives together with predicted progress, risk, and cost, after which the user may accept a repair and resume execution. If no suggestion is accepted, the system uses LLM-guided reasoning to generate additional candidates for partial re-planning (Zhou, 28 Oct 2025).

5. Goal-aware visualization and human oversight

OrchVis adopts a GUI-first, goal-aware visualization strategy. Its display design keeps lower levels collapsed by default in order to reduce cognitive load, while exposing high-level goals, per-goal progress, and conflict status directly in the main interface. This visualization layer is not ancillary; it is the principal mechanism by which the system renders multi-agent execution inspectable and controllable (Zhou, 28 Oct 2025).

The Planning Panel is the system’s primary visual analytic surface. It is explicitly two-layered: the upper layer displays the goal hierarchy, and the lower layer depicts the corresponding task-level workflows and tool calls. Conflicts are flagged in situ and expanded into this panel, which reveals how the orchestrator routes work to sub-agents and pinpoints which sub-agents are in conflict. The paper’s travel-planning example describes a conflict between an outbound flight and an evening show, with the relevant goal nodes highlighted above and the implicated workflow branches below (Zhou, 28 Oct 2025).

A summary pane complements the Planning Panel. It presents high-level rationales, proposed resolutions, and predicted future states, and also provides controls for setting the desired level of autonomy. This autonomy control is central to the framework’s human-centered design: OrchVis acts autonomously on low-risk repairs, but pauses and asks for input on higher-risk conflicts or goal-level misalignments. Intervention points therefore occur during goal alignment, conflict resolution, and autonomy calibration, rather than at every intermediate tool invocation or agent message (Zhou, 28 Oct 2025).

In adjacent orchestration literatures, visualization has been used to make other complex coordination problems legible. DIVI deconstructs static SVG charts at runtime to infer semantics and orchestrate interactions across views and tools without prior interaction specifications (Snyder et al., 2023), while a unified interface for retinal-data analysis explicitly coordinates workflow execution, tool activation, data exchange, and on-screen layout across heterogeneous software (Röhlig et al., 2023). This suggests a broader lineage in which OrchVis extends orchestration from views and tools to hierarchical multi-agent planning and oversight.

6. Example workflow, evaluation status, limitations, and context

The travel-planning scenario in the system description illustrates the end-to-end loop. A user request such as planning a trip to San Francisco with an arrival-time constraint, a hotel budget and location constraint, and a same-day evening show is parsed into top-level goals for flight, hotel, and itinerary. The orchestrator decomposes these into task sets such as search flights, apply coupons, book; search hotels, filter by SoMa, book; and search shows, select seats, book. Deterministic agent matching assigns tasks to sub-agents with appropriate tools, and a composable task graph is generated with parallel branches where feasible and sequential dependencies where required for consistency (Zhou, 28 Oct 2025).

When execution produces a flight arriving at 18:00 despite a goal requiring arrival by 15:00, the verifier marks the flight goal as hard-constraint unsatisfied and identifies a conflict with the itinerary goal. The Planning Panel then expands to show the conflicting goals and workflow branches, and the conflict resolver proposes repairs such as rebooking an earlier flight, shifting the show, or relaxing the arrival constraint, each with predicted progress impact, availability risk, and cost changes. If the user accepts a repair, the orchestrator pauses only the affected branch, updates the plan, and resumes unaffected branches in parallel (Zhou, 28 Oct 2025).

As of the cited paper, OrchVis remains a system-design contribution rather than an empirically benchmarked platform. The paper explicitly states that it does not report empirical evaluations, benchmark studies, or user studies. It proposes future evaluation across online shopping, office assistance, data analysis, browsing, and OS interaction benchmarks, naming OSWorld, TheAgentCompany, SpreadsheetBench, DSBench, WebShop, and BrowseComp, and also proposes user studies of oversight quality and decision efficiency in long-horizon, multi-agent tasks. Metrics, quantitative results, ablations, and compute or scalability analyses are therefore not yet provided (Zhou, 28 Oct 2025).

The principal limitations follow directly from the design. Machine-checkable predicates depend on correct extraction from unstructured evidence, so extractor errors can produce false positives or negatives. Very large goal hierarchies or dense dependency graphs may challenge visualization clarity even with collapsing and layered views. Grammar-constrained parsing may be brittle under domain shift or atypical user intents. Partial replanning across parallel branches remains a non-trivial consistency problem. The paper also notes the absence of code repositories, datasets, or demo links, so reproducibility artifacts are not yet included (Zhou, 28 Oct 2025).

Relative to related multi-agent frameworks such as AutoGen, Magentic-One, CAMEL, LangGraph or LangChain agent stacks, and debate frameworks, OrchVis is distinguished less by end-to-end automation than by transparency, controllability, early goal alignment, and conflict-aware replanning. Its contribution is therefore best understood as a human-centered orchestration layer: one that separates goals from tasks, grounds execution in machine-checkable predicates, surfaces dependencies through layered visualization, and makes replanning a supervised rather than opaque process (Zhou, 28 Oct 2025).