AgentMaster: Modular Multi-Agent Framework

• AgentMaster is a modular multi-agent conversational framework that supports dynamic coordination among heterogeneous agents.
• It integrates A2A for JSON-RPC inter-agent communication and MCP for precise tool invocation and multimodal processing.
• The framework enables flexible query decomposition, efficient routing, and robust multimodal information retrieval across various domains.

AgentMaster is a modular, multi-agent conversational framework designed for multimodal information retrieval and analysis, integrating explicit support for both Google’s Agent-to-Agent (A2A) protocol and Anthropic’s Model Context Protocol (MCP). Architected to facilitate robust, dynamic coordination among heterogeneous agents, seamless external tool invocation, and servable multi-turn interaction, AgentMaster aims to address longstanding challenges in inter-agent communication, orchestration, and flexible query decomposition in multi-agent systems (MAS), particularly those incorporating LLMs (Liao et al., 8 Jul 2025).

1. System Architecture

AgentMaster is organized into four principal architectural layers:

1. Unified Conversational Interface (Agent Manager): A web-based React/Flask chat widget accepts user input across modalities—text, images, charts, audio—and displays multimodal responses.
2. Multi-Agent Center: Comprises the Coordinator (Orchestrator) Agent, responsible for agent registry, capability discovery, query assessment, decomposition, and synthesis; and Domain and General Agents. Domain Agents (e.g., SQL Agent, IR Agent, Image Agent) are specialized microservices with MCP endpoints. General Agents are LLM-powered catch-alls for open-domain tasks or fallback.
3. Multi-Agent Protocol/Tool Layer: Implements the A2A protocol for structured, JSON-RPC-based inter-agent communication (coordination, delegation, error handling), and MCP for tool invocation, database access, and external API integration.
4. State Management Layer: Deploys a hybrid memory system combining persistent, vector-database-backed semantic memory with in-memory, session-specific context caches.

Agents (typically deployed as Flask microservices) announce themselves at startup by registering their A2A endpoints and capabilities with the Coordinator via an initial handshake. This allows the Coordinator to dynamically maintain an internal routing directory (Liao et al., 8 Jul 2025).

2. Communication Protocols: A2A and MCP

A2A Protocol

A2A operationalizes agent communication through JSON objects containing metadata (message_id, sender, recipient, timestamp, type, payload). Handshake initialization consists of agents sending a REGISTER message with their capability declarations, and the Coordinator responding with ACK_REGISTER.

A2A supports delegated subtask requests, as illustrated in the following JSON template:

{ "message_id": "m1234", "sender": "Coordinator", "recipient": "SQL_Agent", "type": "SUBTASK_REQUEST", "payload": {"query": "Which bridges in Virginia were built in 2019?"} }

MCP Protocol

MCP standardizes context and tool invocation by transmitting ordered sequences of {role, content} tuples corresponding to system, user, and assistant roles. For example, an SQL query to a database via MCP:

[ {"role": "system", "content": "You are an MCP-enabled SQL box."}, {"role": "user", "content": "SELECT COUNT(*) FROM bridge_basic_info WHERE state_name='Virginia';"} ]

The MCP protocol enables precise packaging, invocation, and return of results from tool “boxes” (APIs, DBs, vision systems), preserving agent autonomy while supporting convened orchestration via the Coordinator (Liao et al., 8 Jul 2025).

A2A–MCP Interoperation

AgentMaster’s Coordinator serves as a bridge between direct tool invocation (for simple queries) and distributed subtasking via A2A. Algorithmically, queries are assessed for complexity; simple requests trigger direct MCP calls, while complex requests are decomposed into subtasks, dispatched to capable agents over A2A, and potentially trigger nested MCP tool invocations. The Coordinator then integrates responses from all subtasks and presents the synthetic final answer (Liao et al., 8 Jul 2025).

3. Task Decomposition, Routing, and Assignment

AgentMaster formalizes the decomposition-routing process as an assignment problem. Given a set of subtasks $S = \{s_1,\ldots,s_n\}$ and available agents $A = \{a_1,\ldots,a_k\}$, each subtask $s_i$ is assigned to $a_j$ if $C_{ij}=1$ (agent can handle subtask), subject to binary assignment constraints. A cost function $w_{ij}$ (e.g., based on latency or load) can be optionally introduced for global optimization:

$$\text{minimize} \quad \sum_{i,j} w_{ij} \cdot x_{ij} \quad \text{subject to} \quad \sum_j x_{ij} = 1,\, x_{ij} \in \{0,1\},\, C_{ij}=1 \implies x_{ij} \in \{0,1\}$$

The prototype implementation assigns subtasks greedily by selecting agents with maximized capability scores. Plans for global, learned optimization are identified as a potential extension (Liao et al., 8 Jul 2025).

4. Multimodal Processing and Information Retrieval

AgentMaster's multimodal capabilities are realized through coordinated handling of text, structured data, and images:

• User-uploaded images are base64-encoded and routed via the front end to the Coordinator and then the Image Agent.
• The Image Agent’s MCP box calls an appropriate vision API (e.g., a contour-map interpreter), extracting quantitative features and captions.
• Output, such as “Detected 23 contour lines; average spacing is 5 m. Areas above 100 m elevation correspond to…,” is formatted as an MCP assistant message, relayed back to the Coordinator, integrated, and synthesized into an accessible multimodal response.

Example: given an NDE (non-destructive evaluation) contour map, the Image Agent might identify high-risk corrosion zones, reporting findings in precise language and quantitative terms sourced from vision analysis (Liao et al., 8 Jul 2025).

5. Interface Design and API Access

The client-facing front end is a web-based (React) chat interface served by Flask. The API uses JSON‐RPC over HTTP POST (/api/chat), accepting messages of types “text”, “image”, or “audio” packaged by user/session ID. Responses are streamed back with role and content fields, optionally including attachments for images, tables, or charts.

No prior technical setup or user training is needed; AgentMaster's end users interact naturally, without knowledge of the underlying MAS protocols or structure (Liao et al., 8 Jul 2025).

6. Empirical Evaluation and Results

Experimental Setup

• Domains: Federal Highway Administration (FHWA) bridge database, public IR corpus, vision image APIs.
• 23 queries across SQL, IR, general QA, image/complex QA.
• Agents use GPT-4o mini either locally or through OpenAI API.

Metrics

• BERTScore F1: $F_1 = 2 \cdot (P \cdot R)/(P + R)$, P/R are token-embedding precision/recall.
• G-Eval: LLM-as-a-Judge percentage correctness, as described by Liu et al. (2023).
• Human validation on decomposition, routing, and final answers.

Query Type	G-Eval (%)	BERT P (%)	BERT F1 (%)
SQL Queries	92.0	98.8	98.7
IR Queries	90.2	97.6	97.8
General QA	84.0	95.7	96.8
Image/Complex QA	82.0	90.1	91.9
Average	87.1	95.6	96.3

AgentMaster is the only system among those compared that supports dynamic coordination, flexible shared memory, meaningful fallback strategies, and hybrid task allocation (Liao et al., 8 Jul 2025).

7. Contributions, Case Studies, and Extensions

Key contributions of AgentMaster include the first end-to-end MAS unifying both A2A and MCP, demonstrating robust query decomposition, dynamic routing, and multimodal answer synthesis. Notably, complex queries—e.g., obtaining bridge definitions and statistics, or extracting structured analyses from contour maps—require multi-stage decomposition, subtask orchestration, and integration across domain agents.

Proposed future extensions include replacing the current greedy routing heuristic with a learned optimization approach, layering security and authorization onto MCP, broadening support for diverse multimodal toolboxes (e.g., audio transcription, video analysis), and adapting AgentMaster for domain-specific workflows in finance, biomedicine, or supply chain contexts (Liao et al., 8 Jul 2025).

Adoption of end-to-end reinforcement learning (RL) strategies from frameworks such as MetaAgent-X represents a promising direction for enabling self-designing, self-executing MAS platforms. By transplanting script-based MAS generation, hierarchical rollout credit assignment, and stagewise designer–executor co-evolution, AgentMaster could evolve beyond static orchestration, allowing continual improvement and adaptation. Open challenges include managing increased rollout compute overhead, stabilizing reward signals, and mitigating training non-stationarity as agent workflows themselves become learnable (Zhang et al., 14 May 2026).

This suggests AgentMaster's architectural philosophy and protocol integration position it as a reference design for scalable, extensible, and robust MAS frameworks in rapidly evolving AI-oriented domains.