AgentBeats: Multi-Context Agentic Systems

• AgentBeats is a multifaceted term referring to distinct multi-agent systems for assessment, music recommendation, and beat generation.
• It leverages standardized A2A and MCP protocols to reduce integration complexity and enable reproducible, fair comparisons.
• The framework demonstrates practical applications in agent evaluation, personalized content delivery, and creative beat workflows with clear attribution mechanisms.

AgentBeats is a name used in recent arXiv literature for multiple agentic systems rather than a single canonical platform. In its most explicit and formal usage, AgentBeats denotes a concrete realization of Agentified Agent Assessment (AAA), a framework in which “judge agents” assess “subject agents” through standardized protocols, namely Agent-to-Agent (A2A) for task management and Multi-Channel Protocol (MCP) for tool access. The same name is also used for a CrewAI-based multi-agent music recommendation pipeline, and a related “AgentBeats-style Beat Generator” appears as a beat-making specialization of a content-based Music AI Agent architecture centered on attribution, provenance, and micro-settlement (Liu et al., 11 Jun 2026, Boadana et al., 7 Aug 2025, Kim et al., 23 Oct 2025).

1. Nomenclature and research contexts

The term AgentBeats spans at least three research contexts.

Usage	Domain	Core characterization
AgentBeats	Agent assessment	Concrete realization of AAA using A2A and MCP
“AgentBeats”	Music recommendation	Multi-agent personalized recommendation pipeline in CrewAI
“AgentBeats-style Beat Generator”	Beat-making infrastructure	Specialization of a content-based Music AI Agent architecture

In the agent-assessment literature, AgentBeats is presented as a practical system for open, standardized, and reproducible evaluation. In the music-recommendation literature, the same label denotes a collaborative multi-agent pipeline that combines LLM reasoning with content-based signals. In music-creation infrastructure, the expression “AgentBeats-style Beat Generator” is used more loosely, referring to a beat-oriented adaptation of a block-based, provenance-aware Music AI Agent architecture. This suggests that “AgentBeats” functions both as a specific system name and as a broader label for agentic workflows in music-adjacent settings (Liu et al., 11 Jun 2026, Boadana et al., 7 Aug 2025, Kim et al., 23 Oct 2025).

A useful clarification follows from this dispersion of usage: AgentBeats is not, in the available literature, a single unified software stack spanning evaluation, recommendation, and beat generation. Rather, it is a recurrent name attached to distinct systems that share an emphasis on agent decomposition, standardized interfaces, or workflow modularization.

2. AgentBeats as a realization of Agentified Agent Assessment

In "AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility" (Liu et al., 11 Jun 2026), the system is defined as a concrete realization of Agentified Agent Assessment. The motivating problem is that conventional benchmarking requires bespoke integration between each benchmark and each agent, producing $O(N \times M)$ engineering effort, brittle harnesses, test–production mismatch, and low reproducibility. AAA replaces this with a single standardized interaction surface in which benchmarks themselves become judge agents and assessed systems become subject agents.

The central conceptual move is to separate assessment logic entirely from agent implementations. Under this design, judge agents are responsible for data selection, environment setup, tool provisioning, timeout handling, and scoring, while subject agents need only expose a standard A2A service and an MCP client. The paper argues that this reduces the integration burden from $O(N \times M)$ to $O(N + M)$, because each benchmark need only implement the standard protocol once, and each agent need only speak the same protocol once (Liu et al., 11 Jun 2026).

The formal interface is expressed through A2A messages of the form

$$\mathsf{Msg} = \bigl(\mathsf{id},\,\mathsf{sender},\,\mathsf{receiver},\,\mathsf{type},\,\mathsf{payload}\bigr),$$

with key message types \texttt{StartEval}, \texttt{TaskInstruct}, \texttt{TaskResult}, and \texttt{EvalReport}. A single-agent state transition is specified as

$$\langle \text{Idle} \rangle \xrightarrow{\text{StartEval}} \langle \text{Assigned} \rangle \xrightarrow{\text{TaskInstruct}} \langle \text{Running} \rangle \xrightarrow{\text{TaskResult}} \langle \text{Completed} \rangle \xrightarrow{\text{EvalReport}} \langle \text{Idle} \rangle.$$

Tool use is delegated through MCP, modeled as

$$\mathsf{mcp\_call}(\mathsf{tool\_id},\,\mathsf{arguments}) \longrightarrow \mathsf{result\_object}.$$

The paper’s design claim is that assessment can therefore be aligned with production-grade protocols rather than benchmark-specific harnesses (Liu et al., 11 Jun 2026).

3. Unified interface architecture and operation modes

AgentBeats reduces evaluation to a single interface on both sides: A2A for orchestration and MCP for tool use. The judge agent prepares an environment, launches an MCP server exposing the tools required for the task, sends a \texttt{TaskInstruct} message containing task data and the MCP endpoint, waits for a \texttt{TaskResult} or timeout, revokes tool access, computes metrics, and returns a report. Subject agents implement only two capabilities: receiving \texttt{TaskInstruct} and returning \texttt{TaskResult} via A2A, and connecting to the provided MCP endpoint to invoke tools (Liu et al., 11 Jun 2026).

This architecture is operationalized through five modes intended to balance openness, privacy, and reproducibility. In Local Mode, all judge and subject agents run on a single local host; privacy is closed, openness is low, and reproducibility is described as ad-hoc script. In Remote Mode, judge and subject agents are registered as A2A service URLs on a central platform; privacy is maintained through platform-mediated calls and reproducibility is supported by central logs. In Hosted Mode, developers submit agent blueprints such as Git repos or Docker images and the platform instantiates containerized A2A services; openness is high and reproducibility is platform-managed. In Proxy Mode, a subject agent remains under local active development while judge agents run remotely, allowing local iteration without pushing code. In CI Mode, the entire AAA workflow is embedded in a CI pipeline such as GitHub Actions, with reproducibility characterized as high through CI logs and artifacts (Liu et al., 11 Jun 2026).

The significance of these modes is methodological rather than merely infrastructural. They are meant to show that standardized assessment is compatible with heterogeneous deployment constraints, including local-only development, private organizational evaluation, public competitions, and audit-friendly continuous integration. A plausible implication is that the paper treats “reproducibility” not as a single property but as a function of runtime topology, artifact retention, and interface standardization.

4. Empirical studies, fidelity, and harness effects

The empirical validation of AgentBeats combines a community-scale field study with a controlled coding-agent case study. The field study ran as a five-month open competition and included 298 judge agents across 12 categories together with 467 subject agents from independent participants. Categories included tool use, coding, web, healthcare, multi-agent games, and cybersecurity. Submissions were made via GitHub repos in CI mode, and each judge agent defined its own per-subject scoring logic and published a leaderboard endpoint (Liu et al., 11 Jun 2026).

The statistical analysis measured coverage, adoption friction, multi-language support, and iteration volume. Specifically, it reported median LOC of judge code as 5.3 k Python LOC, prompt usage as 78 % of repos containing approximately 241 LOC of prompts and about 5 % of characters, non-Python support at 2 %, and an example iteration volume in which Tau2-Bench saw 347 runs from 42 agents. The reported outcomes were that dozens of existing benchmarks were agentified with no new protocol, that agents written in TypeScript, Rust, and Python could participate with low LOC wrappers, that many repositories were public with preserved CI logs, and that a single subject agent could enter any judge’s leaderboard (Liu et al., 11 Jun 2026).

The controlled case study focused on coding agents. The systems under test were Claude Opus 4.7 with Claude Code harness, GPT-5.4 with Codex CLI harness, Gemini 3.1 Pro with OpenCode harness, and Qwen3.5 397B-A17B with mini-SWE-Agent harness. Benchmarks were DevEval with 1,222 function-generation tasks, SWE-Bench Pro with 731 repo-level issue tasks, and Terminal-Bench 2.0 with 89 CLI tasks. All systems ran under AAA’s A2A+MCP design pattern, with judge agents preparing container sandboxes and either injecting subject-agent executables or exposing an MCP SSH shell on failure. Prompts, timeouts, and per-instance budgets were matched to typical production settings (Liu et al., 11 Jun 2026).

The primary solve-rate metric was

$$R = \frac{\# \text{ successful instances}}{\text{total instances}}.$$

Results showed the following. On DevEval, GPT-5.4/Codex reached 94.8 %, Claude Opus 4.7/Claude Code 91.9 %, Gemini 3.1 Pro/OpenCode 88.8 %, and Qwen3.5 397B-A17B/MSA 71.4 %. On SWE-Bench Pro, Claude Opus 4.7/Claude Code reached 69.1 %, GPT-5.4/Codex 62.1 %, Gemini/OpenCode 54.4 %, and Qwen3.5/MSA 45.8 %. On Terminal-Bench 2.0, Claude Opus 4.7/Claude Code reached 68.5 %, GPT-5.4/Codex 67.4 %, Gemini/OpenCode 65.2 %, and Qwen3.5/MSA 36.0 % (Liu et al., 11 Jun 2026).

A major interpretive point concerns fidelity. The paper reports that public vendor scores sometimes varied because of proprietary harnesses, memorization filters, or “thinking” settings. Under a standardized pipeline, rankings stabilized; for example, Claude Opus 4.7 outperformed GPT-5.4 on Terminal-Bench 2.0 by 1.1 percentage points under unified settings, reversing a 6 percentage point gap reported elsewhere. At the same time, the paper does not claim that standardization erases harness effects. A harness-swapping experiment found a native-harness co-adaptation advantage averaging 5.3 percentage points, formalized as

$$\Delta R(m,h^\dagger;\,h') = R_{m,h^\dagger} - R_{m,h'}.$$

This suggests that AgentBeats supports fairer comparison, but also that harness choice remains an intrinsic part of agent behavior rather than a negligible implementation detail (Liu et al., 11 Jun 2026).

5. AgentBeats as a multi-agent music recommendation system

In "LLM-Based Intelligent Agents for Music Recommendation: A Comparison with Classical Content-Based Filtering" (Boadana et al., 7 Aug 2025), AgentBeats denotes a collaborative multi-agent personalized music recommendation system implemented in the CrewAI framework. Agents run as independent microservices in Docker containers and communicate over REST/JSON, while a central controller orchestrates each request by invoking agents sequentially and passing structured JSON messages. The architecture comprises ReadingAgt, AnalistAgt, ExtractAgt, RecommendAgt, and an implicit FeedbackAgt used in evaluation (Boadana et al., 7 Aug 2025).

The roles are tightly partitioned. ReadingAgt loads and returns the music catalogue of up to 300 tracks through GetMusicCatalogueTool, calling MongoDB or the Spotify API. AnalistAgt loads and returns a specific user’s listening history of top 30 tracks through GetUserHistoryDataTool. ExtractAgt analyzes the history to infer the top-5 preferred genres or artists through LLM reasoning over JSON. RecommendAgt generates a ranked list of 20 personalized tracks and labels each with “known” and “liked” flags. FeedbackAgt collects user feedback such as “like”/“dislike”, “known”/“unknown”, and playlist rating via the front end and computes evaluation metrics (Boadana et al., 7 Aug 2025).

LLM allocation is asymmetric. Gemini 2.0 Flash is used by ReadingAgt and RecommendAgt when large context windows are needed, including up to 1 M tokens for the entire catalogue and user profile. LLaMA-3.3-70B-VERSATILE is used by AnalistAgt and ExtractAgt for deeper semantic inference over the user’s listening history. Prompting follows zero-shot templates with four sections—Goal, Backstory, Input specification, and Output schema—and all prompts use the crew.ai Tools interface so that tool calls appear as simple function invocations (Boadana et al., 7 Aug 2025).

Recommendation combines classical and LLM-derived signals. Each track is represented both as a sparse TF-IDF vector over the 20 most frequent genres and as a dense embedding from the LLM’s penultimate layer. User–item matching is computed through a linear blend of content- and LLM-based cosine similarities,

$$\mathrm{Score}(u,i) = \alpha \,\cos\!\bigl(\mathbf{v}_u^{\mathrm{TFIDF}}, \mathbf{v}_i^{\mathrm{TFIDF}}\bigr) + (1-\alpha)\,\cos\!\bigl(\mathbf{e}_u^{\mathrm{LLM}}, \mathbf{e}_i^{\mathrm{LLM}}\bigr),$$

with novelty quantified as the fraction of recommended tracks not previously heard and diversity encouraged by penalizing overly similar tracks through a threshold $\tau$ on pairwise cosine similarity among candidates (Boadana et al., 7 Aug 2025).

Evaluation used 13 months of real Spotify usage from 19 consenting users, comprising 22,178 unique tracks. The working catalogue was subsampled to 300 tracks across the 20 most frequent genres, with per-user histories consisting of the top 30 most-played tracks. The baseline was classical content-based filtering using purely TF-IDF and cosine similarity. Metrics for playlists of $O(N \times M)$0 recommendations included Like Rate, Novelty Rate, Successful Novelty Rate, playlist rating on a 0–10 scale, and average inference time in seconds (Boadana et al., 7 Aug 2025).

The reported mean ± std over 10 blind users reveals a sharp trade-off. Traditional content-based filtering achieved $O(N \times M)$1 Like Rate, $O(N \times M)$2 Novelty Rate, $O(N \times M)$3 Successful Novelty Rate, rating $O(N \times M)$4, and time $O(N \times M)$5 s. LLaMA-based agents achieved $O(N \times M)$6 Like Rate, $O(N \times M)$7 Novelty Rate, $O(N \times M)$8 Successful Novelty Rate, rating $O(N \times M)$9, and time $O(N + M)$0 s. Gemini-based agents achieved $O(N + M)$1 Like Rate, $O(N + M)$2 Novelty Rate, $O(N + M)$3 Successful Novelty Rate, rating $O(N + M)$4, and time $O(N + M)$5 s (Boadana et al., 7 Aug 2025).

The interpretive conclusion in the paper is specific: LLaMA agents yield the highest user satisfaction but at the cost of novelty and latency; traditional content-based filtering is fast and discovers new tracks more often but scores lower on user appeal; Gemini falls in between. This makes the system relevant not only as an application of multi-agent LLM orchestration, but also as a case study in how agent decomposition can improve subjective preference alignment without necessarily improving discovery or serving-time efficiency.

6. AgentBeats-style beat generation, attribution, and post-streaming music infrastructure

A third usage appears in "From Generation to Attribution: Music AI Agent Architectures for the Post-Streaming Era" (Kim et al., 23 Oct 2025), where the paper describes how the same architectural principles can be specialized for beat-making in an “AgentBeats-style Beat Generator.” The parent architecture is a content-based Music AI Agent system intended to transform music from a static, pre-released catalog into an adaptive, session-based ecosystem in which creation, consumption, and monetization are unified. Its central mechanism is the decomposition of music into atomic Blocks stored in BlockDB, with every retrieval event triggering attribution and settlement (Kim et al., 23 Oct 2025).

A Block is the atomic unit of music and may be an instrumental stem such as drums, bass, piano, or lead synth, or a structural segment such as intro, verse, or chorus. Each Block is categorized along a timbral axis and a temporal axis. BlockDB stores raw audio or pre-extracted stems, metadata including BPM, key, bar length, and creator ID, descriptive text captions, multi-modal embeddings, and provenance links to the original work or artist. Retrieval combines vector similarity search, such as HNSW over embeddings, with inverted indices over metadata fields (Kim et al., 23 Oct 2025).

The paper gives the single-Block scoring function as

$O(N + M)$6

where similarity is cosine similarity and metadata indicators contribute additional weight. Retrieval of top-$O(N + M)$7 Blocks conditions downstream generation, while the Attribution Layer emits an event

$O(N + M)$8

for each use. If a session has royalty pool $O(N + M)$9 and total weight $$\mathsf{Msg} = \bigl(\mathsf{id},\,\mathsf{sender},\,\mathsf{receiver},\,\mathsf{type},\,\mathsf{payload}\bigr),$$0, the corresponding payment is

$$\mathsf{Msg} = \bigl(\mathsf{id},\,\mathsf{sender},\,\mathsf{receiver},\,\mathsf{type},\,\mathsf{payload}\bigr),$$1

Settlement may occur through smart contracts or off-chain micro-payments (Kim et al., 23 Oct 2025).

The orchestration layer is explicitly multi-agent. IntentAgent parses a high-level user prompt and produces a DAG of sub-tasks while selecting tools; QueryAgent translates conditions into BlockDB queries, retrieves top-$$\mathsf{Msg} = \bigl(\mathsf{id},\,\mathsf{sender},\,\mathsf{receiver},\,\mathsf{type},\,\mathsf{payload}\bigr),$$2 Blocks, and emits attribution events; ExecutionAgent invokes generative or analytical tools in the order specified by IntentAgent; the system then returns audio and a natural-language summary, accepts refinement prompts, and repeats the cycle until the user is satisfied (Kim et al., 23 Oct 2025).

The beat-making specialization inherits these principles but redefines the units and search space. Beat-Blocks are short loops or one-shot hits annotated with groove, swing percentage, genre, and tempo. BlockDB is indexed by groove embeddings such as rhythmic AutoEncoder features and tempo metadata. QueryAgent retrieves matched Beat-Blocks, stem-generation tools are specialized for percussion layering, fill generation, and loop variation, the Attribution Layer logs per-bar or per-hit usage and settles micro-payments to drum-loop authors, and IntentAgent scripts beat workflows such as kick/snare to hi-hat layering to variation to FX pass (Kim et al., 23 Oct 2025).

Within this lineage, AgentBeats-style beat generation is less a named standalone benchmarked product than a design pattern: decomposable music units, rich retrieval, agentic orchestration, and attribution that is embedded directly into the creative workflow. The paper’s broader claim is that such a system can support a “Fair AI Media Platform” in which provenance is auditable, rights management is clearance-aware, and compensation is tied deterministically to actual reuse rather than pooled royalty estimates (Kim et al., 23 Oct 2025).