Symphony: A Decentralized Multi-Agent Framework for Scalable Collective Intelligence (2508.20019v1)

Abstract: Most existing LLM-based agent frameworks rely on centralized orchestration, incurring high deployment costs, rigid communication topologies, and limited adaptability. To address these challenges, we introduce Symphony, a decentralized multi-agent system which enables lightweight LLMs on consumer-grade GPUs to coordinate. Symphony introduces three key mechanisms: (1) a decentralized ledger that records capabilities, (2) a Beacon-selection protocol for dynamic task allocation, and (3) weighted result voting based on CoTs. This design forms a privacy-saving, scalable, and fault-tolerant orchestration with low overhead. Empirically, Symphony outperforms existing baselines on reasoning benchmarks, achieving substantial accuracy gains and demonstrating robustness across models of varying capacities.

• The paper introduces a decentralized framework that overcomes centralized limitations by employing beacon-based agent selection and weighted result voting.
• The methodology leverages a decentralized ledger for dynamic agent discovery and achieves up to 41.6% accuracy improvement on benchmark tests.
• The framework scales on heterogeneous, consumer-grade devices and preserves data privacy while offering fault tolerance and efficient orchestration.

Symphony: A Decentralized Multi-Agent Framework for Scalable Collective Intelligence

Motivation and Context

The Symphony framework addresses the limitations of centralized LLM-based agent orchestration, which include high deployment costs, rigid communication topologies, and limited adaptability. By leveraging decentralized coordination among lightweight LLM agents on heterogeneous edge devices, Symphony enables scalable, privacy-preserving, and fault-tolerant collective intelligence. The system is designed to operate efficiently on consumer-grade hardware, facilitating broader accessibility and compliance with privacy regulations.

Architecture and Core Mechanisms

Symphony introduces three principal mechanisms to achieve decentralized orchestration:

1. Decentralized Ledger: Each agent's capabilities and availability are recorded in a distributed ledger, indexed by cryptographic addresses. This enables dynamic discovery and selection of agents based on their expertise and resource status.
2. Beacon-Based Agent Selection: Task allocation is performed via a Beacon protocol, where sub-task requirements are broadcast and agents respond with capability match scores. The executor for each sub-task is selected based on the highest match score, ensuring optimal allocation and robustness against node failures.
3. Weighted Result Voting: Multiple planning agents independently generate diverse chains-of-thought (CoTs) for each user query. Final answers from each CoT are aggregated via weighted majority voting, where weights are derived from the confidence scores accumulated during sub-task execution. This mitigates individual agent errors and enhances overall reliability.

   Figure 1: Overview of Symphony’s decentralized pipeline, illustrating query decomposition, Beacon-based agent selection, and result voting across reasoning paths.

Execution Pipeline

Upon receiving a user query, Symphony broadcasts the task to multiple planning agents, each generating a unique decomposition into executable sub-tasks. For each sub-task, a Beacon is broadcast to all available agents, who compute capability match scores using a similarity function (e.g., cosine similarity between capability and requirement vectors). The agent with the highest score executes the sub-task, passing results and context to subsequent executors in the chain.

This process is repeated for each chain-of-thought, resulting in multiple independent reasoning trajectories. The final answers are then aggregated via weighted voting, with confidence scores reflecting the quality of agent-task matches throughout execution.

Figure 2: Symphony pipeline on a BBH case, showing three planning agents generating diverse CoTs for enhanced solution diversity.

Experimental Results

Effectiveness

Symphony demonstrates substantial accuracy improvements over both direct LLM solving and centralized multi-agent frameworks (AutoGen, CrewAI) on Big-Bench-Hard (BBH) and AMC benchmarks. For instance, on BBH, Symphony achieves up to 41.6% absolute accuracy gain over direct solving and up to 29.1% over AutoGen. On AMC, Symphony outperforms all baselines, with up to 4.46% higher accuracy than AutoGen and 7.41% over direct solving. These results validate the efficacy of decentralized orchestration and multi-CoT voting.

Scalability Across Models

Symphony consistently improves performance across heterogeneous LLMs (Deepseek-7B-instruct, Mistral-7B-instruct-v0.3, Qwen2.5-7B-instruct). Notably, the accuracy gap between models narrows under Symphony, indicating that the framework particularly benefits weaker models and supports heterogeneous device environments.

Robustness

Ablation studies reveal that multi-CoT voting yields BBH gains of +5.3% to +6.2% and AMC gains of +0.72% to +2.63%. Beacon score-based selection outperforms random allocation, with BBH gains of +4.1% to +4.3% and AMC gains of +0.60% to +2.18%. These mechanisms enhance fault tolerance and ensure optimal task-agent matching.

Orchestration Overhead

Symphony’s orchestration mechanisms (ledger registration, beacon broadcast, result voting) contribute less than 5% to total inference latency, confirming negligible overhead relative to model inference.

Implementation Details

System Components

• Worker Nodes: Edge devices with quantized LLMs, stage-specific prompts, and secure communicators. Local inference reduces latency and resource consumption.
• Gateways: Standardized APIs for agent registration, communication, and task execution. Message types include Beacon, Beacon Response, Task, and Task Result.

Prompt Design

• Problem Decomposition Prompt: Used by planning agents to break down complex queries into strictly computable sub-questions in JSON format.
• Subtask Execution Prompt: Used by executor agents to solve individual sub-tasks based on provided context, outputting answers in a standardized format.

Deployment Considerations

Symphony is optimized for consumer-grade GPUs (e.g., RTX 4090, Apple M-series), enabling deployment on local infrastructure. The decentralized design supports privacy preservation, as sensitive data remains on local devices and only minimal sub-task results are shared. This is particularly relevant for domains with strict data governance requirements (e.g., healthcare, finance).

Societal and Theoretical Implications

Symphony’s decentralized paradigm lowers hardware and infrastructure barriers, democratizing access to collective intelligence. The privacy-preserving architecture aligns with regulatory frameworks (HIPAA, GDPR), enabling secure multi-institutional collaborations. The agent-level autonomy and incentive mechanisms support the emergence of decentralized agent economies, where agents bid for tasks and exchange digital tokens for services, fostering adaptive and self-sustaining ecosystems.

Theoretically, Symphony advances the paper of distributed problem solving and planning in multi-agent systems, integrating concepts from distributed systems, blockchain, and collective intelligence. The framework’s robustness and scalability suggest promising directions for future research in decentralized AI, including federated learning, distributed RL, and market-based agent coordination.

Future Directions

Potential future developments include:

• Integration with federated learning for decentralized model training.
• Extension to larger agent networks with dynamic topology adaptation.
• Incorporation of economic incentives and smart contracts for agent collaboration.
• Application to privacy-sensitive domains (e.g., medical diagnostics, legal reasoning).
• Exploration of gossip-based communication and fault detection for enhanced resilience.

Conclusion

Symphony provides a robust, scalable, and privacy-preserving framework for decentralized multi-agent orchestration of LLMs on heterogeneous edge devices. Empirical results demonstrate significant accuracy gains, reduced infrastructure costs, and enhanced robustness compared to centralized approaches. The framework’s design supports broad accessibility, compliance with privacy regulations, and the emergence of decentralized agent economies, marking a substantial advancement in collective intelligence and distributed AI systems.