Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures

Abstract: How much autonomy can multi-agent LLM systems sustain -- and what enables it? We present a 25,000-task computational experiment spanning 8 models, 4--256 agents, and 8 coordination protocols ranging from externally imposed hierarchy to emergent self-organization. We observe that autonomous behavior already emerges in current LLM agents: given minimal structural scaffolding (fixed ordering), agents spontaneously invent specialized roles, voluntarily abstain from tasks outside their competence, and form shallow hierarchies -- without any pre-assigned roles or external design. A hybrid protocol (Sequential) that enables this autonomy outperforms centralized coordination by 14% (p<0.001), with a 44% quality spread between protocols (Cohen's d=1.86, p<0.0001). The degree of emergent autonomy scales with model capability: strong models self-organize effectively, while models below a capability threshold still benefit from rigid structure -- suggesting that as foundation models improve, the scope for autonomous coordination will expand. The system scales sub-linearly to 256 agents without quality degradation (p=0.61), producing 5,006 unique roles from just 8 agents. Results replicate across closed- and open-source models, with open-source achieving 95% of closed-source quality at 24x lower cost. The practical implication: give agents a mission, a protocol, and a capable model -- not a pre-assigned role.

• The paper introduces the endogeneity paradox and shows that a sequential protocol with autonomous role selection achieves up to 44% higher quality than traditional methods.
• Experimental results across 25,000+ tasks and up to 256 agents demonstrate that balancing imposed structure with self-organization optimizes scalability and cost efficiency.
• Findings indicate that integrating capable open-source LLMs with hybrid protocol designs can nearly match state-of-the-art performance at a fraction of the cost.

Self-Organization Trumps Pre-Designed Structure in LLM Multi-Agent Systems

Introduction

The study under review provides an extensive computational analysis of coordination architectures within multi-agent LLM systems, comparing 8 models, 4--256 agents, and 8 protocols with 25,000+ tasks. The paper's core thesis is the identification of an "endogeneity paradox," establishing that neither full exogenous control (pre-designed hierarchies, central assignment) nor full endogenous autonomy (unconstrained agent action) optimally exploits the capabilities of LLM-based agents. Instead, a hybrid protocol—specifically, the Sequential protocol, characterized by fixed agent order but autonomous role selection—drives superior quality, scalability, and emergent collective intelligence.

Experimental Framework

The experimental approach systematically varies the axes of agent capability (spanning cutting-edge closed and open-source LLMs), team size (from 4 to 256), and coordination protocol (ranging from centralized "Coordinator" to fully decentralized "Shared"). Task complexity is stratified into four levels (L1–L4), with L4 presenting adversarial, multi-stakeholder scenarios. Evaluation is rigorous: a multi-criteria LLM-as-judge protocol ensures cross-comparison validity, controlling for model-specific biases and standardizing the quality measure ($Q \in [0.25, 1.0]$).

The Endogeneity Paradox and Protocol Impact

The headline result is the discovery of the endogeneity paradox. Empirical results demonstrate that the Sequential protocol outperforms both fully centralized and fully decentralized approaches, with a 44% quality improvement over the Shared protocol (Cohen’s $d=1.86$, $p<0.0001$) and a 14% improvement over the Coordinator protocol ($p<0.001$), robust across multiple state-of-the-art LLMs.

Figure 1: The Sequential protocol consistently yields the highest solution quality relative to Coordinator, Broadcast, and Shared, in both pilot and final settings.

The Sequential protocol, leveraging sequential observation of completed actions and autonomous role-selection, achieves an optimal balance between imposed structure and adaptive self-organization. This outcome is not monotonic with respect to the "autonomy" axis: excessive centralization stifles adaptation, while maximal autonomy induces redundancy and lack of coordination.

Scaling, Cost, and Emergent Properties

Scaling analyses underscore that system performance remains stable as agent numbers rise by two orders of magnitude. From $N=8$ to $N=256$, quality remains invariant ($p=0.61$), while marginal cost increases sub-linearly, and organizational adaptation accelerates.

Figure 2: Solution quality remains stable and cost scales sub-linearly with an 8x increase in agent count.

The system exhibits nontrivial emergent phenomena under the Sequential protocol:

• Dynamic role invention is evidenced by the growth of unique roles (e.g., over 5,000 unique role names at $N=8$).
• Voluntary self-abstention appears as agents independently withdraw from tasks for which they assess they have insufficient value-add, and this correlates with increased quality.
• Hierarchy depth increases with task complexity, implicating context-sensitive structural adaptation rather than rigid hierarchy.

  Figure 3: Increasing task complexity correlates with greater emergent hierarchy depth, with marked quality degradation in adversarial (L4) settings.

  

  Figure 4: Role assignment heatmap demonstrates extreme fluidity and task-specific reinvention of roles, confirming minimal specialization entrenchment.

  

  Figure 5: Participation quality is highest when abstention is voluntary; endogenous abstention correlates to quality gains over exogenous exclusion.

  

  Figure 6: Role specialization (RSI) amplifies with scale, while graph spectral gap remains stable, indicating robust connectivity and structural resilience.

Model Capability, Open/Closed-Source, and the Self-Organization Threshold

Among high-capability LLMs, protocol choice dominates performance variation ($\sim$44%) relative to model choice ($\sim$14%). However, model capability is a precondition for effective self-organization. Models below a capability threshold (e.g., GLM-5) show performance inversions: rigid, exogenously assigned roles outperform self-organization, primarily due to deficiencies in self-reflection and protocol adherence. Crucially, strong open-source models (DeepSeek v3.2, GLM-5) achieve up to 95% of closed-source (Claude Sonnet 4.6) quality at vastly reduced cost (up to $d=1.86$0 cheaper), validating the economic viability of multi-model, cost-efficient deployments.

Theoretical and Practical Implications

Emergence of AI-Native Organization

Findings directly contradict the practice of rigid, pre-assigned role design, which reflects human institutional patterns rather than computational utility. LLM agents exhibit negligible switching costs and benefit from fluid, endogenous specialization. As such, roles must be interpreted as context-dependent computational affordances rather than static organizational positions.

Coordination Protocols as Multipliers

Protocol selection acts as a force-multiplier on collective intelligence. The paper provides a constitutional framework recommending immutable human-set mission/values, system-proposed (human-approved) standards, and fully autonomous protocol optimization focused on the "how" rather than "why."

Practical Recommendations

1. Specify mission and values; avoid static role assignment.
2. Select coordination protocol (prefer Sequential or similar hybrid variants).
3. Prioritize capable models over team size for complex work.
4. Optimize agent composition using cost-efficient open-source models for routine tasks and reserve state-of-the-art models for high-complexity domains.

Limitations and Future Work

The study's reliance on LLM-as-judge methods introduces the potential for systematic biases; further validation with human evaluators is warranted. Synthetic task design and fixed latency of sequential coordination highlight the need for real-world deployment and batched variants of Sequential to further optimize latency while preserving informational advantages. A follow-up is indicated for bio-inspired protocols and constitutional governance mechanisms.

Conclusion

This work establishes that optimal coordination in LLM multi-agent systems depends on hybrid protocols enabling self-organization within minimal structural constraints, rather than mimicking human-designed hierarchy or maximizing agent autonomy. Strong model capability is a necessary precondition, but protocol choice drives the largest performance gains in capable systems. The findings provide a rigorous empirical foundation for AI-native organizational design principles, recommending that system architects eliminate rigid roles, select protocols that blend structure and autonomy, and compose agent teams with both effectiveness and economic efficiency in mind.

The implications extend broadly to the design of scalable, adaptive, and resilient collective intelligence architectures in AI, signaling a shift toward endogenous, dynamically constructed organization over static, human-inspired control structures.