Autonomous Agents for Collaborative Task under Information Asymmetry

Abstract: LLM Multi-Agent Systems (LLM-MAS) have achieved great progress in solving complex tasks. It performs communication among agents within the system to collaboratively solve tasks, under the premise of shared information. However, when agents' collaborations are leveraged to perform multi-person tasks, a new challenge arises due to information asymmetry, since each agent can only access the information of its human user. Previous MAS struggle to complete tasks under this condition. To address this, we propose a new MAS paradigm termed iAgents, which denotes Informative Multi-Agent Systems. In iAgents, the human social network is mirrored in the agent network, where agents proactively exchange human information necessary for task resolution, thereby overcoming information asymmetry. iAgents employs a novel agent reasoning mechanism, InfoNav, to navigate agents' communication toward effective information exchange. Together with InfoNav, iAgents organizes human information in a mixed memory to provide agents with accurate and comprehensive information for exchange. Additionally, we introduce InformativeBench, the first benchmark tailored for evaluating LLM agents' task-solving ability under information asymmetry. Experimental results show that iAgents can collaborate within a social network of 140 individuals and 588 relationships, autonomously communicate over 30 turns, and retrieve information from nearly 70,000 messages to complete tasks within 3 minutes.

• The paper introduces iAgents, a novel LLM-driven multi-agent system that employs plan-based communication (InfoNav) to tackle tasks with private, distributed information.
• It integrates a hybrid memory system combining structured and semantic retrieval, yielding measurable performance gains in large-scale, privacy-preserving scenarios.
• Experimental analysis reveals that recursive communication and explicit planning enable agents to collaborate over 30 turns, though current models show moderate task accuracy.

Autonomous Agents for Collaborative Tasks under Information Asymmetry

Introduction and Motivation

The paper introduces iAgents, a novel paradigm for LLM-driven Multi-Agent Systems (MAS) designed to address collaborative tasks in environments characterized by information asymmetry. Unlike prior LLM-MAS frameworks, which assume a shared context among agents, iAgents explicitly model the mapping between human social networks and agent networks, where each agent only has access to the private information of its associated human user. This design reflects real-world constraints on privacy and information distribution, making the framework more applicable to practical human-agent collaboration scenarios.

Figure 1: Comparison between previous MAS (shared context) and iAgents (information asymmetry per agent).

The core challenge addressed is enabling effective agent collaboration when no agent has global visibility, and information must be proactively exchanged to resolve distributed tasks. The paper proposes architectural and algorithmic innovations to facilitate this, and introduces InformativeBench, a benchmark specifically designed to evaluate agent collaboration under information asymmetry.

iAgents Architecture and Mechanisms

The iAgents framework is structured to mirror human social networks, equipping each individual with an agent that acts on their behalf. When a collaborative task arises, the relevant agents are summoned, each starting with only the information visible to their human user. Agents then autonomously communicate, exchanging information as needed, and perform a consensus check to finalize the solution.

Figure 2: Overall architecture of iAgents, illustrating agent instantiation, communication, information exchange, and consensus.

InfoNav: Communication Navigation via Explicit Planning

A central innovation is the InfoNav mechanism, which guides agent communication through explicit, plan-based reasoning. Rather than relying on implicit state tracking, InfoNav prompts agents to generate and iteratively update a plan that enumerates the rationales (pieces of information) required to answer the task. Each rationale is marked as known or unknown, and agents use this plan to direct their queries and information sharing.

Figure 3: InfoNav in action—agents generate a plan, fill in unknowns through communication, and perform consensus reasoning.

This explicit plan-based approach generalizes dialogue state tracking and provides a robust scaffold for multi-turn, multi-agent communication, reducing redundant or irrelevant exchanges and improving the reliability of information flow.

Mixed Memory: Structured and Semantic Retrieval

To support accurate and comprehensive information retrieval, iAgents employ a Mixed Memory system comprising:

• Distinct Memory: Structured storage (e.g., SQL-like) for exact retrieval of factual information.
• Fuzzy Memory: Embedding-based, session-level summaries for semantic retrieval, supporting context-rich queries.

Agents dynamically select and adjust retrieval strategies based on communication progress, cross-verifying results from both memory types to enhance robustness.

InformativeBench: Benchmarking Collaboration under Asymmetry

The paper introduces InformativeBench, the first benchmark tailored to evaluate LLM agents' collaborative abilities under information asymmetry. It includes both "Needle-Type" tasks (locating specific information distributed across the network) and "Algorithm-Type" tasks (solving algorithmic problems with distributed inputs), spanning datasets of varying scale and complexity.

Figure 4: Two task types in InformativeBench—information seeking and distributed algorithmic reasoning.

Figure 5: Visualization of the FriendsTV dataset social network, illustrating large-scale, realistic agent interactions.

Figure 6: Distribution of question types in InformativeBench, emphasizing objective, ground-truth queries.

The FriendsTV dataset, in particular, simulates a large-scale social network with 140 individuals and 588 relationships, requiring agents to retrieve and reason over nearly 70,000 messages.

Experimental Results and Analysis

Main Results

iAgents, when instantiated with state-of-the-art LLMs (GPT-4, GPT-3.5, Claude Sonnet, Gemini), demonstrate the ability to autonomously communicate over 30 turns and complete complex tasks within 3 minutes in large social networks. However, even the strongest models achieve only moderate accuracy (e.g., GPT-4: 57.94% on FriendsTV, 22.8% on the hardest Schedule tasks), highlighting the persistent difficulty of collaboration under information asymmetry.

Ablation Studies

Figure 7: Ablation study on InfoNav, Mixed Memory, and Recursive Communication mechanisms.

• Mixed Memory yields a 2.38–6.34% performance gain in large-scale settings, surpassing InfoNav's 0.8% gain, indicating that retrieval is a greater bottleneck than reasoning in high-information environments.
• Recursive Communication (agents initiating new sub-dialogues) provides the largest boost (12.7%) in large networks, underscoring the importance of scalable, distributed communication.
• InfoNav is critical in smaller-scale, reasoning-intensive tasks, with 15–26% performance improvements, and is indispensable for multi-turn, logic-driven collaboration.

Agent Behavior Analysis

Figure 8: Distribution of agent behaviors in memory retrieval adjustment during communication.

• Agents predominantly maintain retrieval parameters, but when adjustments occur, they tend to increase retrieval scope, especially in challenging tasks.
• Successful agents exhibit higher rationale completion and consensus ratios, and fewer hallucinated ("fake solved") rationales, indicating more effective use of InfoNav and memory mechanisms.

Real-World Considerations: Prior Knowledge and Privacy

The study examines two practical concerns:

• Prior Knowledge Distraction: Anonymizing entities in FriendsTV reduces performance by ~3%, suggesting that agents rely primarily on provided information but are not immune to pretraining biases.
• Privacy Preservation: Enforcing privacy-aware output (vague references, minimal disclosure) incurs a ~5% performance drop, reflecting the trade-off between privacy and collaborative efficacy. The challenge of regulating input-side access remains open and is more an engineering than an algorithmic issue.

Implications and Future Directions

The iAgents paradigm advances the state of LLM-based MAS by explicitly modeling information asymmetry, a ubiquitous feature of real-world human collaboration. The explicit plan-based communication (InfoNav) and hybrid memory retrieval mechanisms provide a foundation for scalable, privacy-aware, and effective agent collaboration.

The moderate performance of current LLMs on InformativeBench, especially in large-scale, high-asymmetry settings, indicates substantial headroom for improvement. Key open challenges include:

• Improving retrieval and reasoning under extreme information fragmentation
• Developing lightweight, edge-deployable models for privacy-sensitive applications
• Designing robust HCI paradigms for human-agent-agent collaboration
• Mitigating pretraining bias and ensuring faithful, context-grounded reasoning

The InformativeBench benchmark provides a valuable testbed for future research in these directions.

Conclusion

This work establishes a rigorous framework and evaluation methodology for LLM-based agent collaboration under information asymmetry, moving beyond the shared-context assumption of prior MAS. The iAgents architecture, with its explicit communication planning and hybrid memory, demonstrates the feasibility and challenges of real-world agent-mediated human collaboration. The results highlight both the promise and the limitations of current LLMs, setting a clear agenda for future research in scalable, privacy-preserving, and effective multi-agent systems.