Profile-Aware Maneuvering: A Dynamic Multi-Agent System for Robust GAIA Problem Solving by AWorld

Abstract: The rapid advancement of LLMs has empowered intelligent agents to leverage diverse external tools for solving complex real-world problems. However, this reliance introduces new challenges, as extended contexts and noisy tool outputs can undermine system reliability. To address this, we propose a dynamic Multi-Agent System (MAS) in our AWorld framework, where an Execution Agent is supervised by a Guard Agent that provides on-demand dynamic maneuvering, verifying and correcting the reasoning process to improve robustness over single-agent systems. To move beyond this generic supervision, we enhance the architecture with a methodology inspired by System Identification from control theory. This method first profiles the Execution Agent offline on a benchmark dataset to create a "performance fingerprint" of its unique weaknesses. The Guard Agent then leverages this fingerprint online to deliver profile-aware supervision, making targeted interventions based on known failure patterns rather than merely reacting to immediate logical flaws. Extensive experiments on the GAIA dataset demonstrate that this profile-aware MAS significantly improves both effectiveness and stability, outperforming not only single-agent systems but also its naive counterpart. This superior performance led our system to achieve first place among open-source projects on the prestigious GAIA leaderboard. These findings highlight that building truly trustworthy intelligent systems requires not just collaboration, but a deep, empirically-grounded understanding of each agent's unique capabilities and limitations.

• The paper demonstrates that a dynamic multi-agent system with profile-aware maneuvering improves GAIA problem-solving accuracy, with pass@1 reaching 67.89%.
• It employs a control strategy inspired by marine navigation, using Execution and Guard Agents for real-time error correction and logical convergence.
• The study highlights enhanced stability with reduced variability, paving the way for more robust and adaptive AI systems.

Profile-Aware Maneuvering: A Dynamic Multi-Agent System for Robust GAIA Problem Solving

Introduction

The paper "Profile-Aware Maneuvering: A Dynamic Multi-Agent System for Robust GAIA Problem Solving by AWorld" explores integrating a dynamic supervision and maneuvering framework into a multi-agent system (MAS) architecture. The focus is on enhancing the robustness and stability of intelligent systems as they increasingly rely on external tools. This work responds to challenges faced by agents due to extended contexts from disparate sources and noisy outputs, advocating for adaptive collaboration between agents to bolster system reliability and accuracy.

Figure 1: Performance on the GAIA benchmarks (partial) across systems: Building on Gemini 2.5 Pro, incorporating tools into a Single Agent System enhances performance but also introduces greater uncertainty. By comparison, the Dynamic Multi-Agent System delivers superior results while offering improved stability.

Methodology

Inspired by control theory principles from marine vessel navigation, the study introduces a dynamic maneuvering mechanism analogous to dynamic control in complex navigation environments. Here, an Execution Agent collaborates with a Guard Agent to correct reasoning deviations. This proactive correction parallels a vessel's rudder control, adapting to external forces for optimal navigation. The mechanism leverages a Guard Agent to verify and refine the logical reasoning processes, thus ensuring accurate and stable solution pathways.

The core MAS architecture is designed to dynamically engage agents based on task evolution, context analysis, and correct reasoning fidelity. The Execution Agent initiates tasks and invokes the Guard Agent as necessary for logical oversight, ensuring consistent decision-making throughout the process.

Figure 2: AWorld achieves 1st in GAIA test leaderboard.

Figure 3: The zig-zag test is a standard procedure in System Identification for marine vessels, designed to reveal the ship's unique maneuvering characteristics.

Experimental Setup

The experiments employ the GAIA test set, comprising a mix of Level 1 and Level 2 questions across office and search-related tasks. The tests compare base model performance, Single Agent System (SAS) with tools, and the proposed Multi-Agent System (MAS) integrating dynamic maneuvering. Each version undergoes three runs with performance evaluation focused on the pass@1 and pass@3 accuracy metrics. The MAS configuration demonstrates notable improvements in both accuracy and stability, exemplifying its efficacy over SAS.

Figure 4: Our hierarchical control architectures, built on the AWorld framework.

Results

The experimentation reveals significant accuracy improvements in the problem-solving process with dynamic agent collaboration. The MAS outperformed the base models and SAS, achieving higher pass@1 and pass@3 scores. Importantly, introducing the Guard Agent led to reduced standard deviation in results, indicative of enhanced system stability.

Numerically, the MAS recorded a pass@1 accuracy of 67.89% compared to 31.5% for the base model and 62.39% for SAS. The pass@3 metric also reflected this gain, with the MAS achieving an 83.49% accuracy, demonstrating the importance of dynamic supervision.

Analysis

The investigation highlights key insights into agent collaboration models:

1. Mode Optimization: Transitioning between internal knowledge and external tool reliance affects performance, necessitating improved self-aware switching mechanisms.
2. Logical Convergence: By employing context optimization and maneuver correction, the MAS mitigates lengthy context-related instability, promoting logical convergence through dynamic interaction.

These aspects underline how robust, adaptive systems are crucial for real-world application scenarios.

Future Work

Future development aims to:

• Enhance Guard Agent capabilities to independently call tools for higher cross-validation.
• Improve agent architecture for autonomous mode-switching, facilitating smarter decision-making in complex task environments.

These advances will further solidify AI systems' capabilities, providing greater flexibility and efficiency.

Conclusion

The paper provides a significant contribution to AI agent system design by proposing a dynamic multi-agent framework that enhances stability and effectiveness. The introduction of collaborative agents demonstrates improved performance benchmarks and promises for further advancements in adaptive technology. This work emphasizes the importance of synergistic agent roles in overcoming traditional limitations, paving the path for more resilient AI applications.