LLM-Agent-UMF: LLM-based Agent Unified Modeling Framework for Seamless Design of Multi Active/Passive Core-Agent Architectures

Abstract: In an era where vast amounts of data are collected and processed from diverse sources, there is a growing demand for sophisticated AI systems capable of intelligently fusing and analyzing this information. To address these challenges, researchers have turned towards integrating tools into LLM-powered agents to enhance the overall information fusion process. However, the conjunction of these technologies and the proposed enhancements in several state-of-the-art works followed a non-unified software architecture, resulting in a lack of modularity and terminological inconsistencies among researchers. To address these issues, we propose a novel LLM-based Agent Unified Modeling Framework (LLM-Agent-UMF) that establishes a clear foundation for agent development from both functional and software architectural perspectives, developed and evaluated using the Architecture Tradeoff and Risk Analysis Framework (ATRAF). Our framework clearly distinguishes between the different components of an LLM-based agent, setting LLMs and tools apart from a new element, the core-agent, which plays the role of central coordinator. This pivotal entity comprises five modules: planning, memory, profile, action, and security -- the latter often neglected in previous works. By classifying core-agents into passive and active types based on their authoritative natures, we propose various multi-core agent architectures that combine unique characteristics of distinctive agents to tackle complex tasks more efficiently. We evaluate our framework by applying it to thirteen state-of-the-art agents, thereby demonstrating its alignment with their functionalities and clarifying overlooked architectural aspects. Moreover, we thoroughly assess five architecture variants of our framework by designing new agent architectures that combine characteristics of state-of-the-art agents to address specific goals. ...

• The paper presents a unified framework integrating planning, memory, profile, action, and security modules as core-agent components.
• It leverages modular design to differentiate active and passive core-agents, optimizing multi-agent coordination and task execution.
• The framework offers scalability improvements and addresses security challenges, enhancing the robustness of LLM-based systems.

LLM-Agent-UMF: A Unified Modeling Framework for LLM-Based Agents

This paper presents the LLM-Agent Unified Modeling Framework (LLM-Agent-UMF), which provides a structured approach to designing LLM-based agents. Central to this framework is the introduction of the "core-agent" concept, which serves as the hub for coordinating the activities of LLMs, tools, and other functional components within LLM-based agents. The paper aims to address the lack of modularity and architectural clarity in existing LLM-agent designs by establishing a foundation for distinguishing the diverse modules and interactions necessary for developing robust, scalable, and secure AI systems.

Core-Agent Structure

The framework defines the core-agent as the pivotal component in LLM-based agents, tasked with orchestrating interactions among LLMs and external tools. Its internal architecture is modular, comprising five critical modules: planning, memory, profile, action, and security.

Planning Module

The planning module facilitates task decomposition and plan generation, utilizing strategies such as single-path or multi-path planning, and techniques that can be either rule-based or powered by LLMs. It ensures the agent can tackle complex tasks by breaking them down into manageable subtasks, utilizing iterative decomposition methods such as DARA [141].

Figure 1: Planning module functional perspectives.

Memory Module

Memory management, integral to the core-agent structure, is categorized by scope (short-term and long-term), location (embedded or extension), and format (natural language, embeddings, databases, or structured lists) [104]. This modularity supports efficient data retrieval and storage, essential for the agent's decision-making processes.

Figure 2: Memory module functional perspectives.

Profile Module

The profile module dynamically configures the LLM's persona using methods like Handcrafted In-Context Learning, LLM-Generation, Dataset Alignment, and Fine-tuned Pluggable Modules. This adaptability enables the agent to assume varied roles tailored to specific tasks [163].

Figure 3: Profile module techniques.

Action Module

The action module converts plans into executable actions, leveraging external tools and resources. It operates based on triggers such as plan following or API call requests, effectively utilizing available tools [21].

Figure 4: Action module functional perspectives.

Security Module

Addressing the critical need for robust security measures, the security module implements layered defenses to protect against threats like prompt injections and data breaches using safeguarding mechanisms aligned with the C.I.A. triad. It employs guardrails that are rule-based or powered by LLMs [112].

Figure 5: Security module functional perspectives.

Active/Passive Core-Agent Classification

The paper introduces a taxonomy distinguishing core-agents as either active or passive, based on their authoritative nature. Active core-agents encompass all five modules, facilitating complex task management and coordination, whereas passive core-agents are limited to action and security modules, focusing on specific task execution without reasoning capabilities [108].

Figure 6: The core-agent as the central component of LLM-based agents.

Multi Active/Passive Core-Agent Architectures

The framework proposes both uniform and hybrid multi-core agent architectures to enhance scalability and flexibility. The optimal configuration, the one-active-many-passive architecture, balances the managerial capabilities of a single active core-agent with the simplicity and efficiency of multiple passive core-agents, addressing the complexity challenges of multi-agent systems [40].

Figure 7: One-active-many-passive core-agent architecture.

Evaluation and Future Directions

The application of LLM-Agent-UMF to existing agents reveals common limitations, such as the absence of security measures in many tool-utilizing agents [135]. By leveraging this framework, developers can identify opportunities for integrating and enhancing functionalities, as demonstrated by scenarios like combining Toolformer and Confucius in a multi-passive setup or augmenting ToolLLM with ChatDB's symbolic memory capabilities.

Figure 8: Framework proposed by survey [wang2024].

In conclusion, LLM-Agent-UMF offers a robust foundation for designing modular, scalable, and secure LLM-based agents, facilitating improved terminological consistency and architectural clarity. Future research should focus on simplifying multi-active core-agent implementations and addressing identified vulnerabilities to further enhance LLM-agent systems.