TaskGen: A Task-Based, Memory-Infused Agentic Framework using StrictJSON

Abstract: TaskGen is an open-sourced agentic framework which uses an Agent to solve an arbitrary task by breaking them down into subtasks. Each subtask is mapped to an Equipped Function or another Agent to execute. In order to reduce verbosity (and hence token usage), TaskGen uses StrictJSON that ensures JSON output from the LLM, along with additional features such as type checking and iterative error correction. Key to the philosophy of TaskGen is the management of information/memory on a need-to-know basis. We empirically evaluate TaskGen on various environments such as 40x40 dynamic maze navigation with changing obstacle locations (100% solve rate), TextWorld escape room solving with dense rewards and detailed goals (96% solve rate), web browsing (69% of actions successful), solving the MATH dataset (71% solve rate over 100 Level-5 problems), Retrieval Augmented Generation on NaturalQuestions dataset (F1 score of 47.03%)

• The paper introduces TaskGen, a framework that decomposes complex tasks into manageable subtasks using Equipped Functions and Inner Agents.
• It presents a novel methodology by integrating StrictJSON for concise LLM outputs and employing memory management through Shared Memory and a Memory Bank.
• The paper demonstrates practical success with high performance in dynamic maze navigation, TextWorld puzzles, and web browsing tasks.

TaskGen: A Task-Based, Memory-Infused Agentic Framework using StrictJSON

Overview

TaskGen is an open-source agentic framework designed to solve complex tasks by decomposing them into manageable subtasks. Each subtask is mapped to either an Equipped Function or another Agent. The framework employs StrictJSON to ensure concise JSON-based output from LLMs, reducing verbosity and token usage. This approach facilitates efficient execution across various agentic tasks while managing memory on a need-to-know basis, allowing agents to react dynamically to evolving environments.

Figure 1: An Overview of TaskGen.

Framework Architecture

Task Breakdown with Equipped Functions and Inner Agents

TaskGen operates by breaking a task into subtasks, each executed via Equipped Functions or Inner Agents. Equipped Functions limit the action space, making task execution tractable (Figure 2). For complex tasks, the hierarchical model allows a Meta Agent to delegate subtasks to Inner Agents, enhancing processing capabilities (Figure 3).

Figure 2: Constraining action space by Equipped Functions.

Figure 3: Inner Agents assigned as Equipped Functions to a Meta Agent helps increase processing capability.

Memory Management

Shared Memory is pivotal in TaskGen, allowing components to share crucial information only when necessary. There are two types of Shared Memory: Subtasks Completed and Shared Variables (Figure 4). TaskGen also employs a Memory Bank, storing abstracted memories used for Retrieval Augmented Generation (RAG) based on semantic similarity to tasks.

Figure 4: Two types of Shared Memory: Subtasks Completed and Shared Variables.

StrictJSON Output

StrictJSON is utilized over traditional JSON Schema for its minimal verbosity and efficiency, crucial for reducing token usage in LLM communication. It supports type checking and iterative error correction leading to superior performance (Figure 5).

Figure 5: More concise output using JSON as compared to Free Text using gpt-3.5-turbo on 12 Jul 2024.

Implementation Steps

To implement TaskGen, follow these steps:

1. Install Package: Run pip install taskgen-ai.
2. Define LLM: Use the format def LLM(user_prompt: str, system_prompt: str) -> str.
3. Define Agent: Create an Agent using agent = Agent(name, description, LLM = LLM).
4. Equip Functions: Assign Equipped Functions to the Agent with agent.assign_functions([fn_1, fn_2]).
5. Execute Tasks: Run tasks using agent.run(task).
6. Query Agent: Retrieve information with agent.reply_user(query).

Evaluation and Results

TaskGen has been tested in diverse environments showcasing adaptability:

• Dynamic Maze Navigation: 100% success rate in navigating 40x40 mazes with dynamic obstacles.
• TextWorld Escape Room: Achieved a 96% solve rate where dense rewards and goals were specified.
• Web Browsing: Successfully completed 69% of actions across various web tasks.
• MATH Dataset: Averaged 71% accuracy over Level-5 problems.
• RAG-based QA: Outperformed baseline LLM with an F1 score of 47.03%, precision of 40.75%, and recall of 55.59%.

Conclusion

TaskGen represents a significant advancement in agentic frameworks by integrating efficient task decomposition, strict JSON processing, and strategic memory use. Its practical applications extend to complex problem-solving environments such as dynamic navigation, interactive fiction, web browsing, and mathematical computation. Future developments will focus on enhancing memory management, integrating multi-modal support, and fostering collaborative agent systems. TaskGen is actively developed, with community contributions encouraged for continual improvement and expansion.