RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving (2505.21577v2)

Abstract: The ultimate goal of code agents is to solve complex tasks autonomously. Although LLMs have made substantial progress in code generation, real-world tasks typically demand full-fledged code repositories rather than simple scripts. Building such repositories from scratch remains a major challenge. Fortunately, GitHub hosts a vast, evolving collection of open-source repositories, which developers frequently reuse as modular components for complex tasks. Yet, existing frameworks like OpenHands and SWE-Agent still struggle to effectively leverage these valuable resources. Relying solely on README files provides insufficient guidance, and deeper exploration reveals two core obstacles: overwhelming information and tangled dependencies of repositories, both constrained by the limited context windows of current LLMs. To tackle these issues, we propose RepoMaster, an autonomous agent framework designed to explore and reuse GitHub repositories for solving complex tasks. For efficient understanding, RepoMaster constructs function-call graphs, module-dependency graphs, and hierarchical code trees to identify essential components, providing only identified core elements to the LLMs rather than the entire repository. During autonomous execution, it progressively explores related components using our exploration tools and prunes information to optimize context usage. Evaluated on the adjusted MLE-bench, RepoMaster achieves a 110% relative boost in valid submissions over the strongest baseline OpenHands. On our newly released GitTaskBench, RepoMaster lifts the task-pass rate from 24.1% to 62.9% while reducing token usage by 95%. Our code and demonstration materials are publicly available at https://github.com/wanghuacan/RepoMaster.

Review of "RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving"

The paper presents RepoMaster, a novel framework designed to enhance the autonomous utilization of GitHub repositories for complex task solving by code agents. This framework addresses key challenges in repository exploration — primarily the overwhelming information and tangled dependencies found in repositories that hinder efficient exploration by existing LLMs. The objective is to leverage the vast amounts of modular components available on GitHub to facilitate the autonomous solving of real-world complex tasks.

Core Methodology

RepoMaster introduces a comprehensive approach that combines structural mapping and dynamic exploration to optimize the use of GitHub repositories:

1. Hierarchical Code Structure Analysis: The system maps the repository structure into hierarchical code trees, function-call graphs, and module-dependency graphs. This allows for a deterministic understanding of the repository’s structure, thereby assisting agents in identifying core components essential for task completion.
2. Core Component Identification: Prioritizing essential modules and classes, RepoMaster allocates importance scores based on a variety of features including dependency, complexity, module usage, semantic value, documentation richness, and GitHub-specific metrics. This ensures that only critical pieces of code are processed for context, aiding in efficient problem-solving.
3. Autonomous Exploration and Execution: Throughout interaction turns, RepoMaster dynamically traverses the codebase using specialized exploration tools that afford detailed inspection, dependency analysis, and strategic file searches. This mimics a human-like approach to sorting relevant data amidst noise and magnifies the agent’s comprehension of intricate code repositories.
4. Context-aware Information Selection: RepoMaster incorporates multi-level content reduction strategies to efficiently manage LLM context windows, thereby maintaining an effective and actionable focus on relevant data without overwhelming the model’s processing limits.

Empirical Validation

RepoMaster’s efficacy was rigorously evaluated using both the adjusted MLE-Bench and the newly introduced GitTaskBench benchmarks. The results indicate a dramatic improvement over existing solutions:

• MLE-Bench: RepoMaster delivered a 110% relative increase in valid submissions compared to the strongest baseline. The medal acquisition rate demonstrates superior performance, showcasing RepoMaster’s capability to handle an extensive range of machine learning tasks.
• GitTaskBench: Across tasks in diverse domains such as image processing and office automation, RepoMaster enhanced task pass rates from 24.1% to 62.9% while reducing token use by 95%. This testifies to its efficiency in using repository components for practical problems without unnecessary token consumption.

Implications and Future Directions

The research offers meaningful implications for the future development of AI-driven code exploration frameworks:

• Practical Applications: The ability to automate complex task solving using existing repositories could significantly reduce barriers to software development and augment iterative software improvement processes across industries.
• Theoretical Advancements: By establishing robust mechanisms for efficient code understanding and execution, RepoMaster paves the way for deeper integration of LLMs with diverse information repositories, optimizing AI problem-solving at scale.

Future research may build on these foundations by developing more refined tools for dependency analysis and enhancing adaptability across larger and more diverse codebases. Inquiry into optimizing LLM-context utilization could also refine the information-thrift strategies employed by RepoMaster.

In conclusion, RepoMaster represents a thoughtful advance in autonomous code utilization, demonstrating the potential to transform how code agents interact with and leverage vast open-source repositories to achieve complex, real-world tasks autonomously. Its impact is likely to resonate both in AI research and practical applications within software engineering domains.