One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents (2512.20957v2)

Abstract: Locating the files and functions requiring modification in large open-source software (OSS) repositories is challenging due to their scale and structural complexity. Existing LLM-based methods typically treat this as a repository-level retrieval task and rely on multiple auxiliary tools, which overlook code execution logic and complicate model control. We propose RepoNavigator, an LLM agent equipped with a single execution-aware tool-jumping to the definition of an invoked symbol. This unified design reflects the actual flow of code execution while simplifying tool manipulation. RepoNavigator is trained end-to-end via Reinforcement Learning (RL) directly from a pretrained model, without any closed-source distillation. Experiments demonstrate that RL-trained RepoNavigator achieves state-of-the-art performance, with the 7B model outperforming 14B baselines, the 14B model surpassing 32B competitors, and even the 32B model exceeding closed-source models such as Claude-3.7. These results confirm that integrating a single, structurally grounded tool with RL training provides an efficient and scalable solution for repository-level issue localization.

• The paper introduces RepoNavigator, which replaces multi-tool orchestration with a single, execution-aware jump tool to improve code symbol localization.
• It employs Group Reference Policy Optimization (GRPO) with reinforcement learning to enhance localization accuracy, precision, and sample-F1 performance.
• Empirical evaluations on SWE-bench datasets demonstrate that deeper, RL-optimized tool invocations outperform larger models and traditional multi-tool approaches.

Reinforcement Learning for Repository-Level LLM Agents: A Formal Analysis of RepoNavigator

Problem Context and Motivation

Automating large-scale software engineering tasks with LLM agents confronts the fundamental barrier of repository scale, as codebases in real-world OSS projects vastly exceed LLM context windows. Existing LLM agent frameworks in this domain address issue localization by equipping models with multiple auxiliary tools for repository-level retrieval, symbolic search, and code abstraction graph navigation. However, such multi-tool interfaces suffer from brittle orchestration, compounding tool-call errors, and a fundamental mismatch with genuine code execution semantics, where symbol resolution is continuous and execution unfolds through explicit reference links rather than abstract, static entities. This work rigorously attacks these limitations by proposing RepoNavigator, a repository-level LLM agent trained with RL that employs a single, execution-aware “jump” tool, precisely mirroring runtime symbol resolution.

Figure 1: A LLM agent navigates a code repository using only the jump tool implemented via a language server, aligning actions with code execution semantics.

Unified Tooling: Formalization and Implementation

RepoNavigator dispenses with the prevalent multi-tool paradigm and adopts a single, structurally grounded jump mechanism for symbol definition retrieval. Drawing on the deterministic capabilities of Python language servers (e.g., Pyright), the jump tool, when given a symbol and its context (the calling file), deterministically resolves the symbol’s definition via static scope analysis, type inference, and import dependency graph traversal. Formally, for a symbol occurrence $s$ at position $\ell$ in file $f$, the tool computes $\mathcal{R}(s, \ell) \rightarrow \{(f_i, p_i)\}$, returning all valid source-level definitions.

This reduction in toolset fundamentally restricts the agent’s action and observation space, improving interpretability and aligning tool usage with both LLM pretraining bias (sequential dependencies) and real program structure. The agent’s interaction is characterized by an iterative reasoning–action loop: at each step, the policy alternates between free-form language reasoning and structured tool invocation, recursively gathering localized code context through the jump oracle and updating its internal evidence state until arrival at a termination criterion.

Figure 2: RepoNavigator’s rollout process interleaves natural language reasoning and jump tool invocations, trained end-to-end with RL to optimize code localization.

RL Optimization: End-to-End Joint Reasoning and Tool Mastery

RepoNavigator introduces an RL training regime, Group Reference Policy Optimization (GRPO), to optimize end-to-end repository-level localization. The agent is initialized from a pretrained LLM and directly finetuned with RL using verifiable rewards, with no closed-source teacher distillation. GRPO balances a policy-gradient objective (driven by DICE overlap between predicted and ground-truth locations) and a KL-penalty trust region to ensure stable improvements. Notably, the reward incorporates both localization accuracy and explicit tool-calling success, enforcing not only correct predictions but robust tool usage—an essential property given the high compounding error potential in long tool-action chains.

Ablation studies reveal that RL training with GRPO, as opposed to pure supervised or rejection-based strategies, yields superior policy improvement, especially as LLM initialization strength and data quality increase. Furthermore, experiments show that including a tool success component in the reward explicitly enhances final precision and sample-F1 metrics.

Figure 3: Ablation analysis demonstrates that GRPO outperforms RFT or hybrid approaches, and integrating tool-calling success as a reward signal boosts performance on Qwen2.5-7B-Instruct.

Empirical Results and Scaling Analysis

RepoNavigator’s impact is empirically evaluated on SWE-bench-Verified and SWE-bench-Pro, with detailed comparison against state-of-the-art localization agents. Crucially, reinforcement learning transforms model scalability: with RL training, the 7B RepoNavigator outperforms 14B param baselines, the 14B variant outperforms 32B baselines, and the 32B model surpasses proprietary closed-source competitors such as Claude-3.7 across both function-level and file-level metrics. RepoNavigator achieves leading sample-F1 and IoU, demonstrating strong locality precision with conservative hypothesis sets, and outperforms multi-tool and graph-based agentic approaches on both in-domain and post-dataset-release records.

Figure 4: Allowing more jump tool actions (turns) scales RepoNavigator’s performance, pre- and post-RL, confirming a robust scaling law for tool use capacity.

The scaling law of tool-calling is substantiated: increasing the permitted number of jump actions yields monotonically improved localization outcomes, highlighting the utility of deep, recursive symbol tracing when guided by RL-optimized policies rather than static, shallow retrieval.

Theoretical and Practical Implications

Analysis in this work establishes several critical properties of single-tool agent architectures:

• Action Space Reduction: Limiting the toolkit to an execution-aligned jump reduces the agent’s action surface, restricting error propagation and increasing the coverage-to-error ratio.
• Improved Tool-Calling Success: Given sequential tool invocation, overall success rate $P_{\text{succ}}(k) = \prod_{i=1}^k p_i$ drops multiplicatively with toolchain length, incentivizing solution architectures that minimize tool count while maximizing each tool’s expressiveness.
• Access Scope and Ground Truth Overlap: The access space spanned by exhaustive jump traversal closely matches the actual set of ground-truth locations, as every faulty site lies on some execution path from the entry point. Empirical access scope shows much higher IoU to labeled bug spans compared with whole-repository or multi-tool access.

  Figure 5: The access scope of the jump tool (blue) achieves much higher IoU with the groundtruth than the repository-wide scope (grey), confirming effective coverage.

Additionally, experimental modification (Table presented in the paper) shows that adding narrow, redundant tools (e.g., GetClass, GetFunc, GetStruc) to RepoNavigator does not improve localization, reinforcing the theoretical argument for high-capacity, low-cardinality toolsets in agent design.

Future Directions and Theoretical Advancement

RepoNavigator’s architecture, centered on RL-based tool-agent cooptimization and aligned with code execution structure, sets a reproducible standard for future LLM agents in software engineering. Because RepoNavigator’s RL loop operates independently from closed-source teacher models and does not require costly full-patch validation, it is theoretically extensible to many programming languages and diverse downstream maintenance tasks.

Potential future developments include automated toolset inference for arbitrary language servers, fine-grained RL reward engineering for downstream patch generation, and joint training of repair and localization phases within the same agentic architecture.

Conclusion

RepoNavigator formalizes and validates a new approach to repository-level LLM agents: replacing multi-tool orchestration with a single execution-aligned tool and optimizing reasoning via RL. The framework’s empirical superiority, simplicity, and theoretical soundness establish it as a robust solution for OSS issue localization, diminishing the error-prone complexity of previous approaches and illuminating the path toward more generalizable, interpretable, and performant LLM-based agents for software engineering (2512.20957).