---
title: Repository-Level Code Generation (RLCG)
url: https://www.emergentmind.com/topics/repository-level-code-generation-rlcg
type: topic
---

# Repository-Level Code Generation (RLCG)

Searching arXiv for recent repository-level code generation papers to ground the article.
Repository-Level Code Generation (RLCG) is the problem of generating code that must operate correctly **inside an existing repository**, rather than as an isolated function or snippet. In this setting, a model must account for cross-file dependencies, repository-specific APIs, imports, module boundaries, coding conventions, and often tests or evolving project state. The literature consistently treats RLCG as qualitatively different from ordinary function-level generation: the central difficulty is not only token prediction, but **repository understanding**, context construction, and consistency with a pre-existing codebase [2510.04905][2505.14394][2601.00376].

## 1. Concept and scope

RLCG is commonly formulated as code generation conditioned on repository context. One representative formulation describes the task as generating the full body \(y\) of an unfinished target function \(x\) inside a repository \(\mathcal D\), using repository context \(\mathcal C\) extracted from the rest of the codebase [2601.00376]. Another formulation emphasizes that generation should be **functionally correct and repository-consistent**, meaning that outputs should respect existing APIs, reuse available abstractions, maintain stylistic and architectural consistency, and satisfy inter-file dependencies [2505.14394]. The broader survey literature places repository-level generation beyond function-level and file-level generation because relevant evidence is distributed across files, modules, tests, issues, and other repository artifacts rather than being locally available in a single prompt [2510.04905].

Several papers make the distinction concrete by specifying evaluation settings in which a target function or method body is removed and must be regenerated using repository context. In the EvoCodeBench setup used for knowledge-graph-based retrieval, the target function body is replaced by `pass`, a repository knowledge graph is built, a **two-hop subgraph** is retrieved around the target node, that subgraph is passed to the LLM as context, and the generated body is inserted back into the file before repository tests are executed to compute **pass@1** [2505.14394]. InlineCoder adopts a similar target-function setting, but frames the challenge as repository understanding through caller–callee structure rather than flat retrieval [2601.00376]. CatCoder, Hydra, RepoScope, and RepoCoder also treat repository-level generation as function completion inside a real repository, with correctness depending on cross-file context rather than only in-file text [2406.03283][2602.11671][2507.14791][2303.12570].

A broader, more demanding interpretation appears in work on feature implementation and whole-repository synthesis. FEA-Bench evaluates **incremental feature implementation** in existing repositories, where a model must both create new components and edit other files to integrate them [2503.06680]. WebDesignIter studies iterative front-end development over sequential tasks in evolving repositories and introduces persistent design memory to preserve consistency across development cycles [2607.10621]. CodeTeam goes further to **natural language to repository generation (NL2Repo)**, where the input is a README-like requirements document and the output is an entire repository with a file tree, interfaces, dependencies, and implementation across files [2606.22082]. This suggests that “RLCG” spans a spectrum from repository-grounded function completion to multi-file feature implementation and full repository synthesis.

A recurring misconception in the literature is that RLCG is merely function-level generation with a longer prompt. Multiple papers explicitly reject that view. Knowledge-graph-based retrieval argues that LLMs often generate code in isolation and therefore miss project-specific architecture and dependency structure [2505.14394]. The survey literature similarly argues that repository code should not be treated as a bag of semantically similar text spans, because useful context is structured around symbols, files, imports, containment, and dependency relations [2510.04905][2602.11671].

## 2. Benchmarking and task formulations

RLCG research is strongly shaped by benchmark design. Existing benchmarks differ in whether they evaluate completion, dependency use, incremental editing, or full repository construction.

| Benchmark | Task setting | Notable properties |
|---|---|---|
| EvoCodeBench | Repository-level function/method completion | 275 samples from 25 open-source repositories; Python-only [2505.14394] |
| DevEval | Repository-level function generation | 1,825 Python samples from 115 repositories across 10 domains [2601.00376] |
| RepoExec | Context-controlled repository-level function generation | 355 Python tasks; execution-based; introduces DIR [2406.11927] |
| RepoEval | Repository-level completion | Line, API invocation, and function body completion on recent Python repositories [2303.12570] |
| Web-Bench | Sequential front-end repository generation | 50 projects, each with 20 sequential tasks [2607.10621] |
| FEA-Bench | Incremental feature implementation | 1,401 task instances from 83 Python repositories [2503.06680] |
| RepoClassBench | Natural-language-to-class generation in repositories | 130 Java classes and 97 Python classes from real repositories [2405.01573] |
| NL2Repo-Bench | Natural language to repository generation | 104 tasks across nine categories of Python libraries [2606.22082] |

RepoExec is notable because it isolates the **generation model’s ability to use provided dependencies** rather than the model’s own retrieval ability. It supplies curated dependency context and evaluates executability, functional correctness, and dependency use through **Dependency Invocation Rate (DIR)**, defined as
\[
\text{DIR} = \dfrac{|D_g \cap D_s|}{|D_s|}.
\]
Here \(D_g\) is the set of identifiers in the generated output and \(D_s\) is the set of provided dependencies extracted from the solution [2406.11927]. This benchmark therefore measures not only whether a function works, but whether it uses the repository abstractions that a developer intended.

RepoEval, by contrast, was designed for repository-level completion through line completion, API invocation completion, and function body completion, using repositories created after January 1, 2022 to reduce leakage and including **1,600** line completions, **1,600** API invocations, and **373** function body completions [2303.12570]. RepoClassBench moves the unit of generation upward to the class level and requires the generated class to compile or run in situ and satisfy repository tests [2405.01573]. FEA-Bench emphasizes **feature addition PRs** rather than completion or repair; its tasks require both synthesis of new components and edits to existing files, with average edited files **2.62**, average edited lines **128.5**, average lines of added components **87.1**, average added functions **4.49**, and average added classes **0.78** [2503.06680].

The benchmark landscape implies that RLCG cannot be reduced to a single evaluation protocol. Completion-oriented tasks test whether a model can fill repository-dependent holes. Feature benchmarks test whether it can evolve codebases. NL2Repo tasks test whether it can create a repository architecture from requirements. The survey literature treats this heterogeneity as one of the field’s defining characteristics [2510.04905].

## 3. Context construction and retrieval paradigms

Most RLCG systems are retrieval-augmented, but they differ sharply in what they retrieve and how repository structure is represented. The literature broadly divides approaches into flat retrieval, structure-aware retrieval, and graph-based retrieval [2510.04905].

A foundational flat retrieval method is RepoCoder, which builds a repository snippet database by sliding-window chunking and retrieves code with a similarity-based retriever in an **iterative retrieval-generation pipeline**. Vanilla retrieval-augmented generation retrieves once with the unfinished code \(X\) and generates \(\hat{Y}=\mathcal{M}(C_{ret},X)\). RepoCoder instead refines retrieval using the previous prediction \(\hat{Y}^{i-1}\), so that later iterations retrieve with a query formed from the unfinished code and the beginning of the provisional completion [2303.12570]. The paper reports that RepoCoder improves the In-File completion baseline by **over 10% in all settings** and consistently outperforms one-shot retrieval-augmented completion [2303.12570]. This suggests that the retrieval query itself is a central bottleneck in RLCG.

A different line of work argues that retrieval should follow **program structure** rather than chunked text. Hydra states that chunking-based indexing loses coherence between code units and overlooks structural relationships. It therefore indexes repositories as hierarchical code units—functions, classes, and variables—and adds a **Dependency-Aware Retriever (DAR)** over the current file and imported files [2602.11671]. On DevEval with Qwen2.5-Coder-7B, Pass@1 rises from **8.46** under chunking to **16.67** under structure-aware indexing; on RepoExec it rises from **14.03** to **20.79** [2602.11671]. Hydra further reports that similarity-only retrieval misses **40–60% of ground-truth dependencies**, and that DAR plus BM25 yields higher Pass@1 than sparse-only, dense-only, or dependency-only retrieval [2602.11671]. This directly supports the claim that repository-level retrieval must surface **true dependencies**, not merely similar snippets.

Graph-based approaches encode repository structure explicitly. “Knowledge Graph Based Repository-Level Code Generation” represents each repository as a graph over node types
\[
V = \{\text{File}, \text{Class}, \text{Method}, \text{Function}, \text{Attribute}, \text{Generated Description}\}
\]
and relation types
\[
R = \{\text{defines class}, \text{defines a function}, \text{has a method}, \text{used in}, \text{has an attribute}, \text{has description}\}.
\]
Repositories are parsed file by file using ASTs, ingested into Neo4j, indexed with full-text indexes over function names, class names, method names, and modules, and vector indexes over documentation and LLM-generated descriptions using **all-MiniLM-L6-v2** [2505.14394]. Retrieval combines LLM-assisted entity extraction, full-text search, vector search, graph expansion, and post-expansion semantic filtering, then passes a refined subgraph to the generator [2505.14394]. On EvoCodeBench, the proposed graph-based retrieval achieves **32.00%** pass@1 with GPT-4, **33.45%** with GPT-4o, and **36.36%** with Claude 3.5 Sonnet, compared to **20.73%** for Local File Infilling with GPT-4 and **7.27%** for Without Context with GPT-4 [2505.14394].

RepoScope also uses structural semantics, but with a different graph and retrieval design. It constructs a **Repository Structural Semantic Graph (RSSG)** over classes, functions, and attributes, with views spanning `Contains`, `Returns`, `As Parameter`, `Inherits`, `Calls`, and `Imports`, formally
\[
G = \{(h, r, t) \mid h, t \in \mathcal{E},\ r \in \mathcal{R}_{ST} \cup \mathcal{R}_C \cup \mathcal{R}_I\}.
\]
It retrieves a **four-view context**: callers of the target function, predicted call chains, similar functions, and similar code fragments [2507.14791]. The call-chain predictor combines embedding similarity, cluster-level call evidence, and DFS over structural/type edges, with entity scoring
\[
S_e = \alpha_1 \cdot \text{sim}(V_f, V_e) + \alpha_2 \cdot \phi\left(\sum \alpha_3 w_r\right),
\]
where the implementation uses \(\alpha_1=1.0\), \(\alpha_2=2.0\), \(\alpha_3=2.0\), and \(\phi(x)=\log_2(x+1)\) [2507.14791]. RepoScope reports predicted callee **F1 = 0.603** versus **0.4352** for similarity-only prediction, a **38.56%** improvement, and achieves the best pass@1 across all four backbone models on both CoderEval and DevEval [2507.14791].

A more specialized structural approach is InlineCoder, which treats repository semantics as fundamentally **call-graph-defined**. It first generates a draft completion \(R\), computes a perplexity-based confidence
\[
\mathrm{PPL}(R \mid B) = \exp\left( -\frac{1}{M} \sum_{j=1}^{M} \log p\bigl(r_j \mid B, r_{<j}\bigr) \right),
\]
then uses **Upstream Inlining** to inline the draft into callers and **Downstream Retrieval** to recover likely callees [2601.00376]. The inlining transformation is formalized with parameter substitution \(\sigma\), return normalization \(\tau\), and transformed callee body
\[
\text{Body}_f^{*} = \tau\big(\sigma(\text{Body}_f)\big).
\]
This reframes repository understanding as a function-local prompt construction problem [2601.00376]. On RepoExec, InlineCoder improves over the strongest baseline by **29.73% in EM**, **20.82% in ES**, and **49.34% in BLEU** on average across models [2601.00376].

In statically typed settings, CatCoder argues that type structure is itself a retrieval modality. It combines hybrid code retrieval with **type context extraction** over a type dependency graph \(G=(V,E)\), retrieving fields and method signatures of relevant types and concatenating those with code context [2406.03283]. On **199 Java** and **90 Rust** tasks, CatCoder outperforms RepoCoder by up to **17.35%** in pass@k [2406.03283]. This suggests that in some languages repository context is partly a type-navigation problem rather than only a code-search problem.

## 4. Planning, agents, and persistent repository memory

A second major trend in RLCG is the move from one-shot retrieval-plus-generation to **planning**, **agentic decomposition**, and **persistent repository memory**.

WebDesignIter argues that the missing ingredient in iterative front-end repository generation is **design knowledge**: architectural principles, module responsibilities, and structural constraints. Its persistent knowledge graph, **WebAppArchKG**, stores repository structure, file nodes with design summaries, block-level AST units, cross-file and inter-block dependency relations, historical design records, architectural overview information, and failure feedback [2607.10621]. The pipeline consists of four stages named **Architectural Knowledge Extraction and Evolution**, **Design-informed Planning**, **Design-aware Generation**, and **Repo-Verification** [2607.10621]. Planning retrieves “Design Info” from the graph—Historical Design, Repo Structure, and ordered file-level Overview Design—then produces an Implementation Plan and Test Scripts; generation executes the plan through targeted diff patches; verification runs syntax repair and sandbox execution; successful changes update the graph, while failures are recorded for later planning [2607.10621]. On Web-Bench, WebDesignIter improves average Pass@1 by **7.98 percentage points** and average Pass@2 by **9.55 percentage points** over Web-Agent, and removing Design causes an **11.40** percentage-point drop in Pass@1 [2607.10621]. The paper interprets this as evidence that repository-level generation benefits from persistent architectural memory rather than only code retrieval.

TICoder brings test information into the planning loop. It first generates implementation steps
\[
\mathbf{s} = \{s_{1}, \cdots, s_{n_{s}}\},
\]
then retrieves callee functions and usage patterns before final generation
\[
Code=Generate(\mathbf{fc}, \mathbf{fp}, t, r).
\]
Its planner is conditioned on both requirement \(r\) and tests \(t\), and a judge scores plans along **Functional Coverage and Consistency**, **Independence**, and **Granularity**, each on a 0–100 scale, refining until scores exceed **90** or the maximum iteration count is reached [2606.08135]. Retrieval itself is “implementation-aware”: for each step \(s_i\), TICoder generates a step-aware code representation \(c_i\), computes functional similarity \(Sim_F\) to function descriptions and implementation similarity \(Sim_I\) to function bodies, then combines them by
\[
Sim(i, j) = \alpha\times Sim_F(i, j)+\beta\times Sim_I(i, j),
\]
with \(\alpha+\beta=1\) [2606.08135]. It further selects usage patterns with structure-based clustering over the repository call graph and perplexity-based filtering using
\[
\operatorname{PPL}(r \mid c) = \exp\left(-\frac{1}{n_r} \sum_{i=1}^{n_r} \log P(r_i \mid r_{< i}, c)\right).
\]
On DevEval with GPT-4o-mini, full TICoder reaches **31.01** Pass@1; removing planning drops it to **28.88**, removing tests in generation drops it to **27.78**, and replacing the original requirement with implementation steps collapses performance to **14.63** [2606.08135]. This suggests that tests help both as behavioral specifications and as generation-time constraints, while plan artifacts are not a substitute for the original task description.

ProjAgent introduces a different planning/retrieval concept: **procedural similarity**. Rather than retrieving code by lexical, semantic, or graph similarity alone, it decomposes the target function into logical steps, projects step representations into a reasoning-oriented subspace,
\[
\text{proj}_R(h_l) = V_R^\top \cdot h_l,
\]
applies PCA-based debiasing,
\[
\tilde{h} = (\text{proj}_R(h_l) - \mu) - \left((\text{proj}_R(h_l) - \mu)^\top \cdot \text{PC}_1\right)\text{PC}_1,
\]
and retrieves repository functions that implement the **same computational operation regardless of application domain** [2607.08691]. Procedural retrieval is paired with semantic retrieval over symbols accessible from the target file, scored by
\[
v = 0.5 \cdot \widehat{\text{BM25}(q,s)} + 0.5 \cdot \widehat{\text{dense}(q,s)}.
\]
On REPOCOD, ProjAgent reaches **41.14% Pass@1**, outperforming Sparse (**26.58%**), Dense (**28.83%**), Same\_File (**14.98%**), and SpecAgent (**34.52%**) [2607.08691]. Its ablation shows that removing procedural retrieval drops Pass@1 to **25.76%**, removing semantic retrieval drops it to **32.29%**, and removing the feedback loop drops it only slightly to **40.29%** [2607.08691]. This indicates that in that setup retrieval design, especially procedural retrieval, dominates repair.

At the most expansive end of the spectrum, CodeTeam treats repository-level generation as **multi-agent software engineering**. Architect agents propose SDS designs, a CTO agent scores them on structural validity, interface consistency, implementability, and appropriateness of developer count and ownership, then normalizes the winning design into a machine-checkable contract [2606.22082]. Developer agents implement files under dependency-aware scheduling with
\[
\mathsf{prio}(f)=(\mathsf{depth}(f), \mathsf{fanout}(f), -\mathsf{order}(f)),
\]
Git-based coordination, bounded per-file context, and QA-driven repair [2606.22082]. On SketchEval, CodeTeam improves over CodeS from **47.6 ± 2.2** to **51.7 ± 1.8** in the prompt-engineering setting and from **58.0 ± 1.6** to **60.9 ± 1.4** in the SFT setting [2606.22082]. On NL2Repo-Bench, it reaches the highest average pass rate in both settings, **34.6%** PE and **42.3%** SFT [2606.22082]. The paper’s ablation attributes **9.9%** relative improvement to project-specific developer allocation and **8.1%** to retrieval-augmented planning [2606.22082].

## 5. Evaluation outcomes and empirical patterns

Across benchmarks, the most stable empirical pattern is that **repository context substantially improves correctness**, but the type and organization of that context matter.

Knowledge-graph retrieval improves GPT-4 on EvoCodeBench from **7.27%** pass@1 in the Without Context setting to **32.00%** with graph-based retrieval [2505.14394]. Hydra reports that on RepoExec with GPT-4.1 mini, Pass@1 rises from **21.58** with No Context to **43.55** with Hydra; on DevEval it rises from **19.72** to **31.91** [2602.11671]. RepoScope achieves the best pass@1 across all four backbone models on both CoderEval and DevEval, with the paper’s headline being a **36.35%** relative improvement on DevEval with Claude-3.5-Haiku, from **30.48** to **41.56** [2507.14791]. InlineCoder reports especially large relative gains on the harder RepoExec benchmark, where its average improvements over the strongest baseline are **29.73% in EM**, **20.82% in ES**, and **49.34% in BLEU** [2601.00376]. CatCoder shows that adding type context yields consistent additive benefit over retrieval-only systems in Java and Rust [2406.03283].

The literature also shows that **context-free or local-only baselines are usually far behind repository-aware ones**, but “more context” is not automatically better. In A\(^3\)-CodGen, top-\(k\) global retrieval performs best at **\(k=5\)**, with precision **0.542**, recall **0.703**, F1 **0.612**, and accuracy **0.851**, while performance drops at \(k=10\) and \(k=15\) [2312.05772]. CatCoder observes that full code-plus-type context is best, but certain partial-context configurations are worse than more compact ones [2406.03283]. RepoExec shows that increasing BM25 prompt size from **27K** to **40K** slightly increases retrieval recall but does not improve model performance, and can reduce it [2406.11927]. FEA-Bench likewise reports that longer-context access is not enough to solve incremental feature implementation: even the best model, **DeepSeek-R1**, resolves only **9.92%** of tasks on the full benchmark under the best prompt setting [2503.06680].

Another repeated pattern is that **retrieval quality and context utilization are distinct bottlenecks**. RepoExec reports that pretrained models are stronger on raw correctness, while instruction-tuned models are stronger on dependency use and debugging; dependency-focused fine-tuning yields modest pass@1 gains but much larger gains in DIR [2406.11927]. ToolGen makes the same point from the decoding side: integrating autocompletion tools into generation improves **Dependency Coverage** by **31.4% to 39.1%** and **Static Validity Rate** by **44.9% to 57.7%** across three LLMs, while maintaining competitive BLEU-4, CodeBLEU, Edit Similarity, and Exact Match [2401.06391]. RepoGenReflex adds a verbal-feedback loop to retrieval-augmented completion and reports consistent but modest EM and ES improvements over standalone code models; its main ablation shows that removing Reflector and Experience causes the largest degradation [2409.13122]. This suggests that repository-aware generation is partly a retrieval problem, partly a decoding-and-repair problem, and partly a prompt/control problem.

The strongest negative result in the literature is that **real repository evolution remains very hard**. FEA-Bench shows low resolved ratios even for frontier models, with **DeepSeek-R1 at 9.92%** on the full benchmark and performance dropping sharply as the number of added functions increases: **18.96%** when one function is added, **8.24%** for two functions, and **5.47%** for three or more [2503.06680]. RepoClassBench likewise finds that current models struggle with natural-language-to-class generation inside repositories, and that retrieval-only baselines depend heavily on finding structurally similar sibling classes; tool-mediated RRR improves over Repocoder, but failures remain dominated by reasoning errors and functional ambiguity [2405.01573].

## 6. Limitations, controversies, and research directions

RLCG research is unified less by a single best method than by a common set of unresolved limitations.

A first limitation is **under-specification or incompleteness of structural modeling**. The knowledge-graph approach over files, classes, methods, functions, attributes, and generated descriptions is effective, but its authors explicitly note that it does not model richer semantics such as explicit call graphs, inheritance hierarchies beyond simple containment, import alias resolution, type information, test dependencies, version constraints, or build/runtime configuration [2505.14394]. Hydra’s DAR narrows candidates to the current file and imported files, which is efficient, but also a deliberate restriction rather than full-repository dependency recovery [2602.11671]. RepoScope relies only on static analysis and therefore inherits the limits of static callability approximation in Python [2507.14791]. This suggests that current graph- and structure-aware methods remain partial approximations to full repository semantics.

A second limitation is **evaluation realism**. RepoGenReflex uses Exact Match and Edit Similarity inside its iterative loop, which is natural in offline benchmarking but unrealistic for deployment because the true completion is unknown in real IDE use [2409.13122]. RepoExec improves evaluation by emphasizing executability, functional correctness, and dependency use, but it only includes **directly called dependencies** and remains Python-only [2406.11927]. FEA-Bench focuses on feature implementation but is also Python-only and uses single-round generation for cost reasons [2503.06680]. CodeTeam addresses full repository generation but on Python-only benchmarks and under a controlled document-only setting [2606.22082]. The survey literature repeatedly highlights benchmark fragmentation, Python bias, limited multilinguality, and insufficient coverage of evolving repositories as open problems [2510.04905].

A third limitation is **scalability and latency**. Knowledge-graph retrieval explicitly flags subgraph retrieval and filtering as computationally intensive for large repositories [2505.14394]. WebDesignIter requires AST parsing, graph maintenance, block-level extraction, dependency inference, and periodic rule-based refactoring [2607.10621]. RepoGenReflex adds iterative retrieval plus separate Actor and Reflector model calls without giving a full complexity analysis [2409.13122]. CatCoder notes noticeable latency from retrieval and static analyzers [2406.03283]. RepoScope is relatively efficient, with average input tokens around **3679** and retrieval latency lower than multi-round baselines, but it still requires RSSG construction via Pytype and Tree-sitter [2507.14791]. Practical RLCG systems therefore still face a three-way trade-off among structure, adaptivity, and cost.

A fourth limitation is **the gap between structural integration and deep logic correctness**. Hydra reports that NameError, TypeError, and AttributeError decrease, while AssertionError becomes relatively more prominent, implying that retrieved dependencies help code get far enough to fail on semantics rather than symbol resolution [2602.11671]. WebDesignIter reduces unresolved Regression errors from **29.17%** to **4.17%**, Dependency errors from **25.00%** to **16.67%**, and Cross-File errors from **12.50%** to **10.42%**, but Function errors rise from **18.75%** to **39.58%** [2607.10621]. The case is similar in CodeTeam, where packaging and import/module-resolution failures drop, leaving more logic errors among the remaining failures [2606.22082]. A plausible implication is that current RLCG systems are increasingly able to preserve repository structure and dependencies, but still struggle with the deeper reasoning needed for behaviorally correct implementations.

A fifth limitation is **uncertainty about what the right context representation should be**. Flat retrieval, type-aware retrieval, call-graph inlining, knowledge graphs, persistent architectural memory, procedural similarity, autocompletion-guided decoding, and multi-agent planning all yield gains on some benchmarks [2601.00376][2406.03283][2505.14394][2607.10621][2607.08691][2401.06391]. The survey literature therefore does not converge on a single dominant paradigm; instead, it organizes the field around retrieval modality, graph use, training paradigm, and evaluation protocol [2510.04905]. This suggests that RLCG remains a systems-design problem with multiple viable decompositions rather than a settled modeling problem.

The most consistent forward-looking theme is that repository-level generation likely requires a combination of: **structured repository representations**, **selective retrieval**, **planning or decomposition**, **bounded-context generation**, **tool- or test-backed verification**, and **evaluation that measures both correctness and repository-faithful reuse** [2510.04905][2606.08135][2406.11927]. The field’s strongest recent results support that direction, but the broader benchmark evidence indicates that reliable repository-scale software generation remains far from solved.

Source: https://www.emergentmind.com/topics/repository-level-code-generation-rlcg